Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, DeepSeek, Qwen, Baichuan, Mixtral, Gemma, Phi, etc.) on Intel CPU and GPU (e.g., local PC with iGPU, discrete GPU such as Arc…

Code for studying the super weight in LLM

A high-throughput and memory-efficient inference and serving engine for LLMs

BigDL: Distributed TensorFlow, Keras and PyTorch on Apache Spark/Flink & Ray

An open source framework that provides a simple, universal API for building distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyp…

Embed Python in Java