inference

Microsoft BitNet 1.58: The Era of 1-Bit Large Language Models

TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate

SGLang: Structured Generation Language for Efficient LLM Serving

vLLM: High-Throughput LLM Inference at Scale

Ollama: Run Local LLMs on Your Own Hardware