model serving

MLflow: The Open Platform for the Machine Learning Lifecycle

Ray: The Distributed Computing Engine for AI at Scale

SGLang: Structured Generation Language for Efficient LLM Serving

vLLM: High-Throughput LLM Inference at Scale

Ollama: Run Local LLMs on Your Own Hardware

Hugging Face: The Platform That Democratized Machine Learning

FastAPI: The Modern Framework for ML Model Serving