vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
Open on GitHubDetails
- Maker
- @vllm-project
- Language
- Python
- stars
- 95.3K
- stars today
- 108
- AI relevance
- 0.90
A high-throughput and memory-efficient inference and serving engine for LLMs
Open on GitHub