vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
Open op GitHubDetails
- Maker
- @vllm-project
- Taal
- Python
- stars
- 95.3K
- stars today
- 108
- AI relevance
- 0.90
A high-throughput and memory-efficient inference and serving engine for LLMs
Open op GitHub