2 results found Sort:

bentoml
BentoML bentoml
829
7.6k
apache-2.0
74
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
llm mlops llmops python multimodal llm-serving ai-inference deep-learning generative-ai llm-inference model-serving ml-engineering machine-learning inference-platform model-inference-service
Created 2019-04-02
3,530 commits to main branch, last one 18 hours ago
InftyAI
llmaz InftyAI
18
110
apache-2.0
6
☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!
llm vllm ollama sglang llamacpp inference kubernetes modelscope huggingface inference-platform text-generation-inference
Created 2023-11-20
397 commits to main branch, last one 4 days ago