50 results found Sort:

4.5k
29.7k
apache-2.0
242
A high-throughput and memory-efficient inference and serving engine for LLMs
Created 2023-02-09
3,290 commits to main branch, last one 7 hours ago
792
7.1k
apache-2.0
77
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
Created 2019-04-02
3,343 commits to main branch, last one 21 hours ago
In this repository, I will share some useful notes and references about deploying deep learning-based models in production.
Created 2018-05-03
220 commits to master branch, last one about a month ago
787
4.2k
apache-2.0
118
FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on a...
Created 2020-07-21
12,120 commits to master branch, last one 5 months ago
1.1k
3.6k
apache-2.0
64
Standardized Serverless ML Inference Platform on Kubernetes
Created 2019-03-27
1,618 commits to master branch, last one 14 hours ago
🚀 Awesome System for Machine Learning ⚡️ AI System Papers and Industry Practice. ⚡️ System for Machine Learning, LLM (Large Language Model), GenAI (Generative AI). 🍻 OSDI, NSDI, SIGCOMM, SoCC, MLSys...
Created 2019-01-07
509 commits to master branch, last one 2 months ago
203
2.6k
apache-2.0
23
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
Created 2023-07-22
343 commits to main branch, last one 5 days ago
143
2.2k
apache-2.0
33
Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
Created 2023-10-20
821 commits to main branch, last one a day ago
158
2.0k
apache-2.0
22
🏕️ Reproducible development environment
Created 2022-04-11
1,113 commits to main branch, last one 4 months ago
78
1.9k
mit
23
AICI: Prompts as (Wasm) Programs
Created 2023-09-26
1,615 commits to main branch, last one 2 months ago
252
1.4k
apache-2.0
28
MLRun is an open source MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your development and CI/CD environment and automates t...
Created 2019-09-01
5,508 commits to development branch, last one 13 hours ago
144
1.2k
agpl-3.0
35
Hopsworks - Data-Intensive AI platform with a Feature Store
Created 2018-07-26
6,412 commits to master branch, last one 2 days ago
The simplest way to serve AI/ML models in production
Created 2022-07-06
1,236 commits to main branch, last one 8 hours ago
69
789
other
19
Model Deployment at Scale on Kubernetes 🦄️
Created 2021-07-30
639 commits to main branch, last one 6 months ago
60
782
apache-2.0
12
A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine
Created 2021-03-13
386 commits to main branch, last one 6 days ago
211
671
apache-2.0
31
A scalable inference server for models optimized with OpenVINO™
Created 2018-09-26
2,564 commits to main branch, last one 11 hours ago
24
627
apache-2.0
6
A throughput-oriented high-performance serving framework for LLMs
Created 2024-08-19
43 commits to main branch, last one about a month ago
87
562
apache-2.0
41
Python + Inference - Model Deployment library in Python. Simplest model inference server ever.
Created 2022-04-04
55 commits to main branch, last one about a year ago
49
539
apache-2.0
12
RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
Created 2023-12-27
1,607 commits to main branch, last one 24 days ago
51
469
apache-2.0
13
Securely share and store AI/ML projects as OCI artifacts in your container registry.
Created 2024-02-02
599 commits to main branch, last one 9 hours ago
FastAPI Skeleton App to serve machine learning models production-ready.
Created 2020-02-15
12 commits to main branch, last one 6 months ago
Learn to serve Stable Diffusion models on cloud infrastructure at scale. This Lightning App shows load-balancing, orchestrating, pre-provisioning, dynamic batching, GPU-inference, micro-services worki...
This repository has been archived (exclude archived)
Created 2022-08-23
314 commits to main branch, last one about a year ago
65
374
apache-2.0
14
This repository has no description...
Created 2023-06-14
346 commits to main branch, last one 7 days ago
25
334
apache-2.0
12
BentoDiffusion: A collection of diffusion models served with BentoML
Created 2023-06-12
56 commits to main branch, last one 6 days ago
29
330
apache-2.0
14
Serverless LLM Serving for Everyone.
Created 2024-01-23
82 commits to main branch, last one 12 hours ago
31
227
apache-2.0
17
JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).
Created 2024-03-01
121 commits to main branch, last one 2 days ago
37
225
apache-2.0
6
A multi-functional library for full-stack Deep Learning. Simplifies Model Building, API development, and Model Deployment.
Created 2020-01-23
372 commits to master branch, last one 5 months ago
77
139
apache-2.0
30
A scalable, high-performance serving system for federated learning models
Created 2019-09-10
157 commits to master branch, last one 3 months ago
33
139
apache-2.0
24
Common library for serving TensorFlow, XGBoost and scikit-learn models in production.
This repository has been archived (exclude archived)
Created 2018-03-12
505 commits to main branch, last one about a year ago
40
138
apache-2.0
11
ClearML - Model-Serving Orchestration and Repository Solution
Created 2021-04-12
140 commits to main branch, last one 4 months ago