40 results found Sort:

ray-project
ray ray-project
5.9k
34.6k
apache-2.0
480
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
ray java rllib automl python pytorch serving parallel deployment tensorflow distributed llm-serving data-science optimization deep-learning model-selection machine-learning hyperparameter-search reinforcement-learning hyperparameter-optimization
Created 2016-10-25
23,378 commits to master branch, last one 19 hours ago
tensorflow
serving tensorflow
2.2k
6.2k
apache-2.0
232
A flexible, high-performance serving system for machine learning models
ml cpp python serving tensorflow deep-learning neural-network machine-learning deep-neural-networks
Created 2016-01-26
8,814 commits to master branch, last one 9 days ago
vespa-engine
vespa vespa-engine
608
5.9k
apache-2.0
157
AI + Data, online. https://vespa.ai
ai cpp java vespa server serving big-data tensorflow search-engine vector-search machine-learning serving-recommendation
Created 2016-06-03
87,538 commits to master branch, last one 19 hours ago
SeldonIO
seldon-core SeldonIO
834
4.4k
other
86
An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models
aiops mlops serving deployment kubernetes machine-learning machine-learning-operations production-machine-learning
Created 2017-12-20
7,332 commits to master branch, last one about a month ago
ahkarami
Deep-Learning-in-Production ahkarami
688
4.3k
unknown
148
In this repository, I will share some useful notes and references about deploying deep learning-based models in production.
flask keras mxnet react caffe2 python pytorch serving rest-api tutorial angularjs production tesnorflow c-plus-plus deep-learning model-serving tensorflow-models deep-neural-networks convert-pytorch-models serving-pytorch-models
Created 2018-05-03
221 commits to master branch, last one about a month ago
pytorch
serve pytorch
866
4.2k
apache-2.0
57
Serve, optimize and scale PyTorch models in production
cpu gpu mlops docker metrics pytorch serving kubernetes optimization deep-learning machine-learning
Created 2019-10-03
3,885 commits to master branch, last one 21 hours ago
PaddlePaddle
FastDeploy PaddlePaddle
466
3.0k
apache-2.0
56
⚡️An Easy-to-use and Fast Deep Learning Model Deployment Toolkit for ☁️Cloud 📱Mobile and 📹Edge. Including Image, Video, Text and Audio 20+ main stream scenarios and 150+ SOTA models with end-to-end ...
uie onnx intel jetson kunlun yolov5 yolov8 android picodet serving openvino rockchip tensorrt graphcore onnxruntime object-detection stable-diffusion
Created 2022-06-27
2,448 commits to develop branch, last one about a month ago
georgia-tech-db
evadb georgia-tech-db
265
2.6k
apache-2.0
29
Database system for AI-powered apps
ai eva llm agent gpt-4 chatgpt gpt4all serving auto-gpt database labeling langchain huggingface data-analysis hacktoberfest video-analytics object-detection
Created 2018-09-10
2,415 commits to staging branch, last one about a year ago
Lightning-AI
LitServe Lightning-AI
167
2.6k
apache-2.0
27
Lightning-fast serving engine for any AI model of any size. Flexible. Easy. Enterprise-scale.
ai api web fastapi serving rest-api deep-learning developer-tools artificial-intelligence
Created 2023-12-12
520 commits to main branch, last one 3 days ago
tobegit3hub
tensorflow_template_application tobegit3hub
714
1.9k
apache-2.0
185
TensorFlow template application for deep learning
cnn csv mlp lstm libsvm serving inference tfrecords tensorflow tensorboard deep-learning wide-and-deep machine-learning
Created 2016-07-18
205 commits to master branch, last one 3 years ago
ray-project
llm-applications ray-project
233
1.7k
cc-by-4.0
18
A comprehensive guide to building RAG-based LLM applications for production.
ray llms llama2 openai serving anyscale fine-tuning machine-learning
Created 2023-08-16
88 commits to main branch, last one 4 months ago
dingodb
dingo dingodb
271
1.7k
apache-2.0
163
A multi-modal vector database that supports upserts and vector queries using unified SQL (MySQL-Compatible) on structured and unstructured data, while meeting the requirements of high concurrency and ...
serving unified-sql vector-ocean embedding-store structured-data vector-database embedding-search unstructured-data mysql-compatibility real-time-semantic-search key-value-distributed-store
Created 2021-10-14
1,255 commits to develop branch, last one 3 days ago
Delta-ML
delta Delta-ML
288
1.6k
apache-2.0
67
DELTA is a deep learning based natural language and speech processing platform. LF AI & DATA Projects: https://lfaidata.foundation/projects/delta/
asr nlp nlu ops speech seq2seq serving front-end inference custom-ops tensorflow deep-learning tensorflow-lite text-generation speech-recognition tensorflow-serving emotion-recognition text-classification sequence-to-sequence speaker-verification
Created 2019-05-29
932 commits to master branch, last one 4 years ago
ray-project
ray-llm ray-project
93
1.2k
apache-2.0
19
RayLLM - LLMs on Ray
llm ray llmops serving llm-serving transformers llm-inference distributed-systems large-language-models
This repository has been archived (exclude archived)
Created 2023-05-31
198 commits to master branch, last one 6 months ago
PaddlePaddle
Serving PaddlePaddle
252
901
apache-2.0
96
A flexible, high-performance carrier for machine learning models（『飞桨』服务化部署框架）
dag gpu docker paddle python serving pipeline predictor prediction rpc-service deep-learning micro-service online-service paddle-serving microservice-toolkit
Created 2019-03-31
8,762 commits to v0.9.0 branch, last one 10 months ago
tobegit3hub
simple_tensorflow_serving tobegit3hub
191
757
apache-2.0
30
Generic and easy-to-use serving service for machine learning models
http client serving savedmodel tensorflow deep-learning machine-learning tensorflow-models
Created 2018-01-23
269 commits to master branch, last one 4 years ago
openvinotoolkit
model_server openvinotoolkit
212
688
apache-2.0
32
A scalable inference server for models optimized with OpenVINO™
ai dag edge cloud serving openvino inference kubernetes deep-learning model-serving machine-learning
Created 2018-09-26
2,667 commits to main branch, last one 4 hours ago
meta-soul
MetaSpore meta-soul
118
644
apache-2.0
71
A unified end-to-end machine intelligence platform
ai serving training abtesting deeplearning machinelearning
Created 2022-03-24
911 commits to main branch, last one 3 months ago
underneathall
pinferencia underneathall
87
564
apache-2.0
41
Python + Inference - Model Deployment library in Python. Simplest model inference server ever.
ai nlp python predict pytorch serving inference tensorflow huggingface modelserver data-science paddlepaddle transformers deep-learning model-serving computer-vision inference-server machine-learning model-deployment artificial-intelligence
Created 2022-04-04
55 commits to main branch, last one 2 years ago
polyaxon
haupt polyaxon
209
455
agpl-3.0
37
Lineage metadata API, artifacts streams, sandbox, API, and spaces for Polyaxon
ui bokeh mlops models plotly python jupyter lineage pytorch serving tracking matplotlib tensorflow data-science deep-learning visualization data-profiling data-processing machine-learning data-visualization
Created 2016-05-15
497 commits to master branch, last one 2 days ago
zzsza
Boostcamp-AI-Tech-Product-Serving zzsza
368
448
unknown
8
부스트캠프 AI Tech - Product Serving 자료
mlops serving
Created 2021-08-29
151 commits to main branch, last one about a month ago
bodywork-ml
bodywork-core bodywork-ml
22
435
agpl-3.0
11
ML pipeline orchestration and model deployments on Kubernetes.
cicd batch mlops devops python serving pipeline framework kubernetes data-science orchestration machine-learning continuous-deployment
This repository has been archived (exclude archived)
Created 2020-11-17
946 commits to master branch, last one 2 years ago
vectorch-ai
ScaleLLM vectorch-ai
30
397
apache-2.0
17
A high-performance inference system for large language models, designed for production environments.
gpu llm cuda llama model llama3 serving inference efficiency production performance speculative transformer llm-inference
Created 2023-07-24
631 commits to main branch, last one about a month ago
Hydrospheredata
hydro-serving Hydrospheredata
40
271
apache-2.0
24
MLOps Platform
spark models scoring serving realtime pipelines serverless tensorflow scikit-learn machine-learning
Created 2017-05-22
1,275 commits to master branch, last one 2 years ago
deepjavalibrary
djl-serving deepjavalibrary
68
203
apache-2.0
11
A universal scalable machine learning model deployment solution
djl pytorch serving inference deployment deep-learning
Created 2021-08-16
2,155 commits to master branch, last one a day ago
cap-ntu
ML-Model-CI cap-ntu
33
191
apache-2.0
18
MLModelCI is a complete MLOps platform for managing, converting, profiling, and deploying MLaaS (Machine Learning-as-a-Service), bridging the gap between current ML training and serving systems.
onnx mlops pytorch serving profiler tensorrt inference dispatcher deep-learning convert-models tensorflow-serving continuous-integration tensorrt-inference-server
Created 2020-04-22
843 commits to master branch, last one 3 years ago
FasterDecoding
BitDelta FasterDecoding
14
187
apache-2.0
4
This repository has no description...
llm serving quantization
Created 2024-02-14
18 commits to main branch, last one 16 days ago
NetEase-Media
grps NetEase-Media
13
169
apache-2.0
11
【深度学习模型部署框架】支持tf/torch/trt/trtllm/vllm以及更多nn框架，支持dynamic batching、streaming模式，支持python/c++双语言，可限制，可拓展，高性能。帮助用户快速地将模型部署到线上，并通过http/rpc接口方式提供服务。
vllm torch serving tensorrt tensorflow tensorrt-llm dynamic-batching triton-inference-server
Created 2024-07-04
58 commits to master branch, last one about a month ago
krystianity
keras-serving krystianity
27
153
apache-2.0
16
bring keras-models to production with tensorflow-serving and nodejs + docker :pizza:
cpp grpc keras docker nodejs python network serving neuronal production tensorflow
Created 2017-05-19
20 commits to master branch, last one 5 years ago
torchpipe
torchpipe torchpipe
13
147
apache-2.0
6
Serving Inside Pytorch
ray serve pytorch serving tensorrt inference torch2trt deployment llm-serving pipeline-parallelism triton-inference-server
Created 2023-10-24
337 commits to main branch, last one 23 days ago