39 results found Sort:

ray-project
ray ray-project
5.8k
34.1k
apache-2.0
474
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
ray java rllib automl python pytorch serving parallel deployment tensorflow distributed llm-serving data-science optimization deep-learning model-selection machine-learning hyperparameter-search reinforcement-learning hyperparameter-optimization
Created 2016-10-25
23,079 commits to master branch, last one 17 hours ago
tensorflow
serving tensorflow
2.2k
6.2k
apache-2.0
237
A flexible, high-performance serving system for machine learning models
ml cpp python serving tensorflow deep-learning neural-network machine-learning deep-neural-networks
Created 2016-01-26
8,808 commits to master branch, last one 6 days ago
vespa-engine
vespa vespa-engine
604
5.8k
apache-2.0
160
AI + Data, online. https://vespa.ai
ai cpp java vespa server serving big-data tensorflow search-engine vector-search machine-learning serving-recommendation
Created 2016-06-03
87,168 commits to master branch, last one 19 hours ago
SeldonIO
seldon-core SeldonIO
833
4.4k
other
86
An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models
aiops mlops serving deployment kubernetes machine-learning machine-learning-operations production-machine-learning
Created 2017-12-20
7,332 commits to master branch, last one 11 days ago
ahkarami
Deep-Learning-in-Production ahkarami
687
4.3k
unknown
148
In this repository, I will share some useful notes and references about deploying deep learning-based models in production.
flask keras mxnet react caffe2 python pytorch serving rest-api tutorial angularjs production tesnorflow c-plus-plus deep-learning model-serving tensorflow-models deep-neural-networks convert-pytorch-models serving-pytorch-models
Created 2018-05-03
221 commits to master branch, last one 12 days ago
pytorch
serve pytorch
863
4.2k
apache-2.0
57
Serve, optimize and scale PyTorch models in production
cpu gpu mlops docker metrics pytorch serving kubernetes optimization deep-learning machine-learning
Created 2019-10-03
3,880 commits to master branch, last one 28 days ago
PaddlePaddle
FastDeploy PaddlePaddle
465
3.0k
apache-2.0
54
⚡️An Easy-to-use and Fast Deep Learning Model Deployment Toolkit for ☁️Cloud 📱Mobile and 📹Edge. Including Image, Video, Text and Audio 20+ main stream scenarios and 150+ SOTA models with end-to-end ...
uie onnx intel jetson kunlun yolov5 yolov8 android picodet serving openvino rockchip tensorrt graphcore onnxruntime object-detection stable-diffusion
Created 2022-06-27
2,438 commits to develop branch, last one 6 hours ago
georgia-tech-db
evadb georgia-tech-db
262
2.6k
apache-2.0
28
Database system for AI-powered apps
ai eva llm agent gpt-4 chatgpt gpt4all serving auto-gpt database labeling langchain huggingface data-analysis hacktoberfest video-analytics object-detection
Created 2018-09-10
2,415 commits to staging branch, last one 11 months ago
Lightning-AI
LitServe Lightning-AI
158
2.5k
apache-2.0
25
Lightning-fast serving engine for any AI model of any size. Flexible. Easy. Enterprise-scale.
ai api web fastapi serving rest-api deep-learning developer-tools artificial-intelligence
Created 2023-12-12
495 commits to main branch, last one 17 hours ago
tobegit3hub
tensorflow_template_application tobegit3hub
713
1.9k
apache-2.0
186
TensorFlow template application for deep learning
cnn csv mlp lstm libsvm serving inference tfrecords tensorflow tensorboard deep-learning wide-and-deep machine-learning
Created 2016-07-18
205 commits to master branch, last one 3 years ago
ray-project
llm-applications ray-project
229
1.7k
cc-by-4.0
17
A comprehensive guide to building RAG-based LLM applications for production.
ray llms llama2 openai serving anyscale fine-tuning machine-learning
Created 2023-08-16
88 commits to main branch, last one 3 months ago
dingodb
dingo dingodb
270
1.6k
apache-2.0
163
A multi-modal vector database that supports upserts and vector queries using unified SQL (MySQL-Compatible) on structured and unstructured data, while meeting the requirements of high concurrency and ...
serving unified-sql vector-ocean embedding-store structured-data vector-database embedding-search unstructured-data mysql-compatibility real-time-semantic-search key-value-distributed-store
Created 2021-10-14
1,227 commits to develop branch, last one a day ago
Delta-ML
delta Delta-ML
288
1.6k
apache-2.0
67
DELTA is a deep learning based natural language and speech processing platform.
asr nlp nlu ops speech seq2seq serving front-end inference custom-ops tensorflow deep-learning tensorflow-lite text-generation speech-recognition tensorflow-serving emotion-recognition text-classification sequence-to-sequence speaker-verification
Created 2019-05-29
932 commits to master branch, last one 3 years ago
ray-project
ray-llm ray-project
94
1.2k
apache-2.0
20
RayLLM - LLMs on Ray
llm ray llmops serving llm-serving transformers llm-inference distributed-systems large-language-models
This repository has been archived (exclude archived)
Created 2023-05-31
198 commits to master branch, last one 5 months ago
PaddlePaddle
Serving PaddlePaddle
250
901
apache-2.0
96
A flexible, high-performance carrier for machine learning models（『飞桨』服务化部署框架）
dag gpu docker paddle python serving pipeline predictor prediction rpc-service deep-learning micro-service online-service paddle-serving microservice-toolkit
Created 2019-03-31
8,762 commits to v0.9.0 branch, last one 9 months ago
tobegit3hub
simple_tensorflow_serving tobegit3hub
192
757
apache-2.0
30
Generic and easy-to-use serving service for machine learning models
http client serving savedmodel tensorflow deep-learning machine-learning tensorflow-models
Created 2018-01-23
269 commits to master branch, last one 4 years ago
openvinotoolkit
model_server openvinotoolkit
212
675
apache-2.0
31
A scalable inference server for models optimized with OpenVINO™
ai dag edge cloud serving openvino inference kubernetes deep-learning model-serving machine-learning
Created 2018-09-26
2,610 commits to main branch, last one 22 hours ago
meta-soul
MetaSpore meta-soul
118
643
apache-2.0
71
A unified end-to-end machine intelligence platform
ai serving training abtesting deeplearning machinelearning
Created 2022-03-24
911 commits to main branch, last one 2 months ago
underneathall
pinferencia underneathall
87
564
apache-2.0
41
Python + Inference - Model Deployment library in Python. Simplest model inference server ever.
ai nlp python predict pytorch serving inference tensorflow huggingface modelserver data-science paddlepaddle transformers deep-learning model-serving computer-vision inference-server machine-learning model-deployment artificial-intelligence
Created 2022-04-04
55 commits to main branch, last one about a year ago
polyaxon
haupt polyaxon
209
455
agpl-3.0
37
Lineage metadata API, artifacts streams, sandbox, API, and spaces for Polyaxon
ui bokeh mlops models plotly python jupyter lineage pytorch serving tracking matplotlib tensorflow data-science deep-learning visualization data-profiling data-processing machine-learning data-visualization
Created 2016-05-15
489 commits to master branch, last one a day ago
bodywork-ml
bodywork-core bodywork-ml
22
435
agpl-3.0
11
ML pipeline orchestration and model deployments on Kubernetes.
cicd batch mlops devops python serving pipeline framework kubernetes data-science orchestration machine-learning continuous-deployment
This repository has been archived (exclude archived)
Created 2020-11-17
946 commits to master branch, last one 2 years ago
zzsza
Boostcamp-AI-Tech-Product-Serving zzsza
344
403
unknown
8
부스트캠프 AI Tech - Product Serving 자료
mlops serving
Created 2021-08-29
151 commits to main branch, last one 16 days ago
vectorch-ai
ScaleLLM vectorch-ai
30
393
apache-2.0
17
A high-performance inference system for large language models, designed for production environments.
gpu llm cuda llama model llama3 serving inference efficiency production performance speculative transformer llm-inference
Created 2023-07-24
630 commits to main branch, last one 12 hours ago
Hydrospheredata
hydro-serving Hydrospheredata
41
271
apache-2.0
24
MLOps Platform
spark models scoring serving realtime pipelines serverless tensorflow scikit-learn machine-learning
Created 2017-05-22
1,275 commits to master branch, last one 2 years ago
deepjavalibrary
djl-serving deepjavalibrary
67
199
apache-2.0
11
A universal scalable machine learning model deployment solution
djl pytorch serving inference deployment deep-learning
Created 2021-08-16
2,119 commits to master branch, last one a day ago
cap-ntu
ML-Model-CI cap-ntu
33
191
apache-2.0
18
MLModelCI is a complete MLOps platform for managing, converting, profiling, and deploying MLaaS (Machine Learning-as-a-Service), bridging the gap between current ML training and serving systems.
onnx mlops pytorch serving profiler tensorrt inference dispatcher deep-learning convert-models tensorflow-serving continuous-integration tensorrt-inference-server
Created 2020-04-22
843 commits to master branch, last one 3 years ago
FasterDecoding
BitDelta FasterDecoding
13
184
apache-2.0
4
This repository has no description...
llm serving quantization
Created 2024-02-14
17 commits to main branch, last one about a month ago
NetEase-Media
grps NetEase-Media
13
166
apache-2.0
11
【深度学习模型部署框架】支持tf/torch/trt/trtllm/vllm以及更多nn框架，支持dynamic batching、streaming模式，支持python/c++双语言，可限制，可拓展，高性能。帮助用户快速地将模型部署到线上，并通过http/rpc接口方式提供服务。
vllm torch serving tensorrt tensorflow tensorrt-llm dynamic-batching triton-inference-server
Created 2024-07-04
58 commits to master branch, last one 7 days ago
krystianity
keras-serving krystianity
27
153
apache-2.0
16
bring keras-models to production with tensorflow-serving and nodejs + docker :pizza:
cpp grpc keras docker nodejs python network serving neuronal production tensorflow
Created 2017-05-19
20 commits to master branch, last one 5 years ago
torchpipe
torchpipe torchpipe
13
145
apache-2.0
6
Serving Inside Pytorch
ray serve pytorch serving tensorrt inference torch2trt deployment llm-serving pipeline-parallelism triton-inference-server
Created 2023-10-24
247 commits to main branch, last one 27 days ago