37 results found Sort:

ray-project
ray ray-project
5.4k
31.8k
apache-2.0
472
Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
ray java rllib automl python pytorch serving parallel deployment tensorflow distributed llm-serving data-science optimization deep-learning model-selection machine-learning hyperparameter-search reinforcement-learning hyperparameter-optimization
Created 2016-10-25
21,944 commits to master branch, last one a day ago
tensorflow
serving tensorflow
2.2k
6.1k
apache-2.0
240
A flexible, high-performance serving system for machine learning models
ml cpp python serving tensorflow deep-learning neural-network machine-learning deep-neural-networks
Created 2016-01-26
8,577 commits to master branch, last one 5 days ago
vespa-engine
vespa vespa-engine
579
5.4k
apache-2.0
159
AI + Data, online. https://vespa.ai
ai cpp java vespa server serving big-data tensorflow search-engine vector-search machine-learning serving-recommendation
Created 2016-06-03
83,820 commits to master branch, last one 5 hours ago
SeldonIO
seldon-core SeldonIO
827
4.3k
other
83
An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models
aiops mlops serving deployment kubernetes machine-learning machine-learning-operations production-machine-learning
Created 2017-12-20
7,327 commits to master branch, last one 17 days ago
ahkarami
Deep-Learning-in-Production ahkarami
680
4.3k
unknown
149
In this repository, I will share some useful notes and references about deploying deep learning-based models in production.
flask keras mxnet react caffe2 python pytorch serving rest-api tutorial angularjs production tesnorflow c-plus-plus deep-learning model-serving tensorflow-models deep-neural-networks convert-pytorch-models serving-pytorch-models
Created 2018-05-03
219 commits to master branch, last one about a month ago
pytorch
serve pytorch
819
4.0k
apache-2.0
58
Serve, optimize and scale PyTorch models in production
cpu gpu mlops docker metrics pytorch serving kubernetes optimization deep-learning machine-learning
Created 2019-10-03
3,792 commits to master branch, last one 21 hours ago
PaddlePaddle
FastDeploy PaddlePaddle
438
2.8k
apache-2.0
56
⚡️An Easy-to-use and Fast Deep Learning Model Deployment Toolkit for ☁️Cloud 📱Mobile and 📹Edge. Including Image, Video, Text and Audio 20+ main stream scenarios and 150+ SOTA models with end-to-end ...
uie onnx intel jetson kunlun yolov5 yolov8 android picodet serving openvino rockchip tensorrt graphcore onnxruntime object-detection stable-diffusion
Created 2022-06-27
2,422 commits to develop branch, last one 2 months ago
georgia-tech-db
evadb georgia-tech-db
261
2.6k
apache-2.0
27
Database system for AI-powered apps
ai eva llm agent gpt-4 chatgpt gpt4all serving auto-gpt database labeling langchain huggingface data-analysis hacktoberfest video-analytics object-detection
Created 2018-09-10
2,415 commits to staging branch, last one 6 months ago
tobegit3hub
tensorflow_template_application tobegit3hub
718
1.9k
apache-2.0
187
TensorFlow template application for deep learning
cnn csv mlp lstm libsvm serving inference tfrecords tensorflow tensorboard deep-learning wide-and-deep machine-learning
Created 2016-07-18
205 commits to master branch, last one 3 years ago
Delta-ML
delta Delta-ML
292
1.6k
apache-2.0
66
DELTA is a deep learning based natural language and speech processing platform.
asr nlp nlu ops speech seq2seq serving front-end inference custom-ops tensorflow deep-learning tensorflow-lite text-generation speech-recognition tensorflow-serving emotion-recognition text-classification sequence-to-sequence speaker-verification
Created 2019-05-29
932 commits to master branch, last one 3 years ago
ray-project
llm-applications ray-project
201
1.6k
cc-by-4.0
16
A comprehensive guide to building RAG-based LLM applications for production.
ray llms llama2 openai serving anyscale fine-tuning machine-learning
Created 2023-08-16
85 commits to main branch, last one about a month ago
ray-project
ray-llm ray-project
85
1.2k
apache-2.0
20
RayLLM - LLMs on Ray
llm ray llmops serving llm-serving transformers llm-inference distributed-systems large-language-models
This repository has been archived (exclude archived)
Created 2023-05-31
198 commits to master branch, last one 22 days ago
PaddlePaddle
Serving PaddlePaddle
245
884
apache-2.0
97
A flexible, high-performance carrier for machine learning models（『飞桨』服务化部署框架）
dag gpu docker paddle python serving pipeline predictor prediction rpc-service deep-learning micro-service online-service paddle-serving microservice-toolkit
Created 2019-03-31
8,762 commits to v0.9.0 branch, last one 4 months ago
tobegit3hub
simple_tensorflow_serving tobegit3hub
196
757
apache-2.0
30
Generic and easy-to-use serving service for machine learning models
http client serving savedmodel tensorflow deep-learning machine-learning tensorflow-models
Created 2018-01-23
269 commits to master branch, last one 3 years ago
dingodb
dingo dingodb
167
745
apache-2.0
72
A multi-modal vector database that supports upserts and vector queries using unified SQL (MySQL-Compatible) on structured and unstructured data, while meeting the requirements of high concurrency and ...
serving unified-sql vector-ocean embedding-store structured-data vector-database embedding-search unstructured-data mysql-compatibility real-time-semantic-search key-value-distributed-store
Created 2021-10-14
1,061 commits to develop branch, last one a day ago
openvinotoolkit
model_server openvinotoolkit
197
641
apache-2.0
30
A scalable inference server for models optimized with OpenVINO™
ai dag edge cloud serving openvino inference kubernetes deep-learning model-serving machine-learning
Created 2018-09-26
2,368 commits to main branch, last one a day ago
meta-soul
MetaSpore meta-soul
116
630
apache-2.0
70
A unified end-to-end machine intelligence platform
ai serving training abtesting deeplearning machinelearning
Created 2022-03-24
910 commits to main branch, last one 2 months ago
underneathall
pinferencia underneathall
85
559
apache-2.0
41
Python + Inference - Model Deployment library in Python. Simplest model inference server ever.
ai nlp python predict pytorch serving inference tensorflow huggingface modelserver data-science paddlepaddle transformers deep-learning model-serving computer-vision inference-server machine-learning model-deployment artificial-intelligence
Created 2022-04-04
55 commits to main branch, last one about a year ago
polyaxon
haupt polyaxon
213
453
agpl-3.0
37
Lineage metadata API, artifacts streams, sandbox, API, and spaces for Polyaxon
ui bokeh mlops models plotly python jupyter lineage pytorch serving tracking matplotlib tensorflow data-science deep-learning visualization data-profiling data-processing machine-learning data-visualization
Created 2016-05-15
449 commits to master branch, last one 8 days ago
bodywork-ml
bodywork-core bodywork-ml
22
434
agpl-3.0
11
ML pipeline orchestration and model deployments on Kubernetes.
cicd batch mlops devops python serving pipeline framework kubernetes data-science orchestration machine-learning continuous-deployment
This repository has been archived (exclude archived)
Created 2020-11-17
946 commits to master branch, last one about a year ago
zzsza
Boostcamp-AI-Tech-Product-Serving zzsza
344
403
unknown
8
부스트캠프 AI Tech - Product Serving 자료
mlops serving
Created 2021-08-29
150 commits to main branch, last one 4 months ago
vectorch-ai
ScaleLLM vectorch-ai
23
308
apache-2.0
15
A high-performance inference system for large language models, designed for production environments.
gpu llm cuda llama model llama3 serving inference efficiency production performance speculative transformer llm-inference
Created 2023-07-24
536 commits to main branch, last one 13 hours ago
Hydrospheredata
hydro-serving Hydrospheredata
42
270
apache-2.0
24
MLOps Platform
spark models scoring serving realtime pipelines serverless tensorflow scikit-learn machine-learning
Created 2017-05-22
1,275 commits to master branch, last one 2 years ago
cap-ntu
ML-Model-CI cap-ntu
33
188
apache-2.0
18
MLModelCI is a complete MLOps platform for managing, converting, profiling, and deploying MLaaS (Machine Learning-as-a-Service), bridging the gap between current ML training and serving systems.
onnx mlops pytorch serving profiler tensorrt inference dispatcher deep-learning convert-models tensorflow-serving continuous-integration tensorrt-inference-server
Created 2020-04-22
843 commits to master branch, last one 3 years ago
deepjavalibrary
djl-serving deepjavalibrary
58
179
apache-2.0
12
A universal scalable machine learning model deployment solution
djl pytorch serving inference deployment deep-learning
Created 2021-08-16
1,769 commits to master branch, last one 18 hours ago
krystianity
keras-serving krystianity
27
153
apache-2.0
16
bring keras-models to production with tensorflow-serving and nodejs + docker :pizza:
cpp grpc keras docker nodejs python network serving neuronal production tensorflow
Created 2017-05-19
20 commits to master branch, last one 5 years ago
torchpipe
torchpipe torchpipe
12
135
apache-2.0
6
Boosting DL Service Throughput 1.5-4x by Ensemble Pipeline Serving with Concurrent CUDA Streams for PyTorch/LibTorch Frontend and TensorRT/CVCUDA, etc., Backends
ray serve cvcuda pytorch serving tensorrt inference ray-serve torch2trt deployment pipeline-parallelism triton-inference-server
Created 2023-10-24
147 commits to main branch, last one 15 days ago
allegroai
clearml-serving allegroai
39
131
apache-2.0
11
ClearML - Model-Serving Orchestration and Repository Solution
ai mlops devops triton clearml serving kubernetes serving-ml deep-learning model-serving machine-learning tensorflow-serving serving-pytorch-models triton-inference-server
Created 2021-04-12
138 commits to main branch, last one 3 months ago
Lightning-AI
LitServe Lightning-AI
12
119
apache-2.0
15
Deploy AI models at scale. High-throughput serving engine for AI/ML models that uses the latest state-of-the-art model deployment techniques.
ai api serving
Created 2023-12-12
171 commits to main branch, last one 6 hours ago
emacski
tensorflow-serving-arm emacski
16
98
apache-2.0
3
TensorFlow Serving ARM - A project for cross-compiling TensorFlow Serving targeting popular ARM cores
arm arm64 armhf armv7 armv8 docker aarch64 serving tensorflow cross-compile
Created 2018-09-26
49 commits to master branch, last one 2 years ago