18 results found Sort:

NVIDIA
GenerativeAIExamples NVIDIA
566
2.6k
apache-2.0
60
Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.
llm rag nemo tensorrt microservice llm-inference gpu-acceleration large-language-models triton-inference-server retrieval-augmented-generation
Created 2023-10-19
130 commits to main branch, last one 4 days ago
CoinCheung
BiSeNet CoinCheung
315
1.5k
mit
17
Add bisenetv2. My implementation of BiSeNet
ncnn ade20k bisenet pytorch openvino tensorrt cocostuff cityscapes triton-inference-server
Created 2018-11-29
63 commits to master branch, last one 2 days ago
isarsoft
yolov4-triton-tensorrt isarsoft
64
279
other
15
This repository deploys YOLOv4 as an optimized TensorRT engine to Triton Inference Server
docker yolov4 tensorrt yolov4-tiny deep-learning object-detection triton-inference-server
Created 2020-08-10
100 commits to master branch, last one 2 years ago
npuichigo
openai_trtllm npuichigo
27
180
mit
8
OpenAI compatible API for TensorRT LLM triton backend
llm langchain openai-api tensorrt-llm triton-inference-server
Created 2023-11-06
33 commits to main branch, last one 4 months ago
NetEase-Media
grps NetEase-Media
13
169
apache-2.0
11
【深度学习模型部署框架】支持tf/torch/trt/trtllm/vllm以及更多nn框架，支持dynamic batching、streaming模式，支持python/c++双语言，可限制，可拓展，高性能。帮助用户快速地将模型部署到线上，并通过http/rpc接口方式提供服务。
vllm torch serving tensorrt tensorflow tensorrt-llm dynamic-batching triton-inference-server
Created 2024-07-04
58 commits to master branch, last one about a month ago
torchpipe
torchpipe torchpipe
13
147
apache-2.0
6
Serving Inside Pytorch
ray serve pytorch serving tensorrt inference torch2trt deployment llm-serving pipeline-parallelism triton-inference-server
Created 2023-10-24
337 commits to main branch, last one 23 days ago
allegroai
clearml-serving allegroai
40
139
apache-2.0
11
ClearML - Model-Serving Orchestration and Repository Solution
ai mlops devops triton clearml serving kubernetes serving-ml deep-learning model-serving machine-learning tensorflow-serving serving-pytorch-models triton-inference-server
Created 2021-04-12
142 commits to main branch, last one 5 days ago
triton-inference-server
onnxruntime_backend triton-inference-server
57
135
bsd-3-clause
16
The Triton backend for the ONNX Runtime.
backend inference onnx-runtime triton-inference-server
Created 2020-08-26
132 commits to main branch, last one 24 days ago
kamalkraj
stable-diffusion-tritonserver kamalkraj
19
123
apache-2.0
6
Deploy stable diffusion model with onnx/tenorrt + tritonserver
fp16 onnx deploy docker nvidia python3 pytorch tensorrt inference transformers stablediffusion machine-learning tensorrt-inference triton-inference-server
Created 2022-08-31
14 commits to master branch, last one 2 years ago
NVIDIA-ISAAC-ROS
isaac_ros_dnn_inference NVIDIA-ISAAC-ROS
17
106
apache-2.0
4
NVIDIA-accelerated DNN model inference ROS 2 packages using NVIDIA Triton/TensorRT for both Jetson and x86_64 with CUDA-capable GPU
ai dnn gpu ros tao ros2 jetson nvidia triton tensorrt ros2-humble deeplearning deep-learning tensorrt-inference triton-inference-server
Created 2021-10-13
42 commits to main branch, last one 10 days ago
notAI-tech
fastDeploy notAI-tech
17
96
mit
8
Deploy DL/ ML inference pipelines with minimal extra code.
docker falcon gevent python triton pytorch serving gunicorn websocket tf-serving torchserve http-server deep-learning model-serving triton-server streaming-audio inference-server model-deployment tensorflow-serving triton-inference-server
Created 2020-04-09
503 commits to master branch, last one about a month ago
Koldim2001
TrafficAnalyzer Koldim2001
4
57
unknown
4
Анализ трафика на круговом движении с использованием компьютерного зрения
flask hydra nginx yolov8 grafana influxdb postgres bytetrack postgresql bytetracker docker-compose oop-principles multiprocessing object-detection traffic-analysis grafana-dashboard tensorrt-inference transport-detection triton-inference-server multiple-object-tracking
Created 2024-02-16
80 commits to main branch, last one 11 days ago
bug-developer021
YOLOV5_optimization_on_triton bug-developer021
12
48
mit
1
Compare multiple optimization methods on triton to imporve model service performance
gpu yolov5 tensorrt inference triton-inference-server
Created 2022-10-30
8 commits to main branch, last one 11 months ago
akiragy
recsys_pipeline akiragy
12
45
unknown
1
Build Recommender System with PyTorch + Redis + Elasticsearch + Feast + Triton + Flask. Vector Recall, DeepFM Ranking and Web Application.
feast flask redis python pytorch ranking retrieval elasticsearch inverted-index recommendation vector-database recommender-system triton-inference-server
Created 2023-08-05
7 commits to master branch, last one about a year ago
chiehpower
Setup-deeplearning-tools chiehpower
7
43
other
2
Set up CI in DL/ cuda/ cudnn/ TensorRT/ onnx2trt/ onnxruntime/ onnxsim/ Pytorch/ Triton-Inference-Server/ Bazel/ Tesseract/ PaddleOCR/ NVIDIA-docker/ minIO/ Supervisord on AGX or PC from scratch.
ci agx cuda cudnn minio docker nvidia pytorch onnx2trt tensorrt paddleocr onnxruntime supervisord installation deep-learning tesseract-ocr triton-server onnx-simplifier triton-inference-server tensorrt-inference-server
Created 2020-02-27
79 commits to master branch, last one about a year ago
rtzr
tritony rtzr
1
43
bsd-3-clause
1
Tiny configuration for Triton Inference Server
mlops inference tritonclient triton-inference-server
Created 2022-05-26
58 commits to main branch, last one 17 days ago
trinhtuanvubk
Diff-VC trinhtuanvubk
7
40
mit
3
Diffusion Model for Voice Conversion
gradio diffusion-models voice-conversion triton-inference-server
Created 2023-03-10
53 commits to main branch, last one 9 months ago
omarabid59
yolov8-triton omarabid59
9
32
apache-2.0
1
Provides an ensemble model to deploy a YoloV8 ONNX model to Triton
yolov8 deployment ultralytics triton-server triton-inference-server
Created 2023-03-08
6 commits to main branch, last one about a year ago