15 results found Sort:

NVIDIA
GenerativeAIExamples NVIDIA
288
1.8k
apache-2.0
50
Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.
llm rag nemo tensorrt microservice llm-inference gpu-acceleration large-language-models triton-inference-server retrieval-augmented-generation
Created 2023-10-19
37 commits to main branch, last one 8 days ago
CoinCheung
BiSeNet CoinCheung
304
1.4k
mit
17
Add bisenetv2. My implementation of BiSeNet
ncnn ade20k bisenet pytorch openvino tensorrt cocostuff cityscapes triton-inference-server
Created 2018-11-29
59 commits to master branch, last one about a year ago
isarsoft
yolov4-triton-tensorrt isarsoft
63
276
other
15
This repository deploys YOLOv4 as an optimized TensorRT engine to Triton Inference Server
docker yolov4 tensorrt yolov4-tiny deep-learning object-detection triton-inference-server
Created 2020-08-10
100 commits to master branch, last one 2 years ago
torchpipe
torchpipe torchpipe
12
135
apache-2.0
6
Boosting DL Service Throughput 1.5-4x by Ensemble Pipeline Serving with Concurrent CUDA Streams for PyTorch/LibTorch Frontend and TensorRT/CVCUDA, etc., Backends
ray serve cvcuda pytorch serving tensorrt inference ray-serve torch2trt deployment pipeline-parallelism triton-inference-server
Created 2023-10-24
147 commits to main branch, last one 21 days ago
allegroai
clearml-serving allegroai
39
132
apache-2.0
11
ClearML - Model-Serving Orchestration and Repository Solution
ai mlops devops triton clearml serving kubernetes serving-ml deep-learning model-serving machine-learning tensorflow-serving serving-pytorch-models triton-inference-server
Created 2021-04-12
138 commits to main branch, last one 3 months ago
kamalkraj
stable-diffusion-tritonserver kamalkraj
20
116
apache-2.0
6
Deploy stable diffusion model with onnx/tenorrt + tritonserver
fp16 onnx deploy docker nvidia python3 pytorch tensorrt inference transformers stablediffusion machine-learning tensorrt-inference triton-inference-server
Created 2022-08-31
14 commits to master branch, last one about a year ago
triton-inference-server
onnxruntime_backend triton-inference-server
53
115
bsd-3-clause
15
The Triton backend for the ONNX Runtime.
backend inference onnx-runtime triton-inference-server
Created 2020-08-26
119 commits to main branch, last one 19 days ago
npuichigo
openai_trtllm npuichigo
19
110
mit
6
OpenAI compatible API for TensorRT LLM triton backend
llm langchain openai-api tensorrt-llm triton-inference-server
Created 2023-11-06
31 commits to main branch, last one 2 months ago
NVIDIA-ISAAC-ROS
isaac_ros_dnn_inference NVIDIA-ISAAC-ROS
14
101
apache-2.0
4
NVIDIA-accelerated DNN model inference ROS 2 packages using NVIDIA Triton/TensorRT for both Jetson and x86_64 with CUDA-capable GPU
ai dnn gpu ros tao ros2 jetson nvidia triton tensorrt ros2-humble deeplearning deep-learning tensorrt-inference triton-inference-server
Created 2021-10-13
38 commits to main branch, last one 11 days ago
notAI-tech
fastDeploy notAI-tech
17
92
mit
8
Deploy DL/ ML inference pipelines with minimal extra code.
docker falcon gevent python triton pytorch serving gunicorn websocket tf-serving torchserve http-server deep-learning model-serving triton-server streaming-audio inference-server model-deployment tensorflow-serving triton-inference-server
Created 2020-04-09
458 commits to master branch, last one 2 months ago
bug-developer021
YOLOV5_optimization_on_triton bug-developer021
11
45
mit
1
Compare multiple optimization methods on triton to imporve model service performance
gpu yolov5 tensorrt inference triton-inference-server
Created 2022-10-30
8 commits to main branch, last one 5 months ago
chiehpower
Setup-deeplearning-tools chiehpower
7
44
other
2
Set up CI in DL/ cuda/ cudnn/ TensorRT/ onnx2trt/ onnxruntime/ onnxsim/ Pytorch/ Triton-Inference-Server/ Bazel/ Tesseract/ PaddleOCR/ NVIDIA-docker/ minIO/ Supervisord on AGX or PC from scratch.
ci agx cuda cudnn minio docker nvidia pytorch onnx2trt tensorrt paddleocr onnxruntime supervisord installation deep-learning tesseract-ocr triton-server onnx-simplifier triton-inference-server tensorrt-inference-server
Created 2020-02-27
79 commits to master branch, last one 9 months ago
rtzr
tritony rtzr
1
40
bsd-3-clause
2
Tiny configuration for Triton Inference Server
mlops inference tritonclient triton-inference-server
Created 2022-05-26
53 commits to main branch, last one 6 months ago
akiragy
recsys_pipeline akiragy
7
34
unknown
1
Build Recommender System with PyTorch + Redis + Elasticsearch + Feast + Triton + Flask. Vector Recall, DeepFM Ranking and Web Application.
feast flask redis python pytorch ranking retrieval elasticsearch inverted-index recommendation vector-database recommender-system triton-inference-server
Created 2023-08-05
7 commits to master branch, last one 9 months ago
trinhtuanvubk
Diff-VC trinhtuanvubk
5
28
mit
3
Diffusion Model for Voice Conversion
gradio diffusion-models voice-conversion triton-inference-server
Created 2023-03-10
53 commits to main branch, last one 3 months ago