Search Results - RepositoryStats

717

3.0k

apache-2.0

68

Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.

llm rag nemo tensorrt microservice llm-inference gpu-acceleration large-language-models triton-inference-server retrieval-augmented-generation

Created 2023-10-19

147 commits to main branch, last one 5 days ago

BiSeNet CoinCheung

322

1.5k

mit

16

Add bisenetv2. My implementation of BiSeNet

ncnn ade20k bisenet pytorch openvino tensorrt cocostuff cityscapes triton-inference-server

Created 2018-11-29

63 commits to master branch, last one 4 months ago

yolov4-triton-tensorrt isarsoft

64

283

other

14

This repository deploys YOLOv4 as an optimized TensorRT engine to Triton Inference Server

docker yolov4 tensorrt yolov4-tiny deep-learning object-detection triton-inference-server

Created 2020-08-10

100 commits to master branch, last one 3 years ago

openai_trtllm npuichigo

28

205

mit

8

OpenAI compatible API for TensorRT LLM triton backend

llm langchain openai-api tensorrt-llm triton-inference-server

Created 2023-11-06

33 commits to main branch, last one 8 months ago

torchpipe torchpipe

13

160

apache-2.0

5

Serving Inside Pytorch

ray serve pytorch serving tensorrt inference torch2trt deployment llm-serving pipeline-parallelism triton-inference-server

Created 2023-10-24

348 commits to v0 branch, last one 2 months ago

grps NetEase-Media

13

157

apache-2.0

9

Deep Learning Deployment Framework: Supports tf/torch/trt/trtllm/vllm and other NN frameworks. Support dynamic batching, and streaming modes. It is dual-language compatible with Python and C++, offeri...

vllm torch serving tensorrt tensorflow tensorrt-llm dynamic-batching triton-inference-server

Created 2024-07-04

62 commits to master branch, last one 28 days ago

clearml-serving clearml

42

149

apache-2.0

10

ClearML - Model-Serving Orchestration and Repository Solution

ai mlops devops triton clearml serving kubernetes serving-ml deep-learning model-serving machine-learning tensorflow-serving serving-pytorch-models triton-inference-server

Created 2021-04-12

143 commits to main branch, last one 3 months ago

onnxruntime_backend triton-inference-server

62

141

bsd-3-clause

13

The Triton backend for the ONNX Runtime.

backend inference onnx-runtime triton-inference-server

Created 2020-08-26

139 commits to main branch, last one 20 days ago

stable-diffusion-tritonserver kamalkraj

19

123

apache-2.0

4

Deploy stable diffusion model with onnx/tenorrt + tritonserver

fp16 onnx deploy docker nvidia python3 pytorch tensorrt inference transformers stablediffusion machine-learning tensorrt-inference triton-inference-server

Created 2022-08-31

14 commits to master branch, last one 2 years ago

isaac_ros_dnn_inference NVIDIA-ISAAC-ROS

16

112

apache-2.0

4

NVIDIA-accelerated DNN model inference ROS 2 packages using NVIDIA Triton/TensorRT for both Jetson and x86_64 with CUDA-capable GPU

ai dnn gpu ros tao ros2 jetson nvidia triton tensorrt ros2-humble deeplearning deep-learning tensorrt-inference triton-inference-server

Created 2021-10-13

46 commits to main branch, last one about a month ago

fastDeploy notAI-tech

17

97

mit

7

Deploy DL/ ML inference pipelines with minimal extra code.

Created 2020-04-09

503 commits to master branch, last one 5 months ago

TrafficAnalyzer Koldim2001

6

82

unknown

4

Анализ трафика на круговом движении с использованием компьютерного зрения

Created 2024-02-16

127 commits to feature/influx branch, last one about a month ago

Diff-VC trinhtuanvubk

8

51

mit

2

Diffusion Model for Voice Conversion

gradio diffusion-models voice-conversion triton-inference-server

Created 2023-03-10

53 commits to main branch, last one about a year ago

YOLOV5_optimization_on_triton bug-developer021

12

50

mit

1

Compare multiple optimization methods on triton to imporve model service performance

gpu yolov5 tensorrt inference triton-inference-server

Created 2022-10-30

8 commits to main branch, last one about a year ago

recsys_pipeline akiragy

12

48

unknown

1

Build Recommender System with PyTorch + Redis + Elasticsearch + Feast + Triton + Flask. Vector Recall, DeepFM Ranking and Web Application.

feast flask redis python pytorch ranking retrieval elasticsearch inverted-index recommendation vector-database recommender-system triton-inference-server

Created 2023-08-05

7 commits to master branch, last one about a year ago

tritony rtzr

1

45

bsd-3-clause

1

Tiny configuration for Triton Inference Server

mlops inference tritonclient triton-inference-server

Created 2022-05-26

59 commits to main branch, last one 3 months ago

Setup-deeplearning-tools chiehpower

7

43

other

1

Set up CI in DL/ cuda/ cudnn/ TensorRT/ onnx2trt/ onnxruntime/ onnxsim/ Pytorch/ Triton-Inference-Server/ Bazel/ Tesseract/ PaddleOCR/ NVIDIA-docker/ minIO/ Supervisord on AGX or PC from scratch.