Search Results - RepositoryStats

13

147

apache-2.0

11

【深度学习模型部署框架】支持tf/torch/trt/trtllm/vllm以及更多nn框架，支持dynamic batching、streaming模式，支持python/c++双语言，可限制，可拓展，高性能。帮助用户快速地将模型部署到线上，并通过http/rpc接口方式提供服务。

vllm torch serving tensorrt tensorflow tensorrt-llm dynamic-batching triton-inference-server

Created 2024-07-04

53 commits to master branch, last one 22 days ago

1

81

mit

7

Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.

gpt llm python inference deep-learning dynamic-batching performance-optimization

Created 2023-04-17

44 commits to main branch, last one about a year ago

12

66

unknown

3

InsNet Runs Instance-dependent Neural Networks with Padding-free Dynamic Batching.

nlp dynamic-batching deep-learning-library

Created 2018-05-01

650 commits to master branch, last one 2 years ago