NetEase-Media / grps_trtllm

【高性能OpenAI LLM服务】通过GPRS+TensorRT-LLM+Tokenizers.cpp实现纯C++版高性能OpenAI LLM服务,支持chat和function call模式,支持ai agent,支持分布式多卡推理,支持多模态,支持gradio聊天界面。

Date Created 2024-08-21 (6 months ago)
Commits 125 (last one 8 days ago)
Stargazers 95 (1 this week)
Watchers 3 (0 this week)
Forks 5
License apache-2.0
Ranking

RepositoryStats indexes 617,578 repositories, of these NetEase-Media/grps_trtllm is ranked #312,517 (49th percentile) for total stargazers, and #433,830 for total watchers. Github reports the primary language for this repository as C++, for repositories using this language it is ranked #17,575/32,993.

NetEase-Media/grps_trtllm is also tagged with popular topics, for these it's ranked: llm (#1,871/3275),  openai (#1,271/2404),  llama3 (#103/181)

Other Information

NetEase-Media/grps_trtllm has Github issues enabled, there are 4 open issues and 2 closed issues.

Star History

Github stargazers over time

120120100100808060604040202000Sep '24Sep '24Oct '24Oct '24Nov '24Nov '24Dec '24Dec '2420252025Feb '25Feb '25

Watcher History

Github watchers over time, collection started in '23

3333332.52.522222215 Sep15 SepOct '24Oct '2415 Oct15 OctNov '24Nov '2415 Nov15 NovDec '24Dec '2415 Dec15 DecJan '25Jan '2515 Jan15 JanFeb '25Feb '2515 Feb15 Feb

Recent Commit History

125 commits on the default branch (master) since jan '22

140140120120100100808060604040202000Sep '24Sep '24Oct '24Oct '24Nov '24Nov '24Dec '24Dec '2420252025Feb '25Feb '25

Yearly Commits

Commits to the default branch (master) per year

12012010010080806060404020200020242024

Issue History

Total Issues
Open Issues
Closed Issues
66554433221100Oct '24Oct '2415 Oct15 OctNov '24Nov '2415 Nov15 NovDec '24Dec '2415 Dec15 DecJan '25Jan '2515 Jan15 JanFeb '25Feb '2515 Feb15 Feb

Languages

The primary language is C++ but there's also others...

C++C++PythonPythonCMakeCMakeShellShellDockerfileDockerfileCC

updated: 2025-02-19 @ 10:39am, id: 845523645 / R_kgDOMmWqvQ