OpenCSGs / llm-inference

llm-inference is a platform for publishing and managing llm inference, providing a wide range of out-of-the-box features for model deployment, such as UI, RESTful API, auto-scaling, computing resource management, monitoring, and more.

Date Created 2024-02-28 (9 months ago)

Commits 116 (last one 7 months ago)

Stargazers 72 (0 this week)

Watchers 12 (0 this week)

Forks 16

License apache-2.0

Ranking

RepositoryStats indexes 595,856 repositories, of these OpenCSGs/llm-inference is ranked #367,915 (38th percentile) for total stargazers, and #178,842 for total watchers. Github reports the primary language for this repository as Python, for repositories using this language it is ranked #69,645/119,431.

OpenCSGs/llm-inference is also tagged with popular topics, for these it's ranked: transformer (#690/1018), llm-inference (#117/171)

Other Information

OpenCSGs/llm-inference has Github issues enabled, there are 13 open issues and 21 closed issues.

There have been 2 releases, the latest one was published on 2024-04-10 (8 months ago) with the name v0.1.0.

All Topics

ray vllm deepspeed llama-cpp transformer llm-inference

Star History

Github stargazers over time

Watcher History

Github watchers over time, collection started in '23

Recent Commit History

116 commits on the default branch (main) since jan '22

Yearly Commits

Commits to the default branch (main) per year

Issue History

Languages

The primary language is Python but there's also others...

updated: 2024-12-16 @ 02:04pm, id: 764401124 / R_kgDOLY_V5A