2 results found Sort:

huggingface
optimum-benchmark huggingface
56
290
apache-2.0
5
🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Optimum's hardware optimizations & quantization schemes.
pytorch openvino benchmark onnxruntime tensorrt-llm neural-compressor text-generation-inference
Created 2023-04-26
713 commits to main branch, last one about a month ago
InftyAI
llmaz InftyAI
13
80
apache-2.0
5
☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!
llm vllm ollama sglang llamacpp kubernetes modelscope huggingface llm-inference text-generation-inference
Created 2023-11-20
357 commits to main branch, last one 6 days ago