Search Results - RepositoryStats

193

2.8k

gpl-3.0

91

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

llm llms sora vllm deepseek open-sora awesome-llm tensorrt-llm llm-inference flash-attention paged-attention flash-attention-2 flash-attention-3

Created 2023-08-27

405 commits to main branch, last one 6 days ago

cortex.cpp janhq

116

2.1k

apache-2.0

16

Local AI API Platform

gguf onnx llamacpp onnxruntime tensorrt-llm

Created 2023-09-11

1,784 commits to dev branch, last one 16 hours ago

WhisperLive collabora

274

2.0k

mit

30

A nearly-live implementation of OpenAI's Whisper.

obs openai whisper tensorrt dictation translation tensorrt-llm text-to-speech whisper-tensorrt voice-recognition

Created 2023-05-04

445 commits to main branch, last one a day ago

WhisperS2T shashikg

31

305

mit

14

An Optimized Speech-to-Text Pipeline for the Whisper Model Supporting Multiple Inference Engine

asr vad whisper tensorrt tensorrt-llm deep-learning speech-to-text speech-recognition voice-activity-detection

Created 2023-12-16

81 commits to main branch, last one 2 months ago

optimum-benchmark huggingface

48

250

apache-2.0

5

🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Optimum's hardware optimizations & quantization schemes.

pytorch openvino benchmark onnxruntime tensorrt-llm neural-compressor text-generation-inference

Created 2023-04-26

692 commits to main branch, last one 23 days ago