Search Results - RepositoryStats

175

2.6k

gpl-3.0

85

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

llm llms sora vllm deepseek open-sora awesome-llm tensorrt-llm llm-inference flash-attention paged-attention flash-attention-2 flash-attention-3

Created 2023-08-27

390 commits to main branch, last one 20 hours ago

cortex.cpp janhq

108

1.9k

apache-2.0

15

Run and customize Local LLMs.

gguf onnx llamacpp onnxruntime tensorrt-llm

Created 2023-09-11

1,479 commits to dev branch, last one 13 hours ago

WhisperLive collabora

247

1.8k

mit

30

A nearly-live implementation of OpenAI's Whisper.

obs openai whisper tensorrt dictation translation tensorrt-llm text-to-speech whisper-tensorrt voice-recognition

Created 2023-05-04

437 commits to main branch, last one 7 days ago

WhisperS2T shashikg

28

285

mit

13

An Optimized Speech-to-Text Pipeline for the Whisper Model Supporting Multiple Inference Engine

asr vad whisper tensorrt tensorrt-llm deep-learning speech-to-text speech-recognition voice-activity-detection

Created 2023-12-16

81 commits to main branch, last one about a month ago

optimum-benchmark huggingface

41

233

apache-2.0

5

🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Optimum's hardware optimizations & quantization schemes.

pytorch openvino benchmark onnxruntime tensorrt-llm neural-compressor text-generation-inference

Created 2023-04-26

688 commits to main branch, last one 17 hours ago