Statistics for topic inference

A high-throughput and memory-efficient inference and serving engine for LLMs

Port of OpenAI's Whisper model in C/C++

Faster Whisper transcription with CTranslate2

Cross-platform, customizable ML solutions for live and streaming media.

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Simple server to manage compound AI

Dehydrated Ollama CLI Interface

A project that optimizes OWL-ViT for real-time inference with NVIDIA TensorRT.

Python library for YOLO small object detection and instance segmentation

Neural Network-Boosted Importance Nested Sampling for Bayesian Statistics

A high-throughput and memory-efficient inference and serving engine for LLMs

Port of OpenAI's Whisper model in C/C++

Faster Whisper transcription with CTranslate2

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

SGLang is a fast serving framework for large language models and vision language models.

The Qualcomm® AI Hub apps are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

[WIP] Context parallel attention that works with torch.compile

Simple server to manage compound AI

Python library for YOLO small object detection and instance segmentation

Dehydrated Ollama CLI Interface

A high-throughput and memory-efficient inference and serving engine for LLMs

Port of OpenAI's Whisper model in C/C++

Mastering Applied AI, One Concept at a Time

Faster Whisper transcription with CTranslate2

SGLang is a fast serving framework for large language models and vision language models.

Mastering Applied AI, One Concept at a Time

LLM KV cache compression made easy

Dehydrated Ollama CLI Interface

The Qualcomm® AI Hub apps are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

[WIP] Context parallel attention that works with torch.compile

SGLang is a fast serving framework for large language models and vision language models.

On-device Speech Recognition for Apple Silicon

Any model. Any hardware. Zero compromise. Built with @ziglang / @openxla / MLIR / @bazelbuild

The inference module for AgiBot X1.

A throughput-oriented high-performance serving framework for LLMs

A high-throughput and memory-efficient inference and serving engine for LLMs

Port of OpenAI's Whisper model in C/C++

Faster Whisper transcription with CTranslate2

SGLang is a fast serving framework for large language models and vision language models.

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

SGLang is a fast serving framework for large language models and vision language models.

scalable and robust tree-based speculative decoding algorithm

A large-scale simulation framework for LLM inference

On-device Speech Recognition for Apple Silicon

LLM KV cache compression made easy