9 results found Sort:

intel
neural-compressor intel
263
2.4k
apache-2.0
32
SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime
awq fp4 gptq int4 int8 pruning mxformat sparsity sparsegpt auto-tuning smoothquant quantization low-precision large-language-models knowledge-distillation post-training-quantization quantization-aware-training
Created 2020-07-21
3,732 commits to master branch, last one a day ago
shm007g
LLaMA-Cult-and-More shm007g
24
444
mit
33
Large Language Models for All, 🦙 Cult and More, Stay in touch !
gpt llm ggml gpt4 gptq llama alpaca vicuna chatgpt loralib pytorch deepspeed tensorflow transformers
Created 2023-03-30
27 commits to main branch, last one about a year ago
intel
auto-round intel
31
407
apache-2.0
11
Advanced Quantization Algorithm for LLMs/VLMs.
awq gptq int4 rounding quantization neural-compressor
Created 2024-01-04
424 commits to main branch, last one 6 days ago
BobaZooba
xllm BobaZooba
21
400
apache-2.0
4
🦖 X—LLM: Cutting Edge & Easy LLM Finetuning
gpt llm gptq gpt-4 llama torch alpaca llama2 openai vicuna zephyr chatgpt mistral pytorch cerebras bitsandbytes deep-learning deep-neural-networks large-language-models
Created 2023-11-10
62 commits to main branch, last one about a year ago
ModelCloud
GPTQModel ModelCloud
54
384
apache-2.0
4
Production ready LLM model compression/quantization toolkit with hw accelerated inference support for both cpu/gpu via HF, vLLM, and SGLang.
gptq peft vllm sglang optimum quantization transformers
Created 2024-06-17
2,112 commits to main branch, last one a day ago
1b5d
llm-api 1b5d
27
168
mit
4
Run any Large Language Model behind a unified API
llm gptq llama python chatgpt llamacpp langchain huggingface llm-inference machine-learning
Created 2023-04-02
31 commits to main branch, last one about a year ago
chenhunghan
ialacol chenhunghan
17
144
mit
3
🪶 Lightweight OpenAI drop-in replacement for Kubernetes
ai gpu llm cuda ggml gptq helm openai python llamacpp langchain kubernetes cloudnative llm-serving llm-inference
This repository has been archived (exclude archived)
Created 2023-05-23
154 commits to main branch, last one about a year ago
abhinand5
gptq_for_langchain abhinand5
9
40
gpl-3.0
2
A guide about how to use GPTQ models with langchain
ai gpt llm gptq wizardlm langchain quantization language-model
Created 2023-05-11
8 commits to main branch, last one about a year ago
ziwang-com
zero-lora ziwang-com
3
31
unknown
1
zero零训练llm调参
gpt llm gptq lora llama
Created 2023-05-17
40 commits to main branch, last one about a year ago