3 results found Sort:

intel
neural-compressor intel
257
2.2k
apache-2.0
33
SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime
awq fp4 gptq int4 int8 pruning mxformat sparsity sparsegpt auto-tuning smoothquant quantization low-precision large-language-models knowledge-distillation post-training-quantization quantization-aware-training
Created 2020-07-21
3,628 commits to master branch, last one 9 hours ago
ModelTC
llmc ModelTC
37
332
apache-2.0
9
[EMNLP 2024 Industry Track] This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit".
awq llm lvlm tool vllm llama3 quarot mixtral pruning lightllm benchmark internlm2 omniquant spinquant deployment evaluation smoothquant quantization large-language-models post-training-quantization
Created 2024-03-06
404 commits to main branch, last one a day ago
intel
auto-round intel
23
263
apache-2.0
11
Advanced Quantization Algorithm for LLMs/VLMs. This is official implementation of "Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs"
awq gptq int4 rounding quantization neural-compressor
Created 2024-01-04
355 commits to main branch, last one 11 hours ago