2 results found Sort:

intel
neural-compressor intel
248
2.1k
apache-2.0
34
SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime
awq fp4 gptq int4 int8 pruning mxformat sparsity sparsegpt auto-tuning smoothquant quantization low-precision large-language-models knowledge-distillation post-training-quantization quantization-aware-training
Created 2020-07-21
3,480 commits to master branch, last one a day ago
ModelTC
llmc ModelTC
9
116
apache-2.0
8
This is the official PyTorch implementation of "LLM-QBench: A Benchmark Towards the Best Practice for Post-training Quantization of Large Language Models", and also an efficient LLM compression tool w...
awq llm opt ptq tool llama falcon llama2 llama3 pruning internlm benchmark internlm2 omniquant deployment evaluation smoothquant quantization large-language-models post-training-quantization
Created 2024-03-06
62 commits to main branch, last one 4 days ago