4 results found Sort:

intel
neural-compressor intel
248
2.1k
apache-2.0
34
SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime
awq fp4 gptq int4 int8 pruning mxformat sparsity sparsegpt auto-tuning smoothquant quantization low-precision large-language-models knowledge-distillation post-training-quantization quantization-aware-training
Created 2020-07-21
3,480 commits to master branch, last one a day ago
tpoisonooo
how-to-optimize-gemm tpoisonooo
77
555
gpl-3.0
16
row-major matmul optimization
ptx cuda int4 arm64 armv7 vulkan cuda-kernel gemm-optimization
Created 2018-10-28
158 commits to master branch, last one 9 months ago
intel
neural-speed intel
33
303
apache-2.0
8
An innovative library for efficient LLM inference via low-bit quantization
cpu fp4 fp8 gpu nf4 int1 int2 int3 int4 int5 int6 int7 int8 gaudi2 low-bit llamacpp mxformat sparsity llm-inference llm-fine-tuning
Created 2023-11-20
344 commits to main branch, last one 4 days ago
intel
auto-round intel
16
105
apache-2.0
9
SOTA Weight-only Quantization Algorithm for LLMs. This is official implementation of "Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs"
awq gptq int4 rounding weight-only quantization neural-compressor
Created 2024-01-04
208 commits to main branch, last one 2 days ago