Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss. Use when GPU memory is limited, need to fit larger models, or want faster inference. Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers. Works with HuggingFace Transformers.
7.6
Rating
0
Installs
Machine Learning
Category
Excellent quantization skill with comprehensive coverage of bitsandbytes functionality. The description accurately conveys capabilities (8-bit/4-bit quantization, memory savings, formats). Task knowledge is outstanding with three complete workflows (model loading, QLoRA fine-tuning, 8-bit optimizers), concrete code examples, memory calculations, and troubleshooting. Structure is clear with quick start, workflow checklists, comparison tables, and references to external files for advanced topics. Novelty is strong—quantization significantly reduces token costs and enables tasks impossible for CLI agents (loading 70B models on consumer GPUs, QLoRA training). Minor improvement possible: could explicitly state GPU requirements upfront in description. Overall, this is a well-crafted, immediately actionable skill that provides substantial value beyond basic CLI operations.
zechenzhangAGI
Skill Author
Loading SKILL.md…
