Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.
7.0
Rating
0
Installs
AI & LLM
Category
Excellent skill documentation for llama.cpp inference. The description clearly articulates when to use this skill (CPU/Apple Silicon/non-NVIDIA hardware) vs alternatives. SKILL.md provides comprehensive quick-start commands, quantization format tables, hardware-specific build instructions, and performance benchmarks that enable a CLI agent to execute inference tasks confidently. Structure is well-organized with concise overview and proper references to detailed guides. Novelty score reflects that while CPU/edge inference is valuable, the underlying task (running LLM inference) is increasingly commoditized; however, the skill adds meaningful value by consolidating hardware-specific optimizations, quantization strategies, and deployment patterns that would otherwise require extensive research across documentation.
zechenzhangAGI
Skill Author
Loading SKILL.md…
