speculative-decoding

8.1

173

285

Accelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques. Use when optimizing inference speed (1.5-3.6× speedup), reducing latency for real-time applications, or deploying models with limited compute. Covers draft models, tree-based attention, Jacobi iteration, parallel token generation, and production deployment strategies.

inference optimization

8.1

Rating

Installs

AI & LLM

Quick Review

Excellent skill covering advanced inference optimization techniques. The description clearly explains when to use speculative decoding (1.5-3.6× speedup, real-time latency reduction), making it easily invokable by a CLI agent. Task knowledge is comprehensive with complete code examples for all three main approaches (draft model, Medusa, Lookahead), installation instructions, hyperparameter tuning guidance, and production deployment patterns. Structure is clear with logical progression from quick start to advanced patterns, though SKILL.md is fairly lengthy—some advanced content could have been moved to referenced files. Novelty is strong: these are cutting-edge 2024 techniques (Medusa, Lookahead ICML 2024) that would require significant research and experimentation for an agent to implement from scratch. The skill meaningfully reduces complexity by providing ready-to-use implementations, optimal hyperparameters, and method comparison tables. Minor improvement areas: could be more concise in main file, and production considerations could be expanded.