Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.
7.0
Rating
0
Installs
AI & LLM
Category
Excellent skill for TensorRT-LLM optimization. The description is comprehensive and clearly specifies when to use this skill (production NVIDIA GPU deployment, high throughput needs, quantization). Task knowledge is strong with concrete code examples for installation, basic inference, serving, quantization, and multi-GPU deployment. Structure is well-organized with a clean overview and references to separate files for advanced topics. Novelty is moderate-to-good: while TensorRT-LLM setup can be complex (Docker, CUDA dependencies, compilation), some patterns like basic inference are straightforward; the skill adds most value for advanced scenarios (multi-GPU, quantization tuning, production serving) where configuration complexity and performance optimization require significant expertise. Minor improvement areas: could include more troubleshooting patterns and quantization comparison tables. Overall, a high-quality skill that would meaningfully reduce token costs for production LLM deployment on NVIDIA hardware.
zechenzhangAGI
Skill Author
Loading SKILL.md…
