Research AI & Systems
LLM Inference Latency and Cost Efficiency Benchmark 2025
1. Speculative decoding reduced latency by up to 42% on structured output tasks. 2. Small quantized models achieved sub-150ms TTFB suitable for interactive UI flows.
S
Saloni Verma • Jan 26, 2026 • 15 min read