AI Insights: Latency & Cost Optimization · part 10 of 17
GPT-5.1 speed ablation: what actually makes the API faster
Prompt compression, output caps, reasoning_effort, and the Azure TTFT floor
The question
If the paid GPT-5.1 path stays (as fallback or primary), how much faster and cheaper can the identical call get?
Output length is the only real lever: a terse prompt cut wall time 32% and cost 33% with quality held at 8.0.
What worked
Telling the model to write "exactly 6 concise insights" (variant S4) cut wall time from 13.9 s to 9.5 s and cost by a third, with the Opus judge score holding at 8.0. Output tokens are both the latency and the cost — the model writes at a fixed pace and output is priced 8x input, so every word not written is saved twice.
More aggressive compression (S5/S6, ~6.4 s) costs about one judge point — usable as an optional fast tier, not as the default.
What backfired
The reasoning_effort trap on Azure GPT-5.1: "minimal" is a no-op, and "low" actually enables reasoning — TTFT jumped 1.5 s → 13.3 s and cost rose 66%. Never set it. Any reasoning/verbosity parameter also makes the API reject temperature/top_p.
Azure's prompt cache was already active (2,688 tokens cached per call) — confirming the sub-experiment 6 side-finding that keeping the shared prompt identical and in front pays on the paid API too.