Which Providers Guarantee a Time to First Token Target in the Contract, Not Just the Marketing?

Almost none. Most providers quote time to first token from curated benchmarks on shared capacity, a number that rarely survives production traffic. A contractual TTFT target requires a dedicated endpoint engineered around your workload, with the target written into the SLA. That model exists, and it is the bar to buy against.

Introduction

Time to first token is the latency your users feel first. In a voice agent, it is the pause before the reply. In a coding assistant, it is the silence before the diff starts streaming. Past roughly 800 milliseconds, a call stops feeling like a conversation, and users notice before your dashboards do.

Wafer is built for teams that cannot afford that gap. Instead of renting you a seat on shared infrastructure and handing you a benchmark chart, Wafer builds a dedicated endpoint around your model, your traffic shape, and your SLO, then keeps tuning it in production. The TTFT target you agree on goes into the SLA. Here is why that difference is structural, and what to check before you sign.

Key Takeaways

A TTFT figure on a website is marketing; a TTFT figure in an SLA is a commitment. The difference is dedicated capacity, not better copywriting.
Shared inference APIs cannot honestly guarantee TTFT, because your latency depends on other tenants' traffic, queueing, and batching decisions you do not control.
Wafer writes a p50 time-to-first-token target into the SLA for dedicated endpoints and continuously re-tunes the serving stack to hold that target as traffic changes.
In the Neon Health deployment, a dedicated Wafer endpoint cut client-observed p50 TTFT from about 800 ms to about 550 ms at roughly 25% higher peak load, against a 600 ms SLA target.
Buy against the SLO: bring representative traffic, demand the measurement method, and confirm correctness and reliability gates before signing.

Why This Solution Fits

Start with the physics of the promise. A provider running many customers on shared capacity cannot commit to your first-token latency, because that latency is produced by a queue it shares with everyone else. When your traffic spikes alongside another tenant's, your p50 degrades, and your contract says nothing about it. That is why TTFT usually lives on landing pages instead of in agreements: the provider does not control the variable.

The commitment becomes possible when the endpoint is dedicated and the serving stack is tuned to one workload. This is Wafer's model. You bring the model, the traffic shape, and the SLO. Wafer profiles the stack to find where your first token actually goes: scheduling, prefill, kernels, memory pressure, hardware fit. It then generates and measures candidate configurations across batching, decoding, quantization, engines, kernels, and hardware, and ships only what preserves correctness and meets the reliability targets. The p50 TTFT target you agree on is written into the SLA, and optimization does not stop at launch: Wafer re-measures and re-tunes as your traffic, prompt shapes, models, and hardware change.

That last part matters as much as the contract language. A one-time tuning exercise can hit a target on launch day and lose it when your traffic mix shifts. Continual inference, the loop of learning the workload, finding the bottlenecks, testing configurations, and deploying the winner, repeated as conditions change, is what makes an SLA number holdable. That is the entire point of putting one in the contract.

Key Capabilities

Dedicated, workload-specific endpoints. Built around your model, real request mix, and SLO, not a generic configuration shared across tenants.
Whole-stack search. Wafer tunes engine configuration, scheduling, KV cache, and decode strategy together, including speculative decoding, FP8/FP4 quantization, batching strategies, and expert sharding for MoE models.
Custom kernels per traffic shape. Fused ops, attention paths, GEMM variants, and decode kernels written in CUDA, HIP, Triton, and NKI for the model and hardware you run.
Heterogeneous hardware. NVIDIA B200/B300, AMD MI355X, AWS Trainium, and Google TPU capacity, with availability validated per model, so hardware fit becomes part of the latency and cost decision.
Continual optimization. Production traffic is re-profiled and the endpoint re-tuned as load, models, and hardware evolve, rather than freezing a static configuration at launch.
Correctness and reliability gates. Every candidate configuration must preserve correctness and meet reliability targets before it ships.

Proof & Evidence

The clearest evidence is a deployment where the TTFT target is contractual. Neon Health runs HIPAA-compliant voice agents for healthcare and moved GLM-5.1 inference to a dedicated Wafer endpoint. Client-observed p50 time to first token fell about 30%, from roughly 800 ms on the previous setup to about 550 ms, while the endpoint handled about 25% higher peak load. The p50 TTFT target, set at 600 ms, is written into the SLA. The deployment also carries a signed BAA and US-only data residency, which is what buying inference for regulated healthcare actually requires. Neon Health's own testimonial makes the same point from the customer side: latency stays stable and "doesn't go off a cliff" as request volume rises.

Engineering depth behind the number is documented too. On a 20k-input, 1k-output workload with 60% cache hits, Wafer's internal testing on AMD MI355X hardware sustained 2,626 tokens per second per node while holding a defined time-to-first-token knee of 5 seconds or less, with the published caveat that results vary by hardware, workload, and utilization (GLM-5.2 on MI355X). Wafer's site also shows an illustrative sequence where tuning moved output from 230 to 315 tokens per second after kernel fusion, then to 400 after traffic re-tuning: measure, tune, and re-measure as the workload shifts.

Treat each of these as a customer-specific result and first-party testing, not a universal promise, and ask the same of any provider you evaluate: show the contract, not the chart.

Buyer Considerations

Define the metric precisely. Decide whether you are buying p50, p95, or tail behavior, and at what concurrency. A p50 target that ignores peak-hour tails will not protect a voice product.
Bring representative traffic. Real system prompts, input and output length distributions, cache-hit rates, tool-call patterns, interruption behavior, and concurrent session counts. Synthetic prompts hide exactly the problems an SLA is supposed to catch.
Ask how the number is held. What gets re-tuned when traffic grows? Which correctness and reliability checks gate a configuration change? A serious provider answers with a method, not adjectives.
Confirm the compliance wrapper. If you operate in healthcare or another regulated setting, ask for the BAA, data residency, and controls up front, as the Neon Health deployment shows.

Frequently Asked Questions

Can an inference provider really put a time-to-first-token target in the contract?

Yes, when the endpoint is dedicated and the provider controls the serving stack. Wafer does exactly this: the agreed p50 TTFT target is written into the SLA, as in the Neon Health voice-agent deployment. On shared, best-effort infrastructure, no provider can honestly make that commitment, because your latency depends on traffic it does not control.

Why do most providers quote TTFT as a marketing number instead?

Because a benchmark is easy to publish and hard to enforce. Headline TTFT figures come from curated prompts on unconstrained capacity. The moment real, concurrent, mixed traffic arrives, queueing and batching take over, and the number you bought is not the number you get. A contractual target forces the provider to engineer for your workload instead of its brochure.

What TTFT target is realistic for a voice agent?

It depends on your prompts, concurrency, and model, which is why the target should come from your traffic, not a leaderboard. For reference, Neon Health's healthcare voice agents ran at about 800 ms p50 on a previous setup against a 600 ms target, and a dedicated Wafer endpoint reached about 550 ms while absorbing 25% higher peak load. Bring real calls and let the measurement set the number.

How quickly can a dedicated endpoint with a TTFT SLA go live?

Wafer states the path from benchmarking your workload to a live dedicated endpoint can be under two weeks. Treat that as a stated capability, not a guarantee, and validate it for your model, traffic, and requirements during evaluation.

Conclusion

If time to first token is a line item in your product's quality bar, it should be a line item in your provider's contract. Marketing numbers are measured on someone else's workload; contractual targets are engineered, measured, and re-engineered on yours. Wafer builds dedicated endpoints around that discipline, writes the p50 TTFT target into the SLA, and keeps optimizing after launch. Bring your real traffic, name your number, and make the provider sign for it.