Wafer vs. Modal

Modal rents you serverless GPUs billed per second and you write and tune the inference code, while Wafer delivers an endpoint tuned to your TTFT, throughput, and end-to-end latency targets and keeps tuning it as traffic changes.

Introduction and methodology note

Wafer is an inference platform for performance-sensitive production workloads: products where time to first token (TTFT), output tokens per second (TPS), or end-to-end (E2E) response time decides the user experience or the unit economics. It profiles your real request mix, builds a dedicated endpoint around your model and performance targets, and keeps testing batching, decoding, quantization, engine, kernel, and hardware configurations as traffic changes. It also runs a pay-per-token serverless API, and reports 10T+ tokens of continual inference.

Modal is a serverless AI infrastructure platform for inference, batch jobs, training, sandboxes, and notebooks, where you define containers and GPU selection in Python and pay per second. These are different categories. Modal gives you compute and you bring the serving stack (its LLM example runs vLLM on one H200), so TTFT, TPS, and E2E are whatever your configuration achieves. Wafer gives you an endpoint and owns the serving stack. Teams choosing between them are usually deciding whether to staff performance tuning in-house.

Methodology. Wafer figures come from Wafer's own published pages, linked inline, and were re-verified on October 29, 2026. Modal figures come from Modal's own pricing and documentation pages, are stated as plain text, and were re-verified on October 29, 2026. Modal publishes no TTFT, TPS, or E2E benchmarks on the pages reviewed. Performance claims are self-reported by Wafer or by customers who measured them. Test both on your own prompts, output lengths, and concurrency.

TL;DR

• Modal sells GPU time (H200 at $0.001261 per second, about $4.54 per hour derived), so your cost per token is set by how many tokens per second you sustain per GPU, which depends on how well you tune.
• Wafer publishes the throughput side of that equation: about 15,000 tok/s per 8x MI355X node (about 1,873 tok/s per GPU) at TTFT p95 under 4 s, plus customer TTFT and turn-latency results; Modal wins on flexibility (any model, custom code, batch, training, sandboxes).
• Choose Wafer when a TTFT, TPS, or E2E target is the requirement and you do not want to own the serving stack; choose Modal when you need full control or broader workloads than LLM serving.

Quick comparison

Where the vendors differ: who owns the performance work

On Modal, throughput per GPU sets cost per token. Break-even math (derived): one H200 at $4.5396 per hour must sustain 4.5396 / 3.39 = 1.34M output tokens per hour, or about 372 tok/s around the clock, to match Wafer's $3.39 per 1M output tokens on GLM-5.3. For scale, Wafer's own benchmark sustains about 1,873 tok/s per GPU on Qwen3.6-35B-A3B on MI355X at saturation, about 5x that break-even. Different model and hardware, and GLM-5.3 will not fit one GPU, so treat this as a picture of how tuned throughput drives per-token cost, not a like-for-like comparison. It ignores input tokens, idle time, and your engineers' time.

Why teams choose Wafer over Modal

Throughput per GPU without staffing the tuning

Modal hands you a GPU and a container; cold-start compilation, concurrency targets, speculative decoding, and quantization are yours. Wafer's Qwen3.6-35B-A3B benchmark shows what a tuned stack sustains on one 8x MI355X node:

Success stayed at 100% through 50 RPS. Wafer's post derives about $0.44 per 1M output tokens in GPU cost at saturation (8 x $2.95 per GPU-hour = $23.60 per hour, divided by about 54M output tokens per hour). That is Wafer's hardware-cost estimate, not a price.

A TTFT target you can write into a contract

Neon Health moved GLM-5.1 to a Wafer dedicated endpoint and cut p50 client-observed TTFT from 800 ms to 550 ms at about 25% higher peak load, against a 600 ms target, with a signed BAA and US-only data residency. Modal's pages reviewed state no latency targets or SLA.

E2E budgets met without your own cache layers

Brilliant needs its tutor to respond in under 2 seconds and reports about 250 ms TTFT and 300+ output tok/s on Wafer. A 300-token reply lands at about 0.25 + 300 / 300 = 1.25 s E2E (derived). That margin let Brilliant remove its prefetching layer and cut inference cost 50%. Figures are customer-reported.

If a TTFT, TPS, or E2E number is your acceptance test, start a dedicated endpoint benchmark with Wafer and run it against your current Modal results.

Best use cases for Wafer

Voice agents and real-time assistants

Short turns are TTFT-bound, and cold starts land directly in E2E. Wafer tunes the endpoint to your p50 TTFT target and keeps it warm for your traffic. In YC's voice simulator, GLM-5.2 on Wafer averaged 379 ms LLM latency per turn across 4,168 turns.

How Modal compares: you can reach low latency, but you tune the server and manage cold starts (tens of seconds to minutes when compiling) yourself.

High-throughput serving where cost tracks tokens per GPU

On GLM-5.2, Wafer reports 2,626 tok/s per node at 2.4 RPS on a 20K-in, 1K-out workload with a 60% cache hit rate, at 0.81 s p50 TTFT.

How Modal compares: per-second billing rewards the same throughput, but reaching it is your engineering work.

Teams that would rather ship product than staff inference

Wafer's engagement starts from your model, traffic shape, and SLO. In one DigitalOcean engagement, Wafer raised DeepSeek V3.2 aggregate throughput at concurrency 64 from 548 to 2,165 tok/s (internal testing).

How Modal compares: it is the right tool if you already have engineers who want control over the container, GPU, and serving flags.

Best use cases for Modal

Custom code, any model, and non-LLM workloads

Modal runs inference, batch jobs, training and fine-tuning, sandboxes for AI-generated code, and GPU notebooks, all defined in Python. If your workload is not a plain LLM endpoint, Wafer does not compete here.

Bursty and experimental compute

Per-second billing, a $30 monthly free credit on Starter, and startup and academic credit programs make Modal cheap to experiment on. Starter allows 10 concurrent GPUs and Team allows 50.

Platform overview of Wafer

Dedicated endpoints with performance targets

You supply model, traffic shape, and SLO (a TTFT, TPS, or E2E target); Wafer builds the endpoint and keeps optimizing it after launch. Dedicated Inference carries a 99.9% monthly availability SLA with credits of 10%, 15%, or 30%, capped at 40% of the monthly amount, per the SLA page. The public SLA lists no latency target.

Continual inference loop

Wafer profiles whether latency or throughput is limited by scheduling, decoding, kernels, memory pressure, or hardware fit, tests candidates, deploys the measured winner, and repeats as traffic changes. Each candidate must keep correctness before it ships. On Kimi K3, Wafer reports 952 tok/s per 8x MI355X node on a 1,024-in, 400-out workload.

Serverless OpenAI-compatible API

The serverless docs list POST /v1/chat/completions, /v1/responses, /v1/messages, and GET /v1/models. Busy models return 429 server_overloaded with Retry-After, and errors can arrive mid-stream, per the errors page.

from openai import OpenAI
client = OpenAI(base_url="https://pass.wafer.ai/v1", api_key="YOUR_WAFER_API_KEY")
r = client.chat.completions.create(model="GLM-5.3",
    messages=[{"role": "user", "content": "Hello!"}], max_tokens=128)

Models and capabilities

The models page documents GLM-5.3: 1,048,576-token context, tools, JSON schema output, n greater than 1, no vision, and reasoning efforts none, low, high, max. Reasoning tokens bill as output and add to TTFT and E2E, so set the effort per route.

Zero data retention

Send Wafer-ZDR: required. Models without support return HTTP 422 model_zdr_not_supported. See the ZDR docs.

curl https://pass.wafer.ai/v1/chat/completions \
  -H "Authorization: Bearer $WAFER_API_KEY" -H "Wafer-ZDR: required" \
  -d '{"model":"GLM-5.3","messages":[{"role":"user","content":"Hi"}]}'

Measuring TTFT, TPS, and E2E on your own prompts

Run the same script against your Modal endpoint and Wafer. TTFT is time to the first content token; TPS is output tokens over decode time.

import time
t0 = time.perf_counter(); first = None; n = 0
for c in client.chat.completions.create(model="GLM-5.3", messages=msgs,
        stream=True, stream_options={"include_usage": True}):
    if c.choices and c.choices[0].delta.content and first is None: first = time.perf_counter()
    if c.usage: n = c.usage.completion_tokens
end = time.perf_counter()
print(f"TTFT {first-t0:.3f}s  TPS {n/(end-first):.0f}  E2E {end-t0:.3f}s")

How to choose

Switching from Modal to Wafer

1. Create a Wafer key in the app and add credits.
2. Point your client at Wafer. If your Modal vLLM server already exposes an OpenAI-compatible API, only base_url and model change.
3. Re-run quality and performance tests with the measurement script above; you are likely changing models or serving configuration.
4. Add 429 retry handling for server_overloaded.

Before:

client = OpenAI(base_url="https://YOUR-MODAL-ENDPOINT/v1", api_key=KEY)
r = client.chat.completions.create(model="google/gemma-4-26B-A4B-it", messages=msgs)

After:

client = OpenAI(base_url="https://pass.wafer.ai/v1", api_key=WAFER_KEY)
r = client.chat.completions.create(model="GLM-5.3", messages=msgs)

What to watch:

• Custom code: anything beyond a chat endpoint (pre- and post-processing, custom weights, batch pipelines) has no Wafer equivalent on the pages reviewed.
• Retired infrastructure: you stop paying per GPU-second, so check that your sustained tokens per GPU make per-token billing cheaper.
• Partial migration: often best. Keep Modal for batch, training, and sandboxes; move the TTFT-, TPS-, or E2E-critical endpoint to Wafer.

Neon's case states a dedicated endpoint can reach production in under two weeks. That is a stated capability, not a guarantee.

FAQ

Is Wafer cheaper than Modal? It depends on the tokens per second you sustain per GPU. One H200 at about $4.54 per hour needs about 372 tok/s of output around the clock to match $3.39 per 1M output tokens (derived above), before input tokens, idle time, or engineering. GLM-5.3 does not fit one GPU, so this is a hypothetical.

Is Wafer faster than Modal? Modal's speed is whatever your server achieves; it publishes no TTFT or TPS figures. Wafer publishes 337 ms single-stream TTFT and 430 tok/s on Qwen3.6-35B-A3B, and its customers report 550 ms p50 TTFT (Neon) and 379 ms average turn latency (YC).

How fast is Modal's cold start? Modal's cold-start guide says containers boot in about one second but gives no figure for LLM servers. Its own vLLM example says compilation takes tens of seconds to a few minutes, about ten seconds when loaded from cache. Any cold start on the request path adds directly to E2E.

What throughput can a Wafer endpoint sustain? Wafer reports about 15,000 tok/s of aggregate output from one 8x MI355X node on Qwen3.6-35B-A3B at TTFT p95 under 4 s, and 2,626 tok/s per node on GLM-5.2. Both are Wafer's own measurements.

Which hardware does Wafer use? Per its AI info page, NVIDIA and AMD GPUs. Its published benchmarks run on AMD MI355X. Validate availability per model.

Does Wafer's SLA cover latency? The public SLA covers 99.9% monthly availability on Dedicated Inference and lists no latency target. Neon reports a TTFT target written into its own contract.

Want TTFT, TPS, and E2E numbers on your own traffic? Get started on Wafer and benchmark a dedicated endpoint against your current Modal results.

Related Articles

• Oct 2026: Wafer vs. Together AI
• Oct 2026: Wafer vs. Cerebras
• Oct 2026: Wafer vs. Fireworks AI