Wafer vs. Groq

Groq serves a short model catalog at very high published decode speeds on its own hardware, while Wafer builds a dedicated endpoint for the open model you choose and tunes it to your TTFT, throughput, and end-to-end latency targets.

Introduction and methodology note

Wafer is an inference platform for performance-sensitive production workloads: products where time to first token (TTFT), output tokens per second (TPS), or end-to-end (E2E) response time decides the user experience or the unit economics. It profiles your real request mix, builds a dedicated endpoint around your model and performance targets, and keeps testing batching, decoding, quantization, engine, kernel, and hardware configurations as traffic changes. It also runs a pay-per-token serverless API, and reports 10T+ tokens of continual inference.

Groq runs an OpenAI-compatible API on what it calls the LPU, its inference processor, and describes an LPX product that works alongside NVIDIA GPUs. Both serve Chat Completions and Responses endpoints. They diverge on which number gets optimized. Groq competes on single-stream decode speed for the models in its catalog. Wafer tunes whichever of the three metrics your product depends on: TTFT for short turns, TPS for long outputs and throughput per node, and E2E (roughly TTFT plus output tokens divided by TPS, plus queueing) for the full response.

Methodology. Wafer figures come from Wafer's own published pages, linked inline, and were re-verified on October 29, 2026. Groq figures come from Groq's own console documentation, are stated as plain text, and were re-verified on October 29, 2026. Groq's pricing page did not render per-token rates, so model prices come from its models page and are labeled that way. TTFT, TPS, and E2E claims are self-reported by each vendor or by customers who measured them, usually on different models and workload shapes. Test both on your own prompts, output lengths, and concurrency.

TL;DR

• Groq publishes higher single-stream decode speeds (1,000 tok/s for gpt-oss-20b, 500 for gpt-oss-120b) than Wafer (430 tok/s on Qwen3.6-35B-A3B), so long outputs finish sooner on Groq's catalog models.
• Wafer publishes the numbers Groq does not: TTFT (239 to 337 ms p50 at 1 to 8 RPS), aggregate throughput per node (about 15,000 tok/s), and customer-measured turn latency (379 ms average across 4,168 turns at YC).
• Choose Wafer when you need a specific open model held to TTFT, TPS, or E2E targets under your concurrency; choose Groq when raw decode speed on its catalog is the requirement.

Quick comparison

Where the vendors differ: decode speed vs the whole response

Groq's differentiator is decode speed. The table below shows what that buys at two output lengths. Decode time is output tokens divided by published TPS (derived). TTFT is left out because Groq does not publish it, and the models differ, so read it as a sensitivity check, not a benchmark.

On a 1,500-token output, Groq's gpt-oss-20b finishes decoding about 2 s sooner. On a 100-token turn, the decode gap is 30 to 130 ms, so TTFT and queueing under load decide E2E. Wafer's single-stream TTFT is 337 ms on the same benchmark (derived E2E: 0.337 + 0.23 = 0.57 s for 100 tokens); Groq's is not published.

Groq is ahead on single-stream decode and on gpt-oss per-token price. Wafer wins when TTFT under concurrency, a specific model, or capacity sized to your traffic matters more than peak decode on one stream.

Why teams choose Wafer over Groq

Throughput and TTFT held under load, not just on one stream

Single-stream decode is one point on the curve. Wafer's Qwen3.6-35B-A3B benchmark publishes the whole sweep on one 8x MI355X node:

Success stayed at 100% through 50 RPS. Groq publishes per-model decode speed but no TTFT or aggregate throughput on the pages reviewed, so you cannot read its curve from the docs.

E2E latency measured in production

YC ran GLM-5.2 on Wafer against GPT-4.1 mini on OpenAI and Gemma 4 31B on Cerebras across 4,168 LLM turns in its voice office-hours simulator. YC's case reports 379 ms average LLM latency on Wafer, 31% lower than OpenAI and 44% lower than Cerebras, and conversations 2.5 minutes longer on average. Groq was not in the test, and the models differ. Neon Health cut p50 client-observed TTFT from 800 ms to 550 ms on GLM-5.1 at about 25% higher peak load, freeing 250 ms of an 800 ms voice turn budget.

You pick the model and the capacity

Groq's models page lists four text models in production, and its rate-limits page lists base Developer-plan limits for gpt-oss-120b of 30 RPM and 8,000 TPM (its models page shows 1K RPM and 250K TPM, so confirm the limits on your account). A dedicated Wafer endpoint is built around the open model your product needs and sized to your traffic; Wafer has published results on GLM, Qwen, Kimi, and DeepSeek models. Brilliant reports about 250 ms TTFT and 300+ output tokens per second on a GLM-5.2 endpoint, and a 50% inference cost reduction after removing its prefetching layer. Figures are customer-reported.

If a TTFT, TPS, or E2E number is your acceptance test, start a dedicated endpoint benchmark with Wafer and run it against your current Groq results.

Best use cases for Wafer

Voice agents and real-time assistants

Short voice turns are TTFT-bound: at 100 output tokens, decode takes 0.1 to 0.25 s on either vendor, and the rest of the budget is TTFT and queueing. Wafer writes a p50 TTFT target into the engagement, as in Neon's 600 ms contract target.

How Groq compares: its decode speeds are higher on supported models, which shortens long outputs more than short turns, and it does not publish TTFT.

High-throughput serving where cost tracks tokens per node

When the metric is aggregate TPS at a TTFT ceiling, Wafer tunes for tokens per node. Its GLM-5.2 post reports 2,626 tok/s per node at 2.4 RPS on a 20K-in, 1K-out workload with a 60% cache hit rate, at 0.81 s p50 and 2.22 s p95 TTFT.

How Groq compares: its batch tier is 50% off with a 24-hour to 7-day window, which is cheaper for offline throughput where latency does not matter.

Coding and agentic products

Agent chains add E2E latency across calls, so prefix caching and speculative decoding tuned to your prompts pay off on every step. Wafer's Responses and Messages endpoints accept Codex and Claude Code style clients.

How Groq compares: it also serves a Responses endpoint, and its high decode speed helps long code generations; its small catalog limits which coding models you can run.

Best use cases for Groq

Maximum single-stream decode on small and mid-size models

If long outputs from gpt-oss-20b or llama-3.1-8b-instant matter most, Groq publishes 1,000 and 560 tok/s. Wafer does not publish single-stream numbers in that range.

Speech and batch workloads

Groq lists whisper-large-v3 at $0.111 per hour of audio and whisper-large-v3-turbo at $0.04, and its batch tier runs at a 50% discount. Wafer has no speech models on the pages reviewed.

Platform overview of Wafer

Dedicated endpoints with performance targets

You supply model, traffic shape, and SLO (a TTFT, TPS, or E2E target); Wafer builds the endpoint and keeps optimizing it after launch. Dedicated Inference carries a 99.9% monthly availability SLA with credits of 10%, 15%, or 30%, capped at 40% of the monthly amount, per the SLA page. The public SLA lists no latency target; Neon reports a TTFT target in its own contract.

Continual inference loop

Wafer profiles whether latency or throughput is limited by scheduling, decoding, kernels, memory pressure, or hardware fit, tests candidate configurations, deploys the measured winner, and repeats as traffic changes. Each candidate must keep correctness before it ships. In one DigitalOcean engagement, the loop raised DeepSeek V3.2 aggregate throughput at concurrency 64 from 548 to 2,165 tok/s (internal testing).

Serverless OpenAI-compatible API

The serverless docs list POST /v1/chat/completions, /v1/responses, /v1/messages, and GET /v1/models. Busy models return 429 server_overloaded with Retry-After, and errors can arrive mid-stream, per the errors page.

from openai import OpenAI
client = OpenAI(base_url="https://pass.wafer.ai/v1", api_key="YOUR_WAFER_API_KEY")
r = client.chat.completions.create(model="GLM-5.3",
    messages=[{"role": "user", "content": "Hello!"}], max_tokens=128)

Models and capabilities

The models page documents GLM-5.3: 1,048,576-token context, tools, JSON schema output, n greater than 1, no vision, and reasoning efforts none, low, high, max. Reasoning tokens bill as output and count against TTFT and E2E, so set the effort per route.

Zero data retention

Send Wafer-ZDR: required. Models without support return HTTP 422 model_zdr_not_supported. See the ZDR docs.

curl https://pass.wafer.ai/v1/chat/completions \
  -H "Authorization: Bearer $WAFER_API_KEY" -H "Wafer-ZDR: required" \
  -d '{"model":"GLM-5.3","messages":[{"role":"user","content":"Hi"}]}'

Measuring TTFT, TPS, and E2E on your own prompts

Run the same script against both base URLs. TTFT is time to the first content token; TPS is output tokens over decode time.

import time
t0 = time.perf_counter(); first = None; n = 0
for c in client.chat.completions.create(model="GLM-5.3", messages=msgs,
        stream=True, stream_options={"include_usage": True}):
    if c.choices and c.choices[0].delta.content and first is None: first = time.perf_counter()
    if c.usage: n = c.usage.completion_tokens
end = time.perf_counter()
print(f"TTFT {first-t0:.3f}s  TPS {n/(end-first):.0f}  E2E {end-t0:.3f}s")

If a model returns no usage chunk, count output tokens with its tokenizer instead.

How to choose

Switching from Groq to Wafer

1. Create a Wafer key in the app and add credits.
2. Change base_url and model.
3. Re-run quality and performance tests with the measurement script above; you are likely changing models, so compare TTFT, TPS, and E2E at your real output lengths.
4. Add 429 retry handling for server_overloaded.

Before:

client = OpenAI(base_url="https://api.groq.com/openai/v1", api_key=GROQ_KEY)
r = client.chat.completions.create(model="openai/gpt-oss-120b", messages=msgs)

After:

client = OpenAI(base_url="https://pass.wafer.ai/v1", api_key=WAFER_KEY)
r = client.chat.completions.create(model="GLM-5.3", messages=msgs)

What to watch:

• Service tiers: Groq's on_demand, flex, and auto tiers have no Wafer equivalent; a dedicated endpoint replaces them.
• Speech: Whisper has no Wafer equivalent on the pages reviewed.
• Decode speed: expect lower single-stream TPS than Groq's published numbers. Routes with long outputs can get slower E2E; routes with short outputs are decided by TTFT.
• Partial migration: often best. Keep Groq for speech, batch, and long-output routes on its fast models; move TTFT-critical and throughput-at-load paths to Wafer.

Neon's case states a dedicated endpoint can reach production in under two weeks. That is a stated capability, not a guarantee.

FAQ

Is Wafer faster than Groq? On single-stream decode, no: Groq publishes 500 tok/s on gpt-oss-120b against Wafer's 430 tok/s on Qwen3.6-35B-A3B. On TTFT, Wafer publishes 337 ms single-stream and Groq publishes no figure. On E2E, it depends on output length: decode dominates long outputs, TTFT dominates short ones. The models differ, so benchmark your own prompts.

What throughput can a Wafer endpoint sustain? Wafer reports about 15,000 tok/s of aggregate output from one 8x MI355X node on Qwen3.6-35B-A3B at TTFT p95 under 4 s, and 2,626 tok/s per node on GLM-5.2 with a 20K-in, 1K-out workload. Both are Wafer's own measurements.

Is Wafer cheaper than Groq? Not per token on gpt-oss: Groq lists $0.15 input and $0.60 output for gpt-oss-120b. Wafer's GLM-5.3 is $0.22 input and $3.39 output, a different and larger model.

Is Wafer OpenAI-compatible? Yes. Set base_url to https://pass.wafer.ai/v1. Wafer also serves Responses and Messages clients.

Which hardware does Wafer use? Per its AI info page, NVIDIA and AMD GPUs. Its published benchmarks run on AMD MI355X. Validate availability per model.

Does Wafer's SLA cover latency? The public SLA covers 99.9% monthly availability on Dedicated Inference, with service credits, and lists no latency target. Neon reports a TTFT target written into its own contract.

Want TTFT, TPS, and E2E numbers on your own traffic? Get started on Wafer and benchmark a dedicated endpoint against your current Groq results.

Related Articles

• Oct 2026: Wafer vs. Cerebras