$ ./spark-bench --sweep --all-recipes

How many tokens/sec can one or two DGX Sparks serve under parallel load?

Aggregate decode throughput, per-session speed and tail latency as concurrent sessions scale — every number a real measured run. Toggle configs, set a latency budget, read the ceiling. See what these speeds actually look like →

Configurations · model · quant · backend · up to 7 at once

Latency budget — p95 TTFT ≤ 500 ms

View

Percentile focus (chart 4)

Y scale

Temperature unit

01Throughput ↔ latency frontierx: agg tok/s · y: p95 TTFT

The money chart. Each dot = one concurrency level. Down-and-right wins. Pick the rightmost dot still under your budget line.

02Aggregate throughput vs concurrencyx: sessions (log) · y: agg tok/s

Total tok/s across all sessions. The knee is where packing more sessions stops paying off — curves end at each config's KV-cache capacity ceiling.

03Per-session throughput vs concurrencyx: sessions (log) · y: tok/s / session

What one user feels. Falls as you batch more — the cost of higher aggregate throughput.

04TTFT tail vs concurrencyx: sessions (log) · y: TTFT ms

Time-to-first-token at the focused percentile, all visible configs. Switch to p99 to see whose tail blows up first.

05GPU temperature vs concurrencyx: sessions (log) · y: °C

GPU thermal load as concurrency scales. Watch for the knee where thermal throttling may kick in.

06GPU power draw vs concurrencyx: sessions (log) · y: watts

Power envelope as concurrency scales. Shows how much headroom remains before hitting the TDP wall.