LLM Inference Benchmark Explorer

Qwen3.5-35B-A3B on RTX PRO 6000 — FP8, vLLM, TP1 inference benchmark

Open this configuration in the LLM Inference Benchmark Explorer

ModelQwen3.5-35B-A3B
Parameters35B
Intelligence Index19.3
Agentic Index—
DeviceRTX PRO 6000
QuantizationFP8
Max C64+
Chat Capacity150
Agentic Capacity55
TP—
DP—
PP—
EnginevLLM
Speculative Decoding—
CTTFT (ms)TPS (tok/s)Status
147117.92PASS
261102.77PASS
47386.81PASS
876566.93PASS
1613649.26PASS
3220734.76PASS
6431125.38PASS

In this measurement, Qwen3.5-35B-A3B (35B parameters), served in FP8 format with vLLM on RTX PRO 6000, reached a generation speed of 117.9 tok/s per request and a time to first token (TTFT) of 47 ms with a single request.

Considering the speed targets and the available KV cache capacity, the estimated capacity is 150 users for chat use and 55 for agentic use. The realistic capacity will likely fall between these two values. In scenarios dominated by coding, tool use, long workflows and multi-agent use, capacity approaches the agentic estimate; where shorter interactions, standard conversations and lighter tasks dominate, it approaches the chat estimate.

Intelligence Index and Agentic Index values are published by Artificial Analysis and are reproduced here with attribution.