General Q&A · Coding
GLM-5.3-Flash
A multimodal model for general Q&A, coding, and agent tasks. Multi-node validation focuses on text reasoning, 256K context, and concurrent serving performance.
01 / Model parameters
Model architecture & capabilities
Published by Z.ai, GLM-5.3-Flash uses a mixture-of-experts architecture with sparse and linear attention for coding, long-context reasoning, and agent tasks. The model supports image and video understanding; this MegaCube validation focuses on text, long context, and concurrent inference.
- Long-document analysis and Q&A
- Code generation and engineering tasks
- Agent workflows with tools
- Publisher / model
- Z.ai · GLM-5.3-Flash
- Parameter count
- 320B total parameters; 18B active
- Architecture
- 45-layer MoE; 288 routed experts, 8 selected per step, plus 1 shared expert; hybrid sparse / linear attention and mHC
- Model context
- Configured limit of 1,048,576 tokens; MegaCube validated up to 256K on this page
- Input / output
- The model supports text, image, and video input with text output; this test uses text input / output
- License
- MIT
This performance data excludes image and video tasks and does not validate native 1M context on this configuration.
Model information checked on 2026-09-20
02 / Deployment
MegaCube deployment configuration
- Compute devices
- 4 × MegaCube, each with 128 GB unified memory
- Network
- H3C S9855 · 200G RoCE
- Engine & parallelism
- vLLM + Ray · TP=4
- Service settings
- Context 262,144; MaxSeq=8; MTP=3; Eager
- KV & memory
- FP8 E4M3；GPU memory utilization=0.75
Prepare the network
Connect four devices to the RoCE network and verify management and high-speed connectivity.
Match the runtime environments
Use identical GLM-5.3-Flash weights and inference runtimes on all nodes.
Start the cluster
Use Ray to coordinate the TP4 service with MTP3 and an FP8 E4M3 KV cache.
Integrate by workload
Set separate concurrency limits for ordinary and extremely long requests, then integrate applications through the compatible API.
03 / Performance tests
MegaCube performance test results
2026-08-28; 256K service configuration; 17 workload configurations and 552 valid requests. Output throughput is aggregated across the batch and differs from the token generation rate of an individual request.
| Input / output | Concurrency | RPS | Aggregate output throughput (tok/s) | TTFT P50 / P95 | TPOT P50 / P95 | Success rate |
|---|---|---|---|---|---|---|
| 512 / 128 | C1 | 0.2364 | 30.26 | 0.61 / 0.62 s | 28.31 / 31.36 ms | 100% |
| 512 / 128 | C2 | 0.3355 | 42.95 | 0.75 / 1.14 s | 40.47 / 43.80 ms | 100% |
| 512 / 128 | C4 | 0.4757 | 60.89 | 1.01 / 1.24 s | 58.35 / 63.98 ms | 100% |
| 512 / 128 | C8 | 0.6885 | 88.13 | 1.49 / 2.36 s | 79.78 / 88.90 ms | 100% |
| 2048 / 256 | C1 | 0.1288 | 32.98 | 1.24 / 1.30 s | 25.57 / 26.12 ms | 100% |
| 2048 / 256 | C2 | 0.1736 | 44.45 | 1.55 / 2.60 s | 38.97 / 40.14 ms | 100% |
| 2048 / 256 | C4 | 0.2346 | 60.05 | 2.79 / 4.93 s | 55.41 / 61.52 ms | 100% |
| 2048 / 256 | C8 | 0.3015 | 77.17 | 4.17 / 8.80 s | 84.29 / 109.50 ms | 100% |
| 8128 / 1024 | C1 | 0.0322 | 33.01 | 6.06 / 6.09 s | 24.46 / 24.66 ms | 100% |
| 8128 / 1024 | C2 | 0.0429 | 43.91 | 8.14 / 10.43 s | 36.23 / 38.30 ms | 100% |
| 8128 / 1024 | C4 | 0.0588 | 60.23 | 11.31 / 19.26 s | 55.20 / 60.84 ms | 100% |
| 8128 / 1024 | C8 | 0.0780 | 79.82 | 20.13 / 38.77 s | 79.42 / 90.41 ms | 100% |
| 16,384 / 1024 | C1 | 0.0271 | 27.75 | 12.50 / 12.55 s | 23.69 / 24.27 ms | 100% |
| 16,384 / 1024 | C2 | 0.0355 | 36.33 | 16.87 / 21.87 s | 37.96 / 43.20 ms | 100% |
| 16,384 / 1024 | C4 | 0.0421 | 43.14 | 20.83 / 40.53 s | 59.69 / 76.52 ms | 100% |
| 32,768 / 1024 | C1 | 0.0204 | 20.89 | 25.01 / 25.03 s | 23.47 / 23.57 ms | 100% |
| 32,768 / 1024 | C2 | 0.0255 | 26.06 | 33.41 / 43.43 s | 43.35 / 48.87 ms | 100% |
- One 256K context-limit test recorded a TTFT of 232.40 s. The results do not establish that full-length requests can be kept in memory simultaneously at C4.
- Low latency on ordinary requests does not establish input processing time for uncached, very long prompts.
C1 / C4 / C8 denotes 1 / 4 / 8 concurrent requests. TTFT (time to first token) measures elapsed time from submitting a request to receiving its first output token. TPOT (time per output token) measures average time per generated token after the first token. Aggregate throughput is the combined output rate across concurrent requests; a single-request token generation rate is measured separately in tok/s.
Plan model deployment and application integration
Evaluate device configurations and inference services for your model size, workloads, and concurrency needs.