A multimodal mixture-of-experts model for long-context reasoning, coding, and agent tasks. Distributed inference is validated on 16 MegaCubes with MXFP4 weights.
Multimodal understandingText, images, video → text
16 MegaCubesTest report available
01 / Model parameters
Model architecture & capabilities
Kimi K3, published by Moonshot AI, is designed for extended reasoning, multi-step tasks, and long contexts. Its mixture-of-experts architecture activates a subset of parameters at each step and combines linear and sparse attention to process long text. It also supports image and video understanding, with text or tool-call output.
Long-form material and cross-document Q&A
Code development, editing, and troubleshooting
Multi-step agent tasks
Publisher / model
Moonshot AI · Kimi-K3
Parameter count
2.8T total parameters; approx. 104B active per token
Architecture
93-layer MoE; 896 routed experts, 16 selected per step, plus 2 shared experts; KDA + Gated MLA
Model context
Native 1,048,576 tokens; see deployment and tests for the usable device context length
Input / output
Text, image, and video input; text and tool-call output
Weight precision / size
MXFP4 weights, MXFP8 activations; approx. 1.45 TB of weights in this test
License
Kimi K3 License; use and distribution must comply with the model license
MegaCube tests on this page primarily cover text reasoning, long-context retrieval, and concurrent inference. Published vision capabilities do not establish validation of every vision task on this configuration.
H3C S9855-40B · Dual 200G RoCE v2 links · MTU 9000
Inference engine
vLLM · TP16 · DSpark speculative decoding
Context
Service configuration: 333,126 tokens; tested here up to 128K
Application integration
OpenAI-compatible API
Prepare the cluster
Connect 16 devices to the RoCE switching network and verify management connectivity and both high-speed links.
Load the weights
Verify the MXFP4 weights and run TP16 shards, with approx. 93 GB of weights per device.
Start inference
Start vLLM and DSpark with the report's configuration and confirm that all compute nodes join the service.
Connect your application
Check the API, long-document tasks, and tool calling, then connect PINVOU or your business application.
03 / Performance tests
MegaCube performance test results
2026-08-27; 16 devices; 234 model test requests. Per-request token generation and aggregate throughput under concurrency are measured separately. Short-context concurrency results do not apply to 128K input.
Single request: input length and decoding performance
Input length
Time to first token
Token generation rate
128
0.573 s
22.69 tok/s
1K
0.727 s
25.28 tok/s
32K
10.903 s
23.48 tok/s
128K
44.216 s
21.20 tok/s
Short context: concurrent throughput and TTFT
Short-context concurrency
Aggregate system throughput
TTFT P95
Success rate
Recommended use
1 request
21.8 tok/s
0.70 s
100%
Responsive single-user use
2 requests
32.9 tok/s
0.93 s
100%
Responsive shared use
4 requests
42.7 tok/s
1.31 s
100%
Recommended interactive concurrency limit
8 requests
48.4 tok/s
56.05 s
100%
Queued tasks
16 requests
54.3 tok/s
104.47 s
100%
Highest measured throughput / batch processing
Long-context retrieval was tested at 12 positions in the test text. This result does not measure accuracy across general knowledge Q&A.
Higher concurrency increases queueing and time to first token. Aggregate system throughput does not describe the response speed experienced by an individual user.
C1 / C4 / C8 denotes 1 / 4 / 8 concurrent requests. TTFT (time to first token) measures elapsed time from submitting a request to receiving its first output token. TPOT (time per output token) measures average time per generated token after the first token. Aggregate throughput is the combined output rate across concurrent requests; a single-request token generation rate is measured separately in tok/s.
Public benchmark references
Model capability & platform performance benchmarks
The following results are published by model developers or platform providers and are separate from the MegaCube tests above. Capability scores measure task performance; tok/s measures token throughput. Results obtained with different models, numerical precision, hardware, or test protocols are not directly comparable.
Kimi K3 official capability benchmarks
Moonshot AI · Checked on 2026-09-20
Evaluated model
Kimi K3
Test platform
Inference hardware not disclosed by the source
Kimi K3 official capability benchmarks · Data
Benchmark
Score
What it measures
GPQA Diamond
93.5
Challenging science Q&A
DeepSWE v1.1
67.5
Real-world software problem solving
Terminal-Bench 2.1
88.3
Multi-step terminal operations
Test conditionsMaximum reasoning effort, temperature=1. GPQA uses top_p=0.95; coding and terminal tasks use Kimi Code and top_p=1. Scores depend on the agent framework and tool environment.
