MegaCube. Desktop AI SupercomputerDiscuss Your Solution
← All models

General Q&A · Coding

Qwen3.8-27B

A 27B multimodal model for general Q&A, coding, and document understanding. RadixArk NVFP4 weights are deployed on one MegaCube to validate native multi-token prediction (MTP).

Multimodal understandingText, images, video → text
1 MegaCubeTest report available

01 / Model parameters

Model architecture & capabilities

Qwen3.8-27B is Qwen's dense multimodal model for everyday Q&A, document understanding, coding, and tool use. It combines linear and full attention with native multi-token prediction. Inference speed and memory use depend on the quantized weights and inference engine, even when the base model is the same.

  • Knowledge Q&A and information synthesis
  • Code generation and debugging
  • Document and image understanding
Publisher / base model
Alibaba Qwen · Qwen3.8-27B
Parameter count / architecture
27B dense model; 64 layers; Gated DeltaNet + Gated Attention hybrid architecture
Model context
Native 262,144 tokens, expandable to 1M; the extended length is not a device validation result on this page
Input / output
Text, image, and video input; text and tool-call output
Reasoning capabilities
Supports thinking mode, agent tool calling, and native multi-token prediction (MTP)
License
Apache-2.0
Quantization used here
RadixArk ModelOpt mixed quantization: NVFP4 MLP / lm_head, FP8 attention, BF16 retained for MTP and vision components

The model supports multimodal input, but the MegaCube test matrix primarily covers text tasks. Published vision capabilities are separate from local vision validation.

Base model documentation ↗RadixArk quantization & benchmarks ↗

Model information checked on 2026-09-20

02 / Deployment

MegaCube deployment configuration

Compute devices
1 × MegaCube · NVIDIA GB10 · 128 GB
Inference engine
SGLang; public image lmsysorg/sglang:v0.5.16-cu130
Acceleration settings
Native MTP steps=3 / draft tokens=4
Cache & context
FP8 E4M3 KV; configured limit of 262,144 tokens
Memory
Static memory fraction 0.70; model plus MTP residency approx. 20.75 GiB

  1. Check the environment

    Prepare ARM64 Linux, NVIDIA drivers, Docker, and NVIDIA Container Toolkit.

  2. Prepare the model

    Download RadixArk/Qwen3.8-27B-NVFP4 and mount it at /models/target in the container.

  3. Start the service

    Configure NEXTN, steps=3, draft tokens=4, and FP8 KV, then start SGLang.

  4. Connect to PINVOU

    Check /health and /v1/models, then connect through the compatible API using qwen3.8-27b as the model name.

Find and download this model in the PINVOU Model Store, then follow its model card to deploy and enable the service.

Manage models in PINVOU →
View commands and configuration references

Before use, set model paths and service addresses for your devices and check the versions and parameters above.

uname -m
nvidia-smi
docker version
docker compose version
docker run --rm --gpus all ubuntu:24.04 nvidia-smi
mkdir -p "$HOME/models/Qwen3.8-27B-NVFP4"
python3 -m venv "$HOME/.venvs/hf-download"
source "$HOME/.venvs/hf-download/bin/activate"
pip install -U huggingface_hub

HF_ENDPOINT=https://hf-mirror.com \
hf download RadixArk/Qwen3.8-27B-NVFP4 \
  --revision main \
  --local-dir "$HOME/models/Qwen3.8-27B-NVFP4"
# Download from the network
docker pull lmsysorg/sglang:v0.5.16-cu130

# Exact reproduction: export on the original Cube
docker save qwen38-benchmark/sglang-dflash2:dd15fb57 \
  | zstd -T0 -3 -o qwen38-sglang-dd15fb57-arm64.tar.zst

# Import on the new Cube
zstd -dc qwen38-sglang-dd15fb57-arm64.tar.zst | docker load
mkdir -p "$HOME/qwen38-serving-cache/sglang" \
             "$HOME/qwen38-serving-cache/huggingface"

docker run -d \
  --name qwen38-nvfp4-mtp-api \
  --restart unless-stopped \
  --gpus all \
  --network host \
  --ipc host \
  -v "$HOME/qwen38-serving-cache/sglang:/root/.cache/sglang" \
  -v "$HOME/qwen38-serving-cache/huggingface:/root/.cache/huggingface" \
  -v "$HOME/models/Qwen3.8-27B-NVFP4:/models/target:ro" \
  lmsysorg/sglang:v0.5.16-cu130 \
  python3 -m sglang.launch_server \
  --model-path=/models/target \
  --served-model-name=qwen3.8-27b \
  --host=0.0.0.0 \
  --port=30000 \
  --context-length=262144 \
  --mem-fraction-static=0.70 \
  --kv-cache-dtype=fp8_e4m3 \
  --trust-remote-code \
  --reasoning-parser=qwen3 \
  --tool-call-parser=auto \
  --sampling-defaults=model \
  --preferred-sampling-params='{"temperature":1.0,"top_p":0.95,"top_k":20,"min_p":0.0,"presence_penalty":0.0,"repetition_penalty":1.0}' \
  --speculative-algorithm=NEXTN \
  --speculative-num-steps=3 \
  --speculative-eagle-topk=1 \
  --speculative-num-draft-tokens=4
docker logs -f qwen38-nvfp4-mtp-api
# Run in another terminal
curl -fsS http://127.0.0.1:30000/health
curl -fsS http://127.0.0.1:30000/v1/models
curl -fsS http://127.0.0.1:30000/get_model_info
curl http://<Cube-IP>:30000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "qwen3.8-27b",
    "messages": [
      {"role": "user", "content": "Hello, please introduce yourself."}
    ],
    "temperature": 1.0,
    "top_p": 0.95,
    "top_k": 20,
    "max_tokens": 1024,
    "stream": true
  }'

03 / Performance tests

MegaCube performance test results

2026-08-27; one GB10; 8 prompt categories. C1 uses 1 seed, while C4/C8 use 3 seeds. Table throughput values are geometric means calculated using the report's methodology.

23.923 tok/sInstruct C1, geometric mean output throughput
94.198 tok/sInstruct C8, geometric mean output throughput
214.1 msInstruct C1, mean TTFT
Thinking and Instruct: sampled task performance
ModeConcurrencyRequestsOutput tokensGeometric mean output throughput (tok/s)Average TTFTAverage TPOTGeometric mean of P95 values
Thinking12417,86619.604224.5 ms50.85 ms41,741.3 ms
Thinking47254,23955.133336.6 ms59.52 ms50,679.7 ms
Thinking87254,05879.072451.4 ms69.28 ms58,565.0 ms
Instruct12417,27023.923214.1 ms42.05 ms33,293.4 ms
Instruct47251,57965.233309.4 ms49.13 ms40,643.1 ms
Instruct87251,26494.198316.4 ms57.00 ms46,867.2 ms
MTP scheduler acceptance data
ConcurrencySamplesAverage accepted lengthAverage acceptance rate
13792.33244.4%
43662.34344.8%
82342.34444.8%
  • The fixed-length C8 result of 177.14 tok/s uses a different workload from the sampled tasks with naturally terminating outputs. These results cannot be compared directly.
  • Revalidate after changing context, KV precision, MTP, or the memory ratio.

C1 / C4 / C8 denotes 1 / 4 / 8 concurrent requests. TTFT (time to first token) measures elapsed time from submitting a request to receiving its first output token. TPOT (time per output token) measures average time per generated token after the first token. Aggregate throughput is the combined output rate across concurrent requests; a single-request token generation rate is measured separately in tok/s.

Public benchmark references

Model capability & platform performance benchmarks

The following results are published by model developers or platform providers and are separate from the MegaCube tests above. Capability scores measure task performance; tok/s measures token throughput. Results obtained with different models, numerical precision, hardware, or test protocols are not directly comparable.

RadixArk NVFP4 quantized model benchmarks

RadixArk · Checked on 2026-09-20

Evaluated model
Qwen3.8-27B-NVFP4
Test platform
4 × NVIDIA B300 / GB300；TP=4；SGLang
RadixArk NVFP4 quantized model benchmarks · Data
BenchmarkScoreWhat it measures
GSM8K97.27%（1,283 / 1,319）Math word problems
Terminal-Bench 2.1 subset73.81%（62 / 84）Terminal task completion rate

Test conditionsGSM8K uses all 1,319 questions, thinking mode, temperature=1, top_p=0.95, and top_k=20. Terminal-Bench uses an 84-task subset, Claude Code 2.1.228, and pass@1. Neither benchmark is a MegaCube test.

View original benchmarks ↗

Plan model deployment and application integration

Evaluate device configurations and inference services for your model size, workloads, and concurrency needs.

Discuss this model