Can Qwen3.8-Max Run on Jetson Thor? The 2.4T Frontier Model vs Edge Hardware
As of August 13, 2026, the confusing part of the Qwen naming is this: Qwen3.8-Max and Qwen3-8B sound similar, but they are completely different deployment targets.
| Question | Answer |
|---|---|
| What is the latest "Qwen 3.8" model? | Qwen3.8-Max, described by Qwen as a 2.4T-parameter flagship |
| Can Jetson AGX Thor run it locally? | Not as a normal local-weight inference workload |
| What can Thor realistically test? | Qwen3-8B, Qwen3.6-27B, Qwen3.6-35B-A3B, and similar open-weight models |
| What is this post? | A feasibility check, not a fabricated Qwen3.8-Max local benchmark |
Qwen3.8-Max is not an 8B model. If you read "3.8" as "Qwen 3, 8B", the hardware estimate is off by orders of magnitude.
Qwen3.8-Max vs Qwen3-8B
Qwen's official pages describe Qwen3.8-Max as the strongest model in the Qwen family so far, built on the Qwen 3.5 architecture and scaled to 2.4 trillion parameters, with improvements for coding, work, and complex tasks.
Qwen/Qwen3-8B on Hugging Face is different: it is an Apache 2.0 open-weight model with 8.2B parameters, 36 layers, GQA, 32K native context, YaRN extension to 131K tokens, and support across Transformers, vLLM, SGLang, llama.cpp, Ollama, and LM Studio.
So if the goal is local Qwen inference on Thor, use Qwen3-8B or a Qwen3.6 open model. If the goal is Qwen3.8-Max quality testing, that is currently an API or remote-service evaluation, not a Thor local inference benchmark.
Thor Environment
I checked two live Thor machines without interrupting existing services:
| Device | Environment |
|---|---|
thor | NVIDIA Thor, CUDA 13.0, Linux 6.8.12-tegra, 122 GiB system memory |
thor-jd | NVIDIA Thor, CUDA 13.0, Linux 6.8.12-tegra, 122 GiB system memory |
Existing local model directories include Qwen2.5, Qwen3.6-35B-A3B, Qwen3-Coder-30B-A3B, Qwable-v1, and LocateAnything-3B. There is no Qwen3.8-Max local checkpoint on either machine.
Why Qwen3.8-Max Is Outside Thor's Local Range
Thor is excellent for low-power, unified-memory edge inference. It is a good fit for 8B, 30B-A3B, 35B FP8, and some quantized larger models. Qwen3.8-Max is a different class.
Approximate weight-only memory:
| Precision | 2.4T parameters |
|---|---|
| BF16 / FP16 | ~4.8 TB |
| FP8 | ~2.4 TB |
| INT4 | ~1.2 TB |
| 2-bit | ~600 GB |
That excludes KV cache, runtime workspace, routing overhead, and framework memory. A 123 GB Thor cannot simply load a 2.4T-parameter model and run llama-bench.
Even if the model is MoE and only activates part of the network per token, the inactive experts still need to live somewhere unless the runtime supports aggressive expert paging, remote experts, or distributed inference.
What To Test Instead
| Model | Local Thor fit | Best question |
|---|---|---|
| Qwen3-8B | High | How fast is Qwen3 thinking mode on Thor? |
| Qwen3-Coder-30B-A3B | High if weights are available | How good is edge coding inference? |
| Qwen3.6-35B-A3B-FP8 | Already available locally | Can Thor serve a 35B MoE model continuously? |
| Qwen3.8-Max | Low | Better tested through API quality tasks |
Previous Thor testing on this site measured Qwen3.6-35B-A3B-FP8 at about 14.6 t/s generation speed with SGLang and around 0.101s warm TTFT. That is already real-time for many local agent and assistant workflows.
Reproducible Qwen3-8B Thor Path
For a local Qwen 3 test, benchmark a GGUF Qwen3-8B build with llama.cpp:
cd ~/llama.cpp
LD_LIBRARY_PATH=build/bin build/bin/llama-bench \
-m /home/nvidia/models/qwen3-8b/qwen3-8b-instruct-q4_k_m.gguf \
-ngl 999 \
-fa 1 \
-p 128,512,2048 \
-n 128,256 \
-r 3
For SGLang:
python3 -m sglang.launch_server \
--model-path Qwen/Qwen3-8B \
--host 0.0.0.0 \
--port 30000 \
--reasoning-parser qwen3
Then test with an OpenAI-compatible request:
curl http://localhost:30000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Qwen/Qwen3-8B",
"messages": [
{"role": "user", "content": "Write quicksort in Python and explain its complexity. /no_think"}
],
"max_tokens": 512
}'
Measure thinking and non-thinking modes separately. Thinking tokens can make throughput and latency look inconsistent if both modes are mixed in one benchmark table.
Verdict
Qwen3.8-Max matters because Qwen is clearly still pushing frontier-scale models. But for edge deployment, it is not a near-term replacement for Thor-local models.
Thor's practical range remains:
- 8B for fast, concurrent local inference
- 30B-A3B for coding and agent workloads
- 35B-A3B / 35B FP8 for high-quality single-device serving
- quantized 70B-class experiments when KV cache and runtime support are managed carefully
Qwen3.8-Max is worth tracking, but it is not a Jetson Thor local inference model today. For Thor, test Qwen3-8B, Qwen3-Coder-30B-A3B, and Qwen3.6-35B-A3B instead.
Sources
- Qwen official blog:
https://qwen.ai/blog?id=qwen3.8 - Qwen research page:
https://qwen.ai/research - Qwen3-8B Hugging Face model card:
https://huggingface.co/Qwen/Qwen3-8B - Qwen3 release blog:
https://qwenlm.github.io/blog/qwen3/ - Ollama Qwen3:8B page:
https://ollama.com/library/qwen3:8b
Thor environment checked on August 13, 2026. This post does not claim a local Qwen3.8-Max benchmark because the current Thor hardware and public local-weight path do not support that test.