2×Jetson Thor for DeepSeek-V4-Flash-NVFP4: QSFP28 Preflight and Real Blockers

After testing DeepSeek-R1-Distill-Qwen-32B on thor-jd, I checked a much harder target: running NVIDIA's nvidia/DeepSeek-V4-Flash-NVFP4 across two Jetson Thor systems over QSFP28.

This post records the preflight, not a successful V4-Flash run.

Item	Result
Model	`nvidia/DeepSeek-V4-Flash-NVFP4`
Model class	DeepSeek-V4-Flash: 284B total / 13B active; NVIDIA page lists 167B params
Official serving shape	vLLM on GB300 with `TP=4`; SGLang examples use larger TP
Target hardware	2× Jetson Thor
Current state	Did not download or launch V4-Flash weights
Blockers	QSFP28 IP not configured on both ends, insufficient disk on `thor-jd`, unproven Jetson aarch64 NVFP4/V4 distributed stack

Short version: 2×Thor may be an interesting experiment, but this environment is not ready for a V4-Flash-NVFP4 launch yet. First prove QSFP28 plus SGLang two-node TP with a small model.

Why Not Download First

DeepSeek-V4-Flash-NVFP4 is not a 30B or 70B model. It is NVIDIA's NVFP4 version of DeepSeek-V4-Flash, aimed at Blackwell GPUs and served through SGLang or vLLM.

The official examples point toward data-center hardware, not edge SoCs:

NVFP4 quantization
FP4 indexer cache
Blackwell validation
vLLM example on GB300 with TP=4
SGLang examples with multi-GPU tensor parallelism

Current local capacity is also tight:

Machine	Free disk	Available memory
`thor`	~161GB	~44GB
`thor-jd`	~73GB	~61GB

thor-jd does not have enough free disk for a safe full checkpoint download. A shared model directory may be possible later, but that should come after the network and distributed runtime are proven.

Software Check

Both machines have NVIDIA NGC containers:

Machine	SGLang	vLLM
`thor`	`nvcr.io/nvidia/sglang:26.04-py3`, SGLang `0.5.10+516d57ac`	`nvcr.io/nvidia/vllm:26.04-py3`
`thor-jd`	`nvcr.io/nvidia/sglang:26.04-py3`	`nvcr.io/nvidia/vllm:26.04-py3`

SGLang exposes the relevant flags:

--tensor-parallel-size
--nnodes
--node-rank
--dist-init-addr
--moe-runner-backend
--moe-a2a-backend
--modelopt-quant nvfp4

That means the command surface exists. It does not prove the Jetson Thor path is ready. The risky parts are FP4/NVFP4 kernels on aarch64, DeepSeek V4 loader behavior, TP=2 suitability, NCCL over QSFP28, and MoE all-to-all performance.

QSFP28 Check

All four mgbe interfaces report 10GbE link up on both machines.

thor-jd already has IPs:

mgbe0_0  192.168.100.2/24
mgbe1_0  192.168.101.2/24
mgbe2_0  192.168.102.2/24
mgbe3_0  192.168.103.2/24

thor has the links up but no IPv4 addresses. I attempted to add temporary addresses:

sudo ip addr add 192.168.100.1/24 dev mgbe0_0
sudo ip addr add 192.168.101.1/24 dev mgbe1_0
sudo ip addr add 192.168.102.1/24 dev mgbe2_0
sudo ip addr add 192.168.103.1/24 dev mgbe3_0

But the current SSH user does not have passwordless sudo:

sudo: a password is required

So this run could not complete iperf3 or NCCL testing over QSFP28.

Weight Access

For normal SGLang/vLLM multi-node tensor parallelism, each rank generally needs access to the full checkpoint directory, or all ranks need to see the same shared model path.

Option	Assessment
Full checkpoint on both machines	Most robust, highest disk cost
NFS/shared NVMe mounted at the same path	Saves disk, may slow startup
Manually split shards per rank	Not recommended; easy to break safetensors index assumptions

Given the current disk situation, downloading the full checkpoint onto both machines is not the right next move.

Baseline Already Proven

The same thor-jd system successfully ran:

Model	Quantization	Context	Speed
DeepSeek-R1-Distill-Qwen-32B	Q4_K_M, 18.48GiB	4096	~10.4 tok/s

That proves Thor can run a 32B dense reasoning model locally. It does not imply V4-Flash-NVFP4 will run across two nodes.

Next Steps

Before touching V4-Flash weights:

Configure QSFP28 IPs on thor.
Run iperf3 on all four links.
Start a small model with SGLang --nnodes 2 --tp 2.
Only then test V4-Flash with a very small context.

Candidate small-model distributed test:

sglang serve \
  --model-path /models/Qwen2.5-7B-Instruct \
  --host 0.0.0.0 \
  --port 8000 \
  --tp 2 \
  --nnodes 2 \
  --node-rank 0 \
  --dist-init-addr 192.168.100.1:20000 \
  --trust-remote-code

Run rank 1 on thor-jd with --node-rank 1 and the same --dist-init-addr.

Verdict

This preflight did not prove that 2×Thor can run DeepSeek-V4-Flash-NVFP4. It proved the current environment is not ready for that test.

Confirmed:

SGLang/vLLM NVIDIA containers are present.
SGLang exposes multi-node, TP, MoE, and NVFP4-related flags.
QSFP28 physical links are up at 10GbE.
thor-jd has QSFP28 IPs configured.
thor does not have QSFP28 IPs configured, and the current user cannot add them.
Disk capacity is too tight for a safe V4-Flash checkpoint download, especially on thor-jd.

My engineering call: do not download V4-Flash-NVFP4 yet. First prove 2×Thor SGLang TP=2 with a small model over QSFP28.

Test date: August 13, 2026. This is a preflight report, not a successful DeepSeek-V4-Flash-NVFP4 inference benchmark.