MiniMax H3 on Jetson AGX Thor: Early Public Benchmark Plan for FL2VA, Ref2VA, and INT8 ConvRot
MiniMax H3 is fresh enough that public numbers are still scattered across launch notes, community checkpoints, and early experiments. The interesting question for edge AI is simple: can NVIDIA Jetson AGX Thor run H3 in a useful, reproducible way?
This post establishes an early public benchmark protocol for FL2VA, Ref2VA, and INT8 ConvRot on a real Thor-JD machine.
Update on August 7, 2026: I also tested Abiray/Minimax-H3-nvfp4-INT4-INT8-Convrot. The conclusion is now sharper: BF16 FL2VA did not run successfully on Thor-JD, but Abiray NVFP4 through ComfyUI does run when launched inside the local Thor CUDA 13 PyTorch container. A 256x256, 5-frame, 1-step smoke test completed and generated a valid MP4.
Second August 7 update: based on the Chipmark host benchmark log, Thor-JD has now completed a 608x352, 243 frames, 20 steps ComfyUI quick baseline. Thor Docker CUDA 13 + NVFP4 hot run measured about 300.03s external / 284.64s ComfyUI, with sampling around 12.82s/it. The same INT8 ConvRot FL2VA weight measured about 400.04s external / 395.13s ComfyUI hot, with sampling around 18.46s/it.
Thor-JD Test Host
| Item | Value |
|---|---|
| Host | 100.98.98.105 |
| User | nvidia |
| Device | NVIDIA Jetson AGX Thor |
| OS | Linux 6.8.12-tegra, aarch64 |
| NVIDIA-SMI | 580.00 |
| CUDA | 13.0 |
| Memory | 122 GiB |
| Storage | 936 GB NVMe, about 266 GB free after FL2VA download |
| Current state | Stopped sglang-qwen25-7b before H3 loading to free memory |
| Test time | 2026-08-06 20:55 CST |
| SGLang image | nvcr.io/nvidia/sglang:26.04-py3 |
| Base SGLang version | 0.5.10+516d57ac |
| Tested runtime | source SGLang in-container + PYTHONPATH=/tmp/sglang-src/python + Transformers 5.12.1 |
| Swap | 0 GiB |
| ComfyUI tested | ComfyUI 0.30.0 + comfy-kitchen 0.2.26 |
| PyTorch tested | failed path: 2.5.1 / CUDA 12.4, without Thor sm_110 kernels; successful path: 2.11.0+cu130 / CUDA 13.0, with sm_110 support |
Thor's main advantage here is unified memory capacity. MiniMax H3's public partitions are large, and desktop 24 GB GPUs usually need quantization, offload, or multiple cards. Thor gives us a useful edge device for testing whether H3 can be made practical outside a workstation or cloud GPU box.
What Is Being Tested
The SGLang MiniMax-H3 documentation defines three public task profiles:
| Task | API task | Checkpoint partition | Conditioning |
|---|---|---|---|
| Text-to-Video+Audio | t2va | fl2va | Text only |
| First/Last-Frame-to-Video+Audio | fl2va | fl2va | First frame, last frame, or both |
| Reference-to-Video+Audio | ref2va | ref2va | Image, video, and audio references |
Important detail: fl2va serves both t2va and fl2va; ref2va requires a separate server launch with --model-variant ref2va. Video-to-video is a Ref2VA use case, not a separate v2v task value.
References:
- SGLang MiniMax-H3 cookbook
- MiniMaxAI/MiniMax-H3 on Hugging Face
- ComfyUI_RH_MinMaxH3
- DmitryDB MiniMax-H3 ComfyUI Quants
Benchmark Protocol
| Parameter | Value |
|---|---|
| Output duration | 5 seconds |
| Target frame rate | 24 FPS |
| Short edge | 768 |
| Steps | 50 |
flow_shift | 12.0 |
audio_flow_shift | 3.0 |
| Outputs per prompt | 1 |
Fixed test cases:
| Case | Task | Partition | Input |
|---|---|---|---|
t2va_text_only | t2va | FL2VA | Text |
fl2va_first_frame | fl2va | FL2VA | Fixed first-frame PNG |
ref2va_image_audio | ref2va | Ref2VA | Fixed reference PNG + WAV |
ref2va_video | ref2va | Ref2VA | Fixed reference MP4 |
Measured fields include model load result, submit latency, total generation time, realtime factor, output MP4 validity, video/audio metadata from ffprobe, and memory notes.
Reproduction Scripts
The benchmark harness lives in:
scripts/minimax-h3-benchmark/
Generate deterministic media assets:
cd scripts/minimax-h3-benchmark
python3 -m pip install pillow
python3 make_media_assets.py
Run FL2VA:
bash serve_sglang_fl2va.sh
python3 run_sglang_video_benchmark.py \
--base-url http://127.0.0.1:30010 \
--variant fl2va \
--out-dir results/fl2va
Run Ref2VA:
bash serve_sglang_ref2va.sh
python3 run_sglang_video_benchmark.py \
--base-url http://127.0.0.1:30010 \
--variant ref2va \
--out-dir results/ref2va
Summarize:
python3 summarize_results.py results
Initial Test Results
Official SGLang BF16 Path
| Test | Key arguments | Exit code | Result | Notes |
|---|---|---|---|---|
| FL2VA official path | --model-variant fl2va --performance-mode memory | 2 | Argument parsing failure | SGLang 0.5.10+516d57ac does not recognize these arguments |
| Ref2VA official path | --model-variant ref2va --performance-mode memory | 2 | Argument parsing failure | Same failure; did not reach checkpoint download or model loading |
Observed error:
sglang serve: error: unrecognized arguments: --model-variant fl2va --performance-mode memory
sglang serve: error: unrecognized arguments: --model-variant ref2va --performance-mode memory
This proves a narrower but useful point: the stock Thor-JD NVIDIA SGLang image is not the software stack required by the MiniMax H3 cookbook. The test therefore continued by building a source SGLang runtime inside the same NVIDIA container.
Source SGLang / FL2VA Load Test
To enter the MiniMax H3 runtime path, the test container did four things:
- Cloned source SGLang and forced it with
PYTHONPATH=/tmp/sglang-src/python. - Upgraded Transformers to
5.12.1sotransformers.vision_utilsimports correctly. - Applied a small in-container xgrammar compatibility patch for old symbols used by the source checkout.
- Downloaded the
MiniMaxAI/MiniMax-H3FL2VA partition throughHF_ENDPOINT=https://hf-mirror.com, producing about 135 GiB of cache.
| Run | Key arguments | HTTP ready | MP4 | Result | Key logs |
|---|---|---|---|---|---|
fl2va-live5 | --model-variant fl2va --performance-mode memory | No | No | Did not become ready after transformer load | text encoder loaded; SGLang reported consumed GPU mem: 57.86 GB, avail GPU mem: 11.23 GB; container RSS reached about 98 GiB |
fl2va-live6 | --dit-layerwise-offload --layerwise-offload-components dit text_encoder vae --dit-offload-prefetch-size 0 --dit-layerwise-resident-layers 0 --warmup-mode off | No | No | OOM killed | text encoder reported consumed GPU mem: 85.44 GB, avail GPU mem: 18.61 GB; transformer loaded 13 safetensors shards; scheduler died with exit code -9 |
Kernel OOM record:
Out of memory: Killed process 1114391 (sgl_diffusion::) total-vm:782713380kB,
anon-rss:57248772kB, file-rss:115516kB, shmem-rss:52471808kB
The important result is capacity, not speed: on a single 122 GiB no-swap Thor-JD, current source SGLang + BF16 FL2VA did not finish service startup. This is still more useful than a paper estimate because it passed argument parsing, dependency fixes, model download, and FL2VA partition selection before failing during real model loading.
Ref2VA Partition
| Path | Result | Notes |
|---|---|---|
| Ref2VA source SGLang | Not run | The downloaded Hugging Face snapshot contained FL2VA/model_index.json only. Ref2VA needs an additional partition download and a separate load record. Because FL2VA already OOMed during loading, Ref2VA is not inferred as tested. |
INT8 ConvRot Path
| Path | Runtime | Checkpoint | Result | Notes |
|---|---|---|---|---|
| FL2VA INT8 ConvRot | ComfyUI / H3 node | Not found | Not run | No ComfyUI, MiniMax H3 custom node, or INT8 ConvRot checkpoint was present on Thor-JD |
| Ref2VA INT8 ConvRot | ComfyUI / H3 node | Not found | Not run | The next run must install the node and record exact checkpoint hashes |
| SGLang diffusion quantization | source SGLang | No INT8 ConvRot recipe | Not run | The current diffusion runtime exposes fp8 / mxfp4 transformer quantization paths, but no directly reproducible INT8 ConvRot launch recipe was available in this environment |
Abiray NVFP4 / ComfyUI Path
On August 7, 2026, I tested the FL2VA NVFP4 set from Abiray/Minimax-H3-nvfp4-INT4-INT8-Convrot:
| File | Size | SHA256 |
|---|---|---|
MiniMax_H3_FL2VA_pruned_nvfp4.safetensors | 12.5 GB | 6ab7f0c48141e7919b32f925ca3def22e06a6aebeb9e0b6f5a0be0fe8409976f |
qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors | 15.7 GB | 33e69e3edab846d52949bafdb00378bd3f5a93f78124fc83d5ef109dc4a1fcbb |
minimax_h3_audio_vae_fp32.safetensors | 605 MB | 83043bf3afccf4cab4d7e114f6e04607e9687bef16930a2f6088d721a893d3c4 |
minimax_h3_video_vae_fp16.safetensors | 5.21 GB | 88539d729fd7e9f9de765498428cd8e9152b8420317ec0e7542a362fffb7e0e1 |
The files include trailing L2P_bypass_... markers. Standard safetensors rejects them with file not fully covered, so the test kept the originals and created local fixed copies by truncating each file to 8 + header_len + max(data_offsets). The fixed copies load normally. Their keys include comfy_quant, and the README describes the repository as optimized for ComfyUI, so this is not a native SGLang Diffusers partition.
ComfyUI 0.30.0 already includes core MiniMax H3 nodes. object_info showed:
UNETLoaderseesMiniMax_H3_FL2VA_pruned_nvfp4.safetensorsCLIPLoader(type=minimax)seesqwen3vl_32b_minimax_h3_nvfp4_awq.safetensorsMiniMaxH3ImageToVideoandMiniMaxH3SigmaShiftare available
I first submitted a minimal smoke prompt on the host Python environment: 256x256, 5 frames, 1 Euler step, T2VA/FL2VA, SaveVideo output.
| Stage | Result |
|---|---|
| Video VAE / Audio VAE | Loaded |
| Text encoder | Loaded; log reported 14960.20 MB loaded |
| MiniMaxH3 NVFP4 DiT | Detected mixed precision quantization; log reported 11945.45 MB loaded |
| Sampling | Failed |
| Output MP4 | Not generated |
Failure log:
NVIDIA Thor with CUDA capability sm_110 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_50 sm_80 sm_86 sm_89 sm_90 sm_90a.
RuntimeError: CUDA error: no kernel image is available for execution on the device
This moved the bottleneck from capacity to software support. The fix was to use the local Thor CUDA 13 PyTorch Docker image already present on Thor-JD:
fire-smoke-train:cu130
That image reports PyTorch 2.11.0+cu130, CUDA 13.0, device NVIDIA Thor, and CUDA architectures including sm_110. Running ComfyUI inside this container completed the same smoke workflow.
| Item | Value |
|---|---|
| Runtime | Docker fire-smoke-train:cu130 |
| PyTorch | 2.11.0+cu130 |
| CUDA | 13.0 |
| ComfyUI | 0.30.0 |
| comfy-kitchen | 0.2.26, CUDA backend enabled with NVFP4 capability |
| Prompt | 256x256, 5 frames, 1 Euler step, T2VA/FL2VA |
| Execution time | 46.21s |
| Output | minimax_h3_nvfp4_cu130_smoke_00001_.mp4 |
| Status | Success: valid MP4 generated |
ffprobe output for the generated MP4:
| Stream | Result |
|---|---|
| Video | H.264, 256x256, 24 FPS, 5 frames, 0.208 s |
| Audio | AAC LC, 32 kHz, stereo, 0.200 s |
| File size | 30,261 bytes |
So the current conclusion is: Thor-JD can run MiniMax H3 through the Abiray NVFP4 + ComfyUI path when using a Thor-compatible CUDA 13 PyTorch container. This is not yet the full benchmark; it is a confirmed smoke run.
Thor CUDA 13 / ComfyUI Quick Baseline
After the smoke test, I ran a more realistic quick benchmark on the same Thor-JD host. This is not the 768-short-edge, 5-second, 50-step protocol defined earlier in the post. It is the Chipmark-side quick baseline used to compare the Thor machine against RTX 3090 hosts.
| Parameter | Value |
|---|---|
| Resolution | 608x352 |
| Length | 243 frames |
| Steps | 20 |
| Thor runtime | Docker fire-smoke-train:cu130 + ComfyUI |
| PyTorch | 2.11.0+cu130, CUDA 13.0, with Thor sm_110 support |
| NVFP4 FL2VA weight | MiniMax_H3_FL2VA_pruned_nvfp4.safetensors |
| INT8 ConvRot FL2VA weight | minimax_h3_fl2va_pruned_int8_convrot.safetensors |
Thor-JD measured results:
| Host | Mode | Total time | ComfyUI time | Sampling time | Sampling speed | Notes |
|---|---|---|---|---|---|---|
| thor | Docker CUDA 13 NVFP4 cold / --lowvram | 300.04s | 288.13s | 4:16 | 12.81s/it | Thor container NVFP4 FL2VA weight |
| thor | Docker CUDA 13 NVFP4 hot / --lowvram | 300.03s | 284.64s | 4:16 | 12.82s/it | After model warm-up |
| thor | Docker CUDA 13 NVFP4 full-vram cold | 300.03s | 294.92s | 4:15 | 12.77s/it | No --lowvram |
| thor | Docker CUDA 13 NVFP4 full-vram hot | 300.03s | 282.34s | 4:16 | 12.81s/it | No --lowvram; no material speed gain |
| thor | Docker CUDA 13 INT8 ConvRot full-vram cold | 443.51s | 433.23s | 6:14 | 18.71s/it | Same INT8 ConvRot FL2VA weight as the 3090 baseline |
| thor | Docker CUDA 13 INT8 ConvRot full-vram hot | 400.04s | 395.13s | about 6:09 | about 18.46s/it | After model warm-up |
RTX 3090 CUDA 13 quick baseline for context:
| Host | Mode | Total time | Sampling time | Sampling speed | Notes |
|---|---|---|---|---|---|
| giga3090 | CUDA 13 hot | 213.05s | about 3:09 | about 9.46s/it | After model warm-up |
| rog3090 | CUDA 13 cold | 521.55s | 3:07 | 9.37s/it | Includes first model init, about 143s |
| rog3090 | CUDA 13 hot | 206.07s | 3:07 | 9.38s/it | After model warm-up |
Current comparison:
- Thor NVFP4 hot is about
1.38x-1.46xslower than the 3090 CUDA 13 INT8 hot baseline by total time, and about1.36xslower by sampling speed. This is not a pure hardware comparison because the Thor NVFP4 run and the 3090 baseline used different FL2VA weights. - Removing
--lowvramdid not materially improve Thor NVFP4 speed; sampling stayed around12.77-12.82s/it. - With the same INT8 ConvRot FL2VA weight as the 3090 baseline, Thor hot sampling was about
18.46s/it, roughly1.97xslower than the 3090 hot sampling baseline of about9.38s/it.
Thor output records:
/workspace/ComfyUI/output/thor_minimaxh3_cu130_benchmark_cold2_608x352_len243_steps20_00001_.mp4
/workspace/ComfyUI/output/thor_minimaxh3_cu130_benchmark_hot2_608x352_len243_steps20_00001_.mp4
/workspace/ComfyUI/output/thor_minimaxh3_cu130_fullvram_benchmark_cold_608x352_len243_steps20_00001_.mp4
/workspace/ComfyUI/output/thor_minimaxh3_cu130_fullvram_benchmark_hot_608x352_len243_steps20_00001_.mp4
/workspace/ComfyUI/output/thor_minimaxh3_cu130_int8convrot_fullvram_benchmark_cold_608x352_len243_steps20_00001_.mp4
/workspace/ComfyUI/output/thor_minimaxh3_cu130_int8convrot_fullvram_benchmark_hot_608x352_len243_steps20_00001_.mp4
Auditable files:
scripts/minimax-h3-benchmark/actual-results-2026-08-06.json
scripts/minimax-h3-benchmark/results-env/
scripts/minimax-h3-benchmark/results-fl2va-live5/
scripts/minimax-h3-benchmark/results-fl2va-live6/
scripts/minimax-h3-benchmark/results-comfy-nvfp4/
scripts/minimax-h3-benchmark/results-comfy-nvfp4-cu130/
Early Takeaways
The early conclusion is now more concrete: Thor-JD does not stably run the current BF16 SGLang FL2VA path, but it can run MiniMax H3 quantized weights through CUDA 13 PyTorch Docker + ComfyUI, and it has completed a 243-frame, 20-step quick baseline.
- The stock NGC SGLang 26.04 image cannot run the MiniMax H3 cookbook command directly because
--model-variantand--performance-modeare unavailable in0.5.10+516d57ac. - Source SGLang can move Thor-JD into the real FL2VA load path: after dependency fixes, the FL2VA partition is selected correctly and both text encoder and transformer shards begin loading.
- 122 GiB with no swap is still not enough for a stable BF16 FL2VA service in the tested configuration: the most aggressive layerwise/offload run was OOM-killed before ready state.
- Abiray NVFP4 / ComfyUI runs and has a quick baseline: Thor Docker CUDA 13 hot measured about
300.03sexternal /284.64sComfyUI, with sampling around12.82s/it. - INT8 ConvRot FL2VA also has a Thor quick baseline: the same INT8 ConvRot FL2VA weight measured about
400.04sexternal /395.13sComfyUI hot, with sampling around18.46s/it. - Ref2VA still needs a separate test: the Ref2VA partition was not downloaded, so it cannot be inferred from FL2VA / T2VA results.
The next update should scale the successful NVFP4 and INT8 ConvRot paths to the full 768-short-edge, 5-second, 50-step benchmark, record peak memory and generation time, then test Ref2VA as a separate ComfyUI run.