Dual Jetson AGX Thor 4×10GbE Test: Why Did Aggregate Bandwidth Stop at 12 Gbps?

Two Jetson AGX Thor systems provide enough high-speed Ethernet ports to make an attractive back-of-the-envelope calculation: four direct 10GbE links should deliver something close to 40 Gbps in aggregate.

That is not what we measured. Each link individually approached 10 Gbps, while concurrent traffic across multiple links stopped at roughly 12 Gbps in total.

This article documents the method, the measured results, and what that bandwidth ceiling actually means for distributed LLM inference.

Results at a glance

Question	Measured result
Does each 10GbE link work normally?	Yes. Individual links reached 8.98–9.75 Gbps
Can multiple TCP streams exceed one link's line rate?	No. iperf3 -P 4 reached about 9.85 Gbps total
Do two links add useful bandwidth?	Yes, but only up to 12.18 Gbps total
Do four links approach 40 Gbps?	No. They reached about 11.88 Gbps total
Was the CPU saturated?	No. Roughly 76% CPU capacity remained idle during the test
Is 12 Gbps enough for pipeline-style layer splitting?	Usually yes when only small hidden states cross the link
Is it enough for KV cache transfer?	Model-dependent; networking was not the main constraint in our example

Test topology

We directly connected four MGBE interfaces between two Jetson AGX Thor systems. Internal hostnames and full lab addresses are omitted; the interface and subnet layout is preserved below:

Node A interface	Node B address	Purpose
mgbe0_0	192.168.100.2	Link 0
mgbe1_0	192.168.101.2	Link 1
mgbe2_0	192.168.102.2	Link 2
mgbe3_0	192.168.103.2	Link 3

We used iperf3 with runs lasting approximately 8–12 seconds. The test started with each link in isolation, followed by concurrent runs across independent interfaces and server ports.

Test 1: Each 10GbE link in isolation

Interface	Throughput
mgbe0_0	9.75 Gbps
mgbe1_0	8.98 Gbps
mgbe2_0	9.49 Gbps
mgbe3_0	9.69 Gbps

All four links worked. After Ethernet and TCP/IP overhead, roughly 9–10 Gbps is a reasonable result for a 10GbE connection.

The slowest link reached 8.98 Gbps, but that small difference cannot explain the aggregate ceiling observed later.

Test 2: Multiple flows on one link

We then ran four TCP streams over a single interface:

iperf3 -c 192.168.100.2 -P 4

Combined throughput was approximately 9.85 Gbps.

This makes a single-flow scheduling issue unlikely. Multiple streams can fill the physical link, but they do not push it beyond line rate.

Test 3: Two and four links concurrently

The concurrent results were the interesting part:

Configuration	Per-link throughput	Aggregate throughput
2 links: mgbe0_0 + mgbe2_0	6.25 / 5.93 Gbps	12.18 Gbps
4 links	3.39 / 1.67 / 3.40 / 3.42 Gbps	11.88 Gbps

Moving from one link to two increased aggregate throughput from roughly 9.8 to 12.2 Gbps. Adding all four links produced no further gain and was slightly slower.

The practical result is therefore not “4 × 10 Gbps = 40 Gbps,” but this:

Every interface can individually approach 10 Gbps, yet concurrent traffic encounters a shared system-level ceiling near 12 Gbps.

Where might the bottleneck be?

The CPUs remained roughly 76% idle, so the available evidence does not support CPU saturation as the explanation. Each interface also worked normally in isolation, making a single bad port or cable unlikely.

A plausible engineering hypothesis is that the MGBE interfaces share a DMA, memory, or internal I/O path inside the SoC and reach a common ceiling under concurrent load. However, this is an inference from the throughput pattern, not a hardware-level attribution proven with internal interconnect counters.

Useful follow-up tests would include:

• logging EMC, MGBE, CPU, and SoC clocks to rule out power-mode or frequency limits;
• comparing unidirectional and bidirectional traffic;
• varying MTU, TCP windows, and CPU affinity;
• inspecting memory and I/O pressure with device performance counters or NVIDIA profiling tools;
• repeating the test across JetPack, kernel, and power-mode versions.

The current measurements prove that multi-link throughput does not scale linearly. They do not yet prove which specific internal component owns the ceiling.

What does 12 Gbps mean for distributed LLM inference?

Twelve gigabits per second is about 1.5 GB/s before practical overhead. That is far below the apparent sum of four 10GbE ports, but whether it matters depends on what crosses the network.

1. Layer splitting: bandwidth is usually not the first problem

With layer-wise model partitioning, the nodes exchange hidden states at a partition boundary rather than moving the complete model weights for every generated token.

In one 8-bit hidden-state path we validated, the transferred payload was approximately 7 KB per token. Even after protocol overhead, 12 Gbps is ample for that payload size. More important constraints include:

• synchronization latency per token;
• compute balance between nodes;
• overlap between communication and compute;
• support for this partitioning strategy in the inference runtime.

Peak bandwidth alone does not predict pipeline performance.

2. Prefill/decode disaggregation: calculate the KV cache first

Prefill/decode disaggregation sends the KV cache produced during prefill to a decode node. The payload can be much larger, but the correct first step is still to calculate the actual transfer size rather than assume that every distributed inference design requires 100GbE.

For one compressed-KV model we tested, a 32K context produced approximately 0.78 GiB of KV cache. At a rough effective rate of 1.5 GB/s, the pure transfer time would be:

0.78 GiB ÷ 1.5 GB/s ≈ 0.5–0.6 seconds

Prefill for the same context took about 80 seconds. Even without pipelining, network transfer accounted for less than 1% of that time. Prefill/decode disaggregation may still be a poor fit for this workload, but not because the 12 Gbps link is too slow; it does not necessarily improve the decode stage that limits the workload.

This result does not generalize to every model. Standard multi-head attention, larger batches, longer contexts, or uncompressed KV caches can increase the payload substantially. Before deploying, measure three numbers: KV cache bytes, effective network throughput, and separate prefill/decode time.

Can multi-NIC software bypass the ceiling?

If the limitation were route selection or a single-interface policy, UCX multi-rail, NCCL multi-NIC, or NIXL multi-NIC configuration could help.

In this test, however, separate iperf3 processes were already using different physical interfaces and subnets, and their combined throughput still stopped near 12 Gbps. Enabling additional rails at the application layer is therefore unlikely to bypass the same shared lower-level path automatically.

Those frameworks still provide value through interface selection, scheduling, resiliency, and communication overlap. Detecting four interfaces simply does not guarantee four times the throughput.

Deployment checklist

For distributed inference between two Jetson AGX Thor systems, validate the network in this order:

1. Test every interface independently to verify the link, cable, and address configuration.
2. Use -P 4 on one interface to rule out a single TCP stream limitation.
3. Run concurrent tests across independent subnets and server ports.
4. Record CPU idle time, power mode, temperature, and clocks.
5. Calculate communication demand from the model's real tensor sizes.
6. Only then decide whether UCX, NCCL, NIXL, or a more complex scheduler is justified.

Do not substitute port count for measurement, and do not use peak iperf3 throughput alone to declare a distributed inference design viable or impossible. Bandwidth, synchronization latency, tensor size, and compute time on both nodes belong in the same calculation.

Conclusion

This dual-Thor direct-connect test produced three clear findings:

• all four 10GbE links approached line rate when tested independently;
• aggregate throughput stopped near 12 Gbps with two or four links active;
• that bandwidth remained sufficient for small hidden-state transfers and the compressed-KV example in this article.

The useful lesson is broader than the missing 28 Gbps. If the cross-node payload is already small, adding more interfaces has little value. If communication is large, prove that the network is the limiting resource before investing in a more expensive interconnect.