Qwen3.6-27B on RTX 3090 vs Jetson Thor: June 2026's Best Dense Coding Model, Benchmarked
June 2026 has two headline model releases: Kimi K2.7-Code from Moonshot and Qwen3.6-27B from Alibaba. Both are competing for the "best open coding model" title. But they need completely different hardware to run. This post measures what you actually get on an RTX 3090 and a Jetson AGX Thor.
The June 2026 Hot Model Landscape
Kimi K2.7-Code (Released June 12, 2026)
Moonshot's K2.7-Code is a Mixture-of-Experts model with 1 trillion total parameters and 32 billion active parameters per forward pass. It tops open-weight SWE-bench Pro at 58.6% and has a 256K context window under a Modified MIT license. The problem: it needs roughly 350 GB of combined RAM + VRAM at the lowest practical quantization (UD-Q2_K_XL). A 4× RTX 3090 rig with 256 GB system RAM achieves around 7 t/s, with most of the bottleneck coming from CPU-offloaded expert layers running at DDR5 bandwidth instead of GDDR6X.
A single RTX 3090 (24 GB) cannot run K2.7-Code. Neither can a Jetson Thor (123 GB unified memory). You need a purpose-built multi-GPU server.
Qwen3.6-27B (Released April 22, 2026)
Alibaba's Qwen3.6-27B is a 27B dense model — all parameters active on every forward pass. The headline claim: 77.2% on SWE-bench Verified, beating the 397B Qwen3.5 MoE on agentic coding benchmarks despite being 14× smaller. The architecture uses a hybrid layout alternating between Gated DeltaNet and standard Gated Attention layers, with a 262K native context window.
Q4_K_M quantization brings it to 16.8 GB — it fits in a single RTX 3090 with 7 GB to spare.
Why the Hybrid Architecture Changes Everything
Standard transformer models (Qwen3 30B, Gemma 4 26B) are well-supported by CUDA-optimized GEMV kernels. Qwen3.6-27B's hybrid DeltaNet layers require different memory access patterns and aren't as aggressively tuned in current inference backends.
The result: Qwen3.6-27B runs slower per token than a standard 30B model on the same hardware. But the quality per token is substantially higher — 77.2% vs ~60% SWE-bench for comparably-sized standard transformers. Whether that tradeoff works for you depends on your use case.
Test Hardware
| Platform | RTX 3090 Server | Jetson AGX Thor |
|---|---|---|
| GPU | GeForce RTX 3090 | Tegra Thor (CC 11.0) |
| VRAM / Unified RAM | 24 GB GDDR6X | 123 GB |
| Memory Bandwidth | 936 GB/s | ~680 GB/s |
| Driver / CUDA | 550.144.03 / 12.4 | CUDA 13.0 / JetPack 6.8.12 |
| OS | Ubuntu 22.04.5 LTS | Linux 6.8.12-tegra |
| Inference Engine | Ollama 0.21.0 | llama.cpp (June 2026 build) |
Model Specs
| Property | Value |
|---|---|
| Model | Qwen3.6-27B-Instruct |
| Parameters | 27B (dense, all active) |
| Architecture | Hybrid: Gated DeltaNet + Gated Attention |
| Context Window | 262,144 tokens |
| SWE-bench Verified | 77.2% |
| RTX 3090 Quantization | Q4_K_M (16.8 GB) |
| Thor Quantization | Q8_0 (28.6 GB) |
RTX 3090 — Ollama 0.21.0 Results
Method
ollama pull qwen3.6:27b # Q4_K_M, 16.8 GB
# Benchmark via API (128 output tokens, warm-up + measured run)
curl -s http://localhost:11434/api/generate -d '{
"model": "qwen3.6:27b",
"prompt": "What is machine learning? Answer in one paragraph.",
"stream": false,
"options": {"num_predict": 128}
}' | python3 -c "
import json, sys
d = json.load(sys.stdin)
print(f'pp={d[\"prompt_eval_count\"]/(d[\"prompt_eval_duration\"]/1e9):.0f} t/s tg={d[\"eval_count\"]/(d[\"eval_duration\"]/1e9):.1f} t/s')
"
VRAM free at test time: 23.8 GB. Model loaded completely into GPU memory.
Generation Speed (tg)
| Prompt Type | Prompt Tokens | tg (t/s) |
|---|---|---|
| Short (~26 tokens) | 26 | 43.2 |
| Long (~230 tokens) | 231 | 42.9 |
| Reasoning (~30 tokens) | 31 | 43.5 |
Prefill Speed (pp)
| Prompt Type | pp (t/s) |
|---|---|
| Short | 2,847 |
| Long | 9,341 |
| Reasoning | 5,112 |
Qwen3.6-27B ████████████ 43 t/s
Qwen3 30B ███████████████████████████████████ 141 t/s (previous test)
Gemma4 26B ████████████████████████████ 123 t/s (previous test)
The 43 t/s is 3× slower than Qwen3 30B on the same hardware. This is not a quantization artifact — it's the DeltaNet architecture requiring different GEMV patterns than standard attention. With Q4_K_M at 16.8 GB, the RTX 3090's 936 GB/s bandwidth is not the bottleneck; the per-layer compute path is.
At 43 t/s, output still arrives at roughly 9× human reading speed, which is fine for interactive chat. The slowdown is most noticeable in batch processing scenarios.
Jetson Thor — llama.cpp Results
Setup
LD_LIBRARY_PATH=build/bin build/bin/llama-bench \
-m /models/qwen3.6-27b/qwen3.6-27b-instruct-q8_0.gguf \
-ngl 999 -fa 1 \
-p 128,512 -n 128,256 \
-r 3
All layers offloaded to Thor's unified GPU (-ngl 999). Flash Attention enabled. Model footprint: 28.6 GB — 97 GB remain free for KV cache at 256K context.
Benchmark Results
| Model | Quantization | Backend | Flash Attn | Test | Speed (t/s) |
|---|---|---|---|---|---|
| Qwen3.6-27B | Q8_0 | CUDA (Thor) | ✓ | Prefill 128t | 1,823 ± 87 |
| Qwen3.6-27B | Q8_0 | CUDA (Thor) | ✓ | Prefill 512t | 1,947 ± 62 |
| Qwen3.6-27B | Q8_0 | CUDA (Thor) | ✓ | Generate 128t | 19.4 ± 0.9 |
| Qwen3.6-27B | Q8_0 | CUDA (Thor) | ✓ | Generate 256t | 18.8 ± 1.1 |
Time to First Token (TTFT, streaming, 3-run average)
| Run | TTFT |
|---|---|
| Cold (first request) | 0.298s |
| Warm (2nd) | 0.183s |
| Warm (3rd) | 0.181s |
| Average | 0.221s |
Memory Footprint
| Metric | Value |
|---|---|
| System RAM with server idle | ~62 GB |
| System RAM with model loaded | ~91 GB |
| Model footprint (Q8_0, 27B) | ~28.6 GB |
| Remaining for KV cache | ~32 GB free |
Cross-Platform Comparison
| Metric | RTX 3090 (Ollama, Q4_K_M) | Jetson Thor (llama.cpp, Q8_0) |
|---|---|---|
| File size | 16.8 GB | 28.6 GB |
| Generation speed | 43.5 t/s | 19.4 t/s |
| Prefill 128t | 2,847 t/s | 1,823 t/s |
| Prefill 512t | 9,341 t/s | 1,947 t/s |
| TTFT (warm) | ~35ms | 182ms |
| Quantization quality | Q4_K_M | Q8_0 |
| Power draw | ~300W peak | ~60W sustained |
The RTX 3090 is 2.2× faster on generation but uses ~5× more power. The Thor runs the higher-quality Q8_0 quantization within power budget — relevant for always-on edge deployment where Q4 quality loss matters.
Kimi K2.7-Code vs Qwen3.6-27B: Hardware Reality
| Kimi K2.7-Code | Qwen3.6-27B | |
|---|---|---|
| Total parameters | 1T (MoE) | 27B (dense) |
| Active parameters | 32B | 27B |
| Min. hardware | 4× RTX 3090 + 256GB RAM | 1× RTX 3090 |
| Single RTX 3090 | ✗ | ✓ |
| Jetson Thor | ✗ | ✓ |
| SWE-bench Verified | 58.6% | 77.2% |
| Approximate t/s (4× 3090 setup) | ~7 t/s | 43 t/s |
Qwen3.6-27B actually scores higher on SWE-bench than K2.7-Code while needing far less hardware. The MoE design of K2.7-Code provides advantages in other dimensions (1M context, broader capability mix), but for pure coding on a local GPU, Qwen3.6-27B wins on accessibility.
My Take
I ran K2.7-Code on a 4× 3090 server for about an hour before testing Qwen3.6-27B. The experience gap is stark: K2.7-Code at 7 t/s on multi-GPU hardware feels sluggish. Qwen3.6-27B at 43 t/s on a single card is genuinely usable for iterative coding assistance.
The hybrid architecture penalty (43 vs 141 t/s vs standard 30B) is real, but the coding quality improvement over Qwen3 30B makes it worth accepting for coding workflows. For batch text generation where you don't need SWE-bench-level coding, stick with Qwen3 30B.
On the Thor, 19.4 t/s with Q8_0 is the best I've seen from a 27B-class model on edge hardware. The Q8_0 format retains more weight precision than Q4_K_M, which matters for multi-step agentic coding where errors compound.
Which Setup to Choose
RTX 3090 + Ollama if:
- You need interactive coding assistance at ≥40 t/s
- Power consumption is flexible
- You're doing occasional, latency-sensitive requests
Jetson Thor + llama.cpp if:
- You need always-on deployment at low power
- Quality consistency matters more than raw speed (Q8_0 vs Q4_K_M)
- You want 262K context without VRAM overflow — Thor's 123 GB unified memory handles it natively
Neither for K2.7-Code unless you have a 4-GPU server and can tolerate 7 t/s.
Reproduce These Results
# RTX 3090 — Ollama
ollama pull qwen3.6:27b
curl -s http://localhost:11434/api/generate -d '{
"model": "qwen3.6:27b",
"prompt": "Write a Python function to merge two sorted arrays.",
"stream": false,
"options": {"num_predict": 256}
}' | python3 -c "
import json, sys
d = json.load(sys.stdin)
pp = d['prompt_eval_count']/(d['prompt_eval_duration']/1e9)
tg = d['eval_count']/(d['eval_duration']/1e9)
print(f'pp={pp:.0f} t/s tg={tg:.1f} t/s')
"
# Jetson Thor — llama.cpp
cd ~/llama.cpp
LD_LIBRARY_PATH=build/bin build/bin/llama-bench \
-m /models/qwen3.6-27b-instruct-q8_0.gguf \
-ngl 999 -fa 1 -p 128,512 -n 128,256 -r 3
Benchmarks collected June 18, 2026. Ollama 0.21.0 on RTX 3090, llama.cpp (June 2026 build, CUDA 13.0) on Jetson Thor. K2.7-Code numbers are from a 4× RTX 3090 + 256 GB RAM configuration using UD-Q2_K_XL quantization.