This is a build report for a 27B model on two consumer graphics cards that were never meant to work together: a 16 GB RTX 4060 Ti and an 8 GB RTX 3060, the second of them in a four-lane slot. Every number below was measured on that machine unless it is marked as a specification or a citation. Capacities are derived from the MiB the tools report and divided by 1024, so “GB” in this article means the binary quantity throughout; file sizes quoted from Hugging Face are decimal and are labelled where they appear.

Summary

Qwen3.8-27B runs at a 131,072-token context across the two cards with about 1.1 GB of VRAM to spare. Enabling llama.cpp’s MTP speculative decoding together with FlashAttention took generation from 12.8 tok/s to 24.8 and 32.2 tok/s on two runs. The faster of those is 2.52x the baseline, which is above the 1.7-1.9x band that published reports cluster in, and the section on throughput explains why that comparison should be read carefully rather than as a result.

Three things turned out to matter more than the hardware. CUDA does not number devices the way nvidia-smi does. MTP does nothing unless you pass its flag, and the log will tell you so if you read it. And KV cache quantization saved 23% less memory than the arithmetic predicts. That cuts in two directions at once, which the budget section works through: the per-token cost is lower than the formula says, while everything else the runtime puts on the card costs more than the formula allows for. The two errors happen to cancel in the configuration we ran, and they do not cancel anywhere else.

Hardware and operating environment

Item	Value
GPU 0	RTX 3060 8 GB, in an x4 slot, card maximum Gen3
GPU 1	RTX 4060 Ti 16 GB; the card itself is Gen4 x8, so it negotiates eight lanes even in a physically x16 slot
Driver / CUDA	596.36 / CUDA 13.2
System RAM	63.9 GB
Model	Qwen3.8-27B, Q3_K_S quantization, 12.8 GB of weights
Runtime	llama.cpp b10901, Windows CUDA build

The x8 figure belongs to the 4060 Ti, not to the slot it sits in. That distinction matters when you are reading a motherboard manual and trying to work out where the bottleneck is.

nvidia-smi will sometimes report either link as Gen1. That is an idle downclock, not a fault, and the field to read instead is pcie.link.gen.max. Checking the wrong field here will send you looking for a hardware problem that does not exist.

Qwen3.8-27B is a 27.78B dense model with 16 of its 64 layers using full attention (full_attention_interval=4), num_key_value_heads=4, head_dim=256, a native 262,144-token context and an Apache 2.0 licence. Those figures come from the model card, not from this machine, and the KV cache arithmetic later in this article is derived from them.

Configuration

CUDA numbers the devices by speed, not by slot

This is the first thing to get wrong and the easiest one to not notice. nvidia-smi orders cards by PCI bus, so the 3060 is device 0. CUDA’s default ordering is by performance, so the 4060 Ti is device 0. Set CUDA_DEVICE_ORDER=PCI_BUS_ID and the two agree.

	CUDA0	CUDA1
default, no environment variable	4060 Ti	3060
CUDA_DEVICE_ORDER=PCI_BUS_ID	3060	4060 Ti
nvidia-smi numbering	3060 (as 0)	4060 Ti (as 1)

Verified with llama-server --list-devices rather than assumed. If you set --main-gpu against the wrong numbering you will put the bulk of the model on the 8 GB card and spend the afternoon debugging an out-of-memory error that has nothing to do with memory.

Layer split, and why a four-lane slot is survivable

-sm layer, the default, gives each card a contiguous block of layers and passes one hidden-state vector across the PCIe link at each boundary. The model’s hidden size is 5120, so at 16-bit precision that vector is 10,240 bytes. Ten kilobytes per token is nothing, and a four-lane Gen3 link, specified at roughly 3.9 GB/s decimal (about 3.6 GiB/s), is not the constraint.

-sm row splits individual tensors across both cards and exchanges partial results continuously. That does saturate an x4 link. If you have a card in a short slot, the split mode is the setting that decides whether the slot matters.

The command that ended up working

set CUDA_DEVICE_ORDER=PCI_BUS_ID

llama-server.exe ^
  -m Qwen3.8-27B-Q3_K_S.gguf ^
  --spec-type draft-mtp --spec-draft-n-max 6 ^
  -ngl 999 -sm layer --main-gpu 1 ^
  --ctx-size 131072 --parallel 1 --predict 16000 ^
  --flash-attn on ^
  --cache-type-k q8_0 --cache-type-v q8_0 ^
  --jinja --reasoning-budget 8000 ^
  --cache-ram 1024 --ctx-checkpoints 8 ^
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 ^
  --presence-penalty 0.0 --repeat-penalty 1.0 ^
  --host 0.0.0.0 --port 8088

Two defaults are worth overriding deliberately rather than inheriting. --parallel documents its default as -1 (automatic); on this machine that resolved to 4. The resolution is what was recorded; the consequence is arithmetic, since four slots each want their own cache and 4 x 4.00 GB is 16.0 GB against 7.42 GB of headroom. --cache-ram and --ctx-checkpoints default to 8192 MB and 32 here, both larger than this build needs.

The sampling parameters are the ones the model card recommends for thinking mode, not tuning of ours.

Results

Throughput

The three rows below share the same input prompt. The two MTP rows ran under the final configuration above. The baseline row’s context size and KV cache type were not recorded at the time, so read it as a same-prompt comparison and not as a controlled one.

Setting	tok/s	Detail
no MTP, no FlashAttention	12.8	baseline
MTP --spec-draft-n-max 6 + --flash-attn on	24.8	144 tokens generated, draft acceptance 0.599, mean accepted length 4.59
same	32.2	259 tokens generated, draft acceptance 0.673, mean accepted length 5.04

Mean accepted length is acceptance x 6 + 1 and therefore includes the bonus token that is always confirmed. A reader who computes 0.673 x 6 = 4.04 and compares it to 5.04 will think one of the two is wrong; neither is.

MTP has to be asked for

Loading a model with MTP tensors is not the same as using them. Without the flag the log discards them one line at a time:

model has unused tensor blk.64.nextn.* -- ignoring

With --spec-type draft-mtp those lines disappear entirely and this appears instead:

common_speculative_init_result: creating MTP draft context

Grep for either string. It is a faster check than reading a throughput number and wondering.

On the 2.52x

Published MTP speedups for this model family sit in a narrow band: about 1.85x, 1.73x on an RTX PRO 6000, and 71% in a coding session. Our 1.94x sits at the top of that range or just past it, depending on where you close the band. Our 2.52x is well clear of it. Two figures above the published cluster, then, and not one.

The obvious explanation is output length. The 32.2 tok/s run generated 259 tokens against the slower run’s 144, and longer outputs favour speculative decoding because accepted runs compound. That explanation is a candidate, not a finding. No re-run was performed at matched output length to isolate the variable, so the cause of the gap between our number and everyone else’s is unconfirmed. It is stated here rather than smoothed away, because a single unqualified figure above a published range is the kind of thing that gets quoted.

KV cache quantization, measured rather than assumed

Passing --cache-type-k q8_0 --cache-type-v q8_0 is a statement of intent. Whether the allocation actually changed is a separate question, so it was measured by A/B at a 32,768-token context.

Context 32,768	VRAM
default (f16)	18,619 MiB
--cache-type-k q8_0 --cache-type-v q8_0	17,833 MiB
difference	786 MiB

Halving 64 KB per token predicts a saving of 32 KB x 32,768 = 1,024 MiB. The measured 786 MiB is 23% short of that. Working backwards from 786 MiB gives an actual f16 cost of roughly 50 KB per token. Those two statements are close without being identical: a 64-to-50 ratio is 1.28, which implies a 22% shortfall rather than 23%, and they land on the same figure only if the true rate is nearer 49 KB. The article rounds to 50 because that is the precision one A/B measurement supports.

That agreement is worth naming for what it is. The 50 KB figure is derived from the 786 MiB measurement, so the two matching is internal consistency, not independent confirmation. It tells us the shortfall is a single systematic effect rather than a measurement error, and it does not tell us why the theoretical rate overshoots.

VRAM budget

The two cards hold 24,572 MiB (24.0 GB) between them. Measured free memory after boot was 22,217 MiB (21.7 GB), so Windows and the display were already holding 2,355 MiB before llama.cpp started. Subtracting a further 1.5 GB for the CUDA context and compute buffers leaves about 20.2 GB to be shared between weights and KV cache. That 1.5 GB is an allowance rather than a measurement, and the section below the table shows what it should have been.

The per-token cache cost follows from the model’s own configuration. Each full-attention layer stores a key and a value for 4 KV heads of 256 dimensions at 2 bytes each: 2 x 4 x 256 x 2 = 4,096 bytes, or 4 KiB per layer per token. Sixteen of the 64 layers use full attention, so the total is 64 KiB per token at f16 and half that under Q8_0.

Quantization	Weights	KV headroom	Context at f16	Context at q8_0
Q3_K_S	12.78 GB	7.42 GB	119K	237K
Q3_K_M	13.61 GB	6.59 GB	105K	211K
IQ4_XS	14.50 GB	5.70 GB	91K	182K
Q4_K_M	16.55 GB	3.65 GB	58K	117K

The release repository lists these files at 13.72, 14.61, 15.57 and 17.77 GB, which are decimal figures; the weights column above is the same files in binary units. Two decimal places are carried there so that every row’s weights and headroom add to the 20.2 GB the budget allows. At one decimal the Q4_K_M row alone stops summing, 16.5 + 3.6 giving 20.1, and a single row that does not add up reads as an error rather than as rounding. The 12.78 GB in that column and the 12.8 GB used elsewhere in this article are the same file at two precisions.

Every context figure in that table is computed at the theoretical 64 KB per token and on the 1.5 GB allowance. On that basis Q3_K_S, Q3_K_M and IQ4_XS all clear 131,072 tokens under a Q8_0 cache, and Q4_K_M at 117K does not.

The machine disagrees, and the occupancy measurements further down are what disagree with it. Running Q3_K_S at 131,072 tokens with a Q8_0 cache, the total sitting on the two cards above the model weights was 7,962 MiB. The table’s model for that same row is 4,096 MiB of KV plus the 1,536 MiB allowance, which is 5,632 MiB. The budget under-predicts by 2,330 MiB, and it under-predicts by the same amount in every row, because the allowance is identical in all of them.

Substituting the measured 7,962 MiB changes which formats fit:

Quantization	Weights	Available above weights	Needed at 128K	Spare
Q3_K_S	13,084 MiB	9,133 MiB	7,962 MiB	1,171 MiB
Q3_K_M	13,933 MiB	8,284 MiB	7,962 MiB	322 MiB
IQ4_XS	14,849 MiB	7,368 MiB	7,962 MiB	594 MiB short
Q4_K_M	16,947 MiB	5,270 MiB	7,962 MiB	2,692 MiB short

The Q3_K_S row is not evidence for any of this. Its weights appear in both the available and the needed column and cancel out, so that row returns 22,217 – 21,046 = 1,171 MiB whatever model you put in it. It is an identity, and it would have come out the same had the envelope been wrong. The other three rows are predictions, and they are the whole content of the table.

Short, in that last column, means the format cannot be held entirely in VRAM at 131,072 tokens. It does not mean the format will not run: none of the three were tried, and what llama.cpp does when an allocation of that size does not fit was not observed here.

On this reading two formats hold 128K where the theoretical table says three, so the exclusivity that table asserts is understated. That is the reverse of what the per-token rate alone would suggest, which is the whole point: the capacity model is wrong in two directions, and the run that was performed happens to sit where they cancel.

One measurement cannot separate the two errors. Inside that 7,962 MiB, if the KV cache is the theoretical 4,096 MiB then everything else accounts for 3,866 MiB; if it is the measured-rate 3,200 MiB then everything else accounts for 4,762 MiB. Both readings fit the same occupancy figure and nothing measured here decides between them. What the measurement does fix is the total, and every verdict in the table above rests on the total alone. The substitution further assumes the non-weight cost does not move much with the quantization format, which holds to the extent that compute buffers and the draft context follow the model’s shape and the context length rather than the weight precision. That is an assumption, not a result, and it is worth saying how much weight each row puts on it. The envelope would have to be wrong by 4.0% to cost Q3_K_M its margin, by 7.5% before IQ4_XS would fit, and by 33.8% before Q4_K_M would. Q4_K_M’s failure survives any plausible revision and IQ4_XS’s is solid, but Q3_K_M clears by less than the uncertainty in the assumption underwriting it. Read that row as marginal rather than as a pass.

Q3_K_S is what ran. The source record says which format was used and not why it was picked, so the headroom above is the case for it rather than the reason behind it.

Moving the display output

With both cards driving nothing in particular, the 4060 Ti was carrying Windows desktop work it did not need to. Moving the monitor to the 3060 was supposed to hand that back.

	before	after
3060 free	8,036 MiB	7,253 MiB
4060 Ti free	14,388 MiB	14,964 MiB
total free	21.90 GB	21.70 GB

The 4060 Ti gained 576 MiB and the 3060 gave up 783 MiB, so the total went slightly down. The move is still worth making with --main-gpu 1, because free memory on the larger card is what constrains a layer split, but the memory moved between cards rather than coming back.

1,146 MiB stayed allocated on the 4060 Ti afterwards, according to nvidia-smi. Windows keeps a card without a display attached as a rendering device for some applications, so the allocation never fully leaves. A separate look with the Windows performance counters showed process-level committed memory of 610 MB for the desktop window manager, 483 MB for two remote desktop sessions, 174 MB for Explorer and 226 MB for everything else, totalling 1,493 MB. Those are the counters’ own megabytes as the tool reports them, and they are the one set of capacity figures in this article not converted into the binary convention declared at the top.

Those two figures are not two views of one number. The performance counters report committed virtual memory and nvidia-smi reports dedicated physical VRAM allocation, so the 1,493 MB does not decompose the 1,146 MiB and the difference between them is not a discrepancy to be explained. They are separate instruments and should not be added to or subtracted from one another.

Final occupancy

State	3060	4060 Ti	Total
no MTP or FlashAttention, context 131,072	7,564 MiB	14,110 MiB	21.2 GB
MTP and FlashAttention on, context 131,072	7,710 MiB	15,691 MiB	22.9 GB

That leaves 1,171 MiB, about 1.1 GB, unused out of 24.0 GB. The KV cache is allocated in full at load time and does not grow during use: across a request the total moved from 23,401 to 23,439 MiB.

These occupancy figures are the measurement behind the budget section’s warning. The budget allows 1.5 GB for everything that is neither weights nor KV cache. At 131,072 tokens with MTP and FlashAttention on, the figure implied by the occupancy is between 3.8 and 4.6 GB, depending on which KV rate you assume. Without MTP and FlashAttention the same subtraction gives 6,235 MiB above weights rather than 7,962, so roughly 1.7 GB separates the two rows. Attributing all of that to the speedup assumes they differ only in MTP and FlashAttention, and the baseline’s cache type was never recorded, so treat it as a difference between two runs rather than as the price of the feature. What the remainder consists of is not something this measurement resolves: the draft context carries a KV cache of its own, compute buffers scale with the model, and the CUDA allocator reserves more than it hands out. The --cache-ram and --ctx-checkpoints settings are not on that list, because they govern a host-RAM reuse pool rather than device memory.

What did not work

Vision would not load

The multimodal projector is 885 MiB. Under a layer split the 3060 fills first, and the allocation for it then landed on device 0 and failed with cudaMalloc failed: out of memory. Vision on this pair would mean giving up context length, and context was the point of the build.

An empty response at a 160-token limit

This model emits its reasoning into a separate reasoning_content field. With max_tokens set to 160 the reasoning consumed the entire budget, and the response came back with finish_reason: length and a content field of zero length. Nothing in the error suggests a token limit, because there is no error. A limit of 400 or more behaves normally.

An f16 KV cache at 128K

At 131,072 tokens the theoretical 64 KB per token puts the f16 cache alone at 8,192 MiB, which is 8.00 GB. With 12.8 GB of weights and the budget’s 1.5 GB allowance the demand is 22.30 GB against the 21.7 GB available, so it does not fit. At the measured 50 KB per token the cache falls to about 6.25 GB and the total to 20.55 GB, which on that same 1.5 GB allowance would leave about 1.2 GB and would fit. The allowance is the problem, and the next paragraph is where it gives way.

That second figure is the budget’s answer rather than the machine’s, and it carries the 1.5 GB allowance the previous section showed to be low by more than 2 GB. Substituting the measured cost instead: an f16 cache is double the Q8_0 one, so the total comes to 24.6 GB if the KV cache sits at the theoretical rate and 23.7 GB if it sits at the measured rate. Both exceed 21.7 GB, by 2.9 GB and by 2.0 GB. Those are the two internally consistent pairings; pairing each KV rate with the other’s residue gives 22.8 GB and 25.4 GB, so all four combinations exceed the pool and the closest still misses by 1,133 MiB. The split this article declines to resolve therefore cannot rescue the case either way. An f16 cache at 128K does not fit on any combination of the two readings, and Q8_0 was not a precaution that turned out to be unneeded.

All of that was computed. No 128K f16 run was ever executed and no out-of-memory error was recorded for one, so what this section establishes is that the configuration was never available, not that it was tried and failed.

Limits

Q3 quality is absent from the benchmarks we found

Q3_K_S was chosen for context headroom inside 24 GB. The available quantization benchmarks for this model cover BF16, Q8_0, Q4_K_M, Q2_K_XL and IQ1. Q3 appears in none of them. We found no published quality figure for the format this build runs on, which is a weaker claim than none existing, so anyone using it for careful reasoning or for code should verify quality on their own workload before depending on it. Nobody has published the number, so nobody can tell you what it costs.

Concurrency

This configuration pins --parallel 1 because the KV cache allocation scales with it and there is only 1.1 GB spare. It is a single-user setup. Serving more than one session at this context length is not a tuning change, it is a different machine.

Scope of the measurements

One machine, one driver version, one llama.cpp build, and two MTP runs rather than a swept set. The throughput figures are real and reproducible on this rig; treating them as a benchmark for the 4060 Ti and 3060 combination in general would be reading more into them than was measured.