This is an observed result, not a memory estimate. The reviewed evidence includes the pinned model and GGUF revisions, independent artifact SHA-256, raw server output, repeated measurements, memory sampling and every failure used to locate the boundary.
The direct answer
Qwen3.8-27B Q4_K_M loaded with 8K, 32K and 64K FP16 KV allocations on one 24GB RTX 4090 and completed the measured 512-input/128-output workload. A binary search found a 124,928-token allocation succeeded; 125,952 and the fixed 128K allocation failed with CUDA out of memory.
What was actually tested
The artifact was Qwen3.8-27B-Q4_K_M.gguf from the pinned GGUF revision f1bfb127c64f7072bdd2cad55f258b9c8b2910fe. Its true size was 17,106,775,008 bytes and its independently calculated SHA-256 was 7e78da5d7e3ae28d178121f58646953305f3e5bd3cb46f4a75584e8b6c6fe169. The model record pins upstream base revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0; the evidence does not independently establish that the GGUF was converted from that exact upstream commit.
llama.cpp was built at commit 6d05498314db1b57f81c271080018aa2d0b89be9. Each server used FP16 K and V cache, a 512 batch and micro-batch, one parallel slot, and -ngl 999. No CPU layer offloading or automatic fallback was allowed.
Fixed-allocation results
| Allocated context | Result | Peak GPU | Prompt median | Generation median | TTFT median |
|---|---|---|---|---|---|
| 8K | Success | 16.2 GiB | 2036.0 tok/s | 47.7 tok/s | 259 ms |
| 32K | Success | 17.7 GiB | 2006.8 tok/s | 47.7 tok/s | 262 ms |
| 64K | Success | 19.8 GiB | 2016.0 tok/s | 47.7 tok/s | 261 ms |
| 128K | CUDA OOM | 15.4 GiB sampled | — | — | — |
Successful allocations received one warm-up followed by three measured repetitions. Every measured request used exactly 512 prompt tokens and generated 128 tokens. Medians are shown above; minimums, maximums and raw repeats remain in the hash-validated receipts. These results establish allocation fit and short-request operation, not near-full-context prompt processing.
Why the 128K allocation failed when 124,928 fit
At a 124,928-token allocation, sampled GPU use reached 23.5 GiB and all short-request repetitions completed. At 125,952, sampled use reached 23.5 GiB before CUDA allocation failed during the warm-up attempt. The fixed 131,072 allocation failed earlier while initializing its cache. Failed measurements are reported as failures—never replaced with estimated speeds.
What the result means in practice
With a 32K allocation and a 512-token measured prompt, median generation was 47.7 tokens per second, median prompt processing was 2006.8 tokens per second, and median time to first token was 262 ms. This confirms practical short-request operation on the tested cloud-hosted 4090; it does not predict long-prompt throughput or guarantee identical desktop thermals, clocks or CPU-side behavior.
Important limitations
No measured request contained more than 512 prompt tokens, so this benchmark does not establish long-context recall, quality, stability or throughput near the allocation limit. It used a RunPod Secure Cloud allocation, not a retail desktop. llama.cpp logged that the artifact’s extra blk.64/MTP tensors were unused, so this result describes the runtime behavior of that exact supported graph. We did not score output quality or quantify the loss from FP16/BF16 to Q4_K_M. Results must be re-tested after changing the artifact, runtime commit, cache precision, batch settings or hardware.
Receipts and raw evidence
All ten success and failure receipts pass the published Draft 2020-12 JSON Schema, internal consistency checks, canonical receipt-hash verification and review-status checks. The evidence archive contains environment capture, setup and download logs, server output, raw completion responses and 100 ms GPU-memory samples.