Weights: the size of the published file for that quantisation (GGUF builds by unsloth, ggml-org, bartowski and the authors; MXFP4 for GPT-OSS), read from the Hugging Face API.
KV cache: tokens × bytes per token of the full-attention layers, plus sliding-window layers up to their window, plus the fixed state of linear-attention layers, so hybrid, sliding and MLA models are not overcounted. Q8 cache is about half of FP16.
Runtime: about 0.75 GiB of CUDA context and buffers plus the logits of a 512-token batch.
All sizes are GiB (1024³ bytes); a card "fits" when the total is within 97.5% of its memory. vLLM reserves most of the card for its cache by design, so nvidia-smi looks full there.