How GPU Memory Works: VRAM, Shared Memory, and Where Isolation Assumptions Break
For executives
GPU memory is not the same as CPU memory. It has different physical characteristics, different performance properties, different allocation mechanisms, and critically different isolation guarantees. Understanding how GPU memory is structured and managed is essential context for the memory-based vulnerability classes in this library: residual VRAM data, the Whispering Pixels register vulnerability, model weight exfiltration, and GPU memory snooping.
The memory hierarchy in a GPU
VRAM (Video RAM / High-Bandwidth Memory): the GPU's main memory. Modern AI GPUs use High-Bandwidth Memory (HBM3) stacked on the same chip package as the GPU die, achieving bandwidths of 3.35 TB/s (H100) compared to approximately 100 GB/s for DDR5 CPU memory. VRAM is large (80GB on an H100, 192GB on an MI300X), persistent across operations, and not automatically cleared when a GPU process ends.
Shared memory (SMEM): a small, fast scratchpad memory local to each streaming multiprocessor. On-chip, extremely fast, and partitioned between the threads running on that multiprocessor. SMEM is automatically cleared when a kernel completes — its contents do not persist between kernel launches.
Registers: the fastest storage, entirely local to each CUDA thread. CVE-2024-21969 demonstrated that on AMD hardware, register clearing between processes is not guaranteed by default.
L2 cache: shared across all streaming multiprocessors on the GPU. A shared resource, creating the cache-based side-channel attack surface documented in the Prime+Probe article.
How VRAM allocation works
When a CUDA program calls cudaMalloc(), it requests a block of VRAM from the GPU's memory allocator. The allocator returns a pointer to an available block. That block is not zeroed before it is returned — it contains whatever the previous allocation in that address range wrote.
On the CPU, the OS enforces that freshly allocated virtual memory pages are presented to processes with zero content. On the GPU, no equivalent OS-level guarantee exists. The VRAM block returned by cudaMalloc() may contain data from a previous allocation by the same process, a previous process, or in shared GPU environments, a previous tenant.
Unified Virtual Memory (UVM)
NVIDIA's Unified Virtual Memory allows GPU and CPU to share a virtual address space. Memory allocated with cudaMallocManaged() is accessible from both GPU and CPU code, with the driver transparently migrating pages between VRAM and system RAM as needed.
The GPUBreach research (published at IEEE S&P 2026, presented by Stealthium) demonstrated that UVM's page management behaviour creates observable patterns — specific allocation sizes, eviction timing, and page migration sequences — that are detectable by monitoring and that distinguish legitimate AI workloads from exploitation attempts.
Multi-GPU memory and NVLink
In multi-GPU systems, NVLink allows GPU memories to be accessed directly by peer GPUs. A CUDA program on GPU 0 can read from or write to GPU 1's VRAM via NVLink Peer-to-Peer (P2P) access. P2P access between GPUs owned by different tenants requires explicit agreement from both parties. An attacker with kernel-level access on the host can bypass this requirement, enabling cross-tenant P2P VRAM access without the target tenant's awareness.