Bandwidth Utilisation Side-Channel: Inferring Model Architecture from Memory Bus Patterns on AI Accelerators
For executives
AI models have proprietary architectures that represent significant research investment. That architecture can sometimes be inferred without accessing the model's weights or making inference queries: by monitoring the bandwidth utilisation pattern on the memory bus between the AI accelerator and its weight storage. Different architectures produce different bandwidth utilisation signatures during inference. A co-located tenant on the same infrastructure can observe those signatures.
Why bandwidth patterns leak architecture
During AI model inference, the compute accelerator reads model weights from memory (HBM or DRAM) as it processes each layer. The sequence and volume of memory reads corresponds to the model's architecture: how many layers it has, how large each layer's weight matrices are, and how much inter-layer data movement the architecture requires.
A Transformer model with 32 attention layers reads memory in a pattern of 32 attention block weight accesses followed by 32 feedforward block accesses. A different architecture produces a different bandwidth signature.
The multi-tenant scenario
In cloud GPU environments where multiple tenants share the same physical accelerator hardware, the memory bus between the GPU and HBM is shared. A tenant whose inference workload also uses that memory bus can measure bandwidth utilisation patterns via timing observations: if the attacker's memory accesses take longer than expected, the target is competing for bus bandwidth.
Research (Banerjee et al., 2021, "Bandwidth Utilization Side-Channel on ML Inference Accelerators") demonstrated that this bandwidth side-channel can reveal the approximate layer structure of Transformer models.
What the architecture reveals
Knowing the architecture of a target model reduces the query budget needed for model extraction via inference attacks. If an attacker knows a target model has approximately 32 Transformer layers with 4096 hidden dimensions, they can design a substitute model with those parameters and use a smaller number of inference queries to fit the weights.
Mitigation
Hardware partitioning via MIG provides separate memory paths for different MIG instances, reducing shared bandwidth access patterns. Memory access pattern obfuscation — padding memory accesses to uniform size and cadence — reduces the information content of bandwidth observations, at a performance cost.