How Kubernetes Namespace Isolation Fails for GPU Workloads: Why RBAC Is Not Enough
For executives
Kubernetes namespace isolation is the standard mechanism for separating workloads from different teams, projects, or tenants within a cluster. Role-Based Access Control (RBAC) controls who can create, modify, or delete resources in each namespace. Together, namespaces and RBAC are the primary multi-tenancy mechanism most Kubernetes operators use. For CPU workloads, this provides meaningful isolation. For GPU workloads, it is incomplete. This article explains the gaps.
What Kubernetes namespace isolation provides
A Kubernetes namespace creates a scope for names. Resources (pods, services, configmaps, secrets) in one namespace are separate from resources with the same name in another namespace. RBAC can restrict which users or service accounts can create or modify resources in each namespace.
For CPU workloads, namespace isolation combined with network policies and RBAC provides a reasonable multi-tenancy model: different teams' workloads run in different namespaces, cannot see each other's resources, and cannot modify each other's configurations.
What namespace isolation does not provide
Kubernetes namespaces are a resource naming and access control mechanism. They do not provide:
Process isolation: pods in different namespaces run on the same physical nodes. Their processes share the host kernel. A process in namespace A and a process in namespace B that happen to be scheduled on the same node share the host's kernel, CPU resources, and — for GPU workloads — the GPU and its VRAM.
Network isolation: without explicit network policies, pods in all namespaces can communicate with each other. Network policies provide isolation but must be explicitly configured and verified.
GPU isolation: GPU resources are shared at the node level, not the namespace level. Two pods in different namespaces scheduled on the same GPU node share that node's GPUs unless MIG or other hardware isolation is in place. The namespace boundary provides no protection for GPU memory or compute.
The RBAC gap for GPU resources
RBAC can prevent a user in namespace B from creating pods in namespace A or reading namespace A's secrets. It cannot prevent a pod in namespace B, running on the same node as a pod in namespace A, from exploiting a GPU driver vulnerability to access namespace A's GPU memory.
The GPU driver is a host resource. Accessing it requires a CUDA context, which requires access to the GPU device files. If both pods have GPU device file access (because both requested GPUs and were allocated them), both have a path to the GPU driver via their device file access. RBAC controls Kubernetes API access; it does not control GPU hardware access.
The scheduling gap
Kubernetes schedules GPU pods to nodes based on GPU resource availability. Two pods in different namespaces requesting the same GPU model will be scheduled to the same node if that is where available GPUs are — namespace is not a scheduling constraint unless explicitly configured with node affinity rules that separate different tenants to different nodes.
In clusters with high GPU utilisation, different tenants' GPU pods will frequently land on the same physical nodes. The namespace boundary provides Kubernetes-level separation (different resources, different RBAC, different service accounts) but no physical separation. The GPU driver, the Container Toolkit, and the VRAM are all shared.
What additional controls are required
Effective GPU multi-tenancy requires controls beyond namespaces and RBAC:
Node-level tenant separation: assigning specific GPU nodes to specific tenants via Kubernetes node taints and tolerations, combined with node affinity rules that prevent cross-tenant pod scheduling on the same node.
Hardware isolation via MIG: on supported hardware, MIG partitions provide hardware-level isolation between workloads even when they share the same physical node.
Network policy enforcement: explicit network policies blocking cross-namespace pod communication.
Runtime monitoring at the GPU layer: detecting cross-tenant access attempts via GPU telemetry, not just via Kubernetes API audit logs.