Kubernetes Multi-Tenancy and Workload Isolation

Multi-tenancy in Kubernetes is not one feature—it is a set of boundaries you choose based on who trusts whom. A namespace with RBAC may be enough for two internal teams on one platform. It is not enough when a customer uploads untrusted code or when a regulator expects an independent control plane. This guide maps tenant types to isolation models and points to implementation articles instead of repeating their manifests.

Define a tenant

A tenant is whoever shares—or must not share—a trust boundary. Common definitions:

• Team or platform squad
• External customer or SaaS subscriber
• Environment (dev, staging, prod)
• Business unit or cost center
• Application group sharing one release cadence
• Regulated workload subject to audit scope
• Untrusted code (CI jobs, serverless user functions, ML notebooks)

For each tenant, list who must trust whom:

The answer drives whether namespace policy is sufficient or you need dedicated nodes, sandboxes, or separate clusters.

Soft versus stronger isolation

Isolation strength increases as you move from API policy toward separate control planes:

Namespaces are the default starting point—they are not sufficient alone when tenants are hostile to each other or when compromise must not expose the host kernel. Treat namespace isolation as policy on a shared cluster, not as a hypervisor substitute.

Namespace-based soft multi-tenancy

A namespace is the unit of soft isolation. Stack controls rather than relying on naming alone:

Namespace
├── RBAC (Roles and RoleBindings)
├── ResourceQuota
├── LimitRange
├── NetworkPolicy
├── Pod Security Admission labels
├── ServiceAccounts (namespaced workload identities)
└── PVC policy and approved StorageClasses

Platform teams usually own cluster-scoped objects—StorageClass, IngressClass, PriorityClass, and admission configuration—while tenant admins receive namespace-scoped Roles only.

API isolation

Kubernetes multi-tenancy fails when API access is wider than you assume. Review these paths beyond basic Role rules:

• Namespaced Roles — grant get, list, watch on Secrets, ConfigMaps, or Pods only where intended
• Role escalation — escalate on Roles or ClusterRoles allows users to define permissions they do not already possess
• Binding escalation — bind allows users to grant a Role or ClusterRole whose permissions they do not already possess
• Secret access — a token in namespace A must not read Secrets in namespace B
• Exec and port-forward — pods/exec and pods/portforward provide API-mediated access to workloads and require separate RBAC restrictions. Port forwarding may bypass ordinary network-level access controls
• Impersonation — impersonate allows a caller to act as another user, group, or ServiceAccount; it is separate from bind and escalate
• Cluster-scoped resources — PersistentVolume, ClusterRole, ValidatingWebhookConfiguration, and CRDs are outside namespace walls
• Admission changes — tenants who edit webhook configs or PSA labels can weaken enforcement for everyone

See authentication and admission control for how requests reach policy checks after RBAC.

Resource fairness

Quotas address capacity hogging, not cryptographic isolation:

• ResourceQuota — cap Pods, Services, PVCs, and aggregate CPU/memory/storage per namespace
• LimitRange — enforce minimum and maximum container resources so one Pod cannot claim an entire node
• Requests and limits — give the scheduler and kubelet predictable footprints
• PriorityClasses — prefer platform or production workloads during contention (use sparingly across tenants)
• Storage quotas — limit PVC count and total storage per tenant

API Priority and Fairness protects the shared API server from noisy controllers or tenants by classifying, queueing, and fairly dispatching API requests. It is different from Pod PriorityClass: one protects API-server request capacity, while the other affects workload scheduling and preemption.

Quotas stop a tenant from creating ten thousand Pods or exhausting cluster IP space. They do not reserve dedicated CPUs or isolate disk I/O on a shared node. Pair quotas with monitoring and node pools when SLA fairness matters.

Network isolation

Default-allow cluster networking is the opposite of tenant isolation. A practical design:

• Default deny — start with a deny-all NetworkPolicy per namespace, then allow explicit flows
• Tenant paths — permit only required Pod selectors and ports between namespaces
• DNS egress — allow UDP/TCP 53 to kube-dns or CoreDNS without opening the whole cluster
• Shared platform Services — document which cluster Services (ingress controller, metrics, logging) every tenant may reach
• External egress — control outbound Internet with egress NetworkPolicy, firewall, or cloud security groups
• Separate ingress — dedicated hostnames, certificates, or ingress controllers per tenant when branding or compliance requires it

Implementation patterns live in NetworkPolicy examples. Service mesh mTLS is out of scope here—treat mesh policy as an additional layer on top of namespace design.

Workload isolation

Even with RBAC and network policy, a compromised container can abuse the kernel unless the Pod spec is constrained. Layer these controls (each has a dedicated guide):

• Pod Security Admission — Baseline or Restricted profiles per namespace
• seccomp profiles — syscall filtering
• AppArmor profiles — path-oriented LSM rules where enabled
• Linux capabilities — drop CAP_SYS_ADMIN and friends
• SecurityContext — non-root UIDs, read-only root filesystem
• Pod user namespaces — Kubernetes 1.36 can map root inside a container to an unprivileged UID on the node by setting spec.hostUsers: false. This reduces the impact of container escape and cross-Pod UID overlap. It remains a shared-kernel mechanism and requires compatible kernels, filesystems, and container runtimes, so it does not replace gVisor, Kata, or separate nodes
• Immutable container images — pin digests and reject mutable tags in admission (see image signing guides in the CKS track)
• RuntimeClass — schedule untrusted Pods to sandboxed runtimes when node pools support them

Workload policy reduces blast radius inside a tenant namespace; it does not replace network or API boundaries between tenants.

Node isolation

When policy alone is not enough, pin tenants to dedicated hardware:

• Dedicated node pools — separate instance types or zones per tenant class
• Taints and tolerations — only tenant Pods tolerate tenant=customer-a:NoSchedule
• Node affinity — require labels such as workload=regulated
• RuntimeClass scheduling — gVisor or Kata nodes for sandboxed tenants
• Dedicated storage — encryption keys and storage classes that do not share backends across tenants
• Restricted node administration — SSH and cloud console access limited to platform SREs, not tenant users

Watch for bypass: if tenants can set nodeSelector, affinity, or tolerations without admission review, they can schedule onto another tenant's nodes. Use admission policy or OPA-style validators to block unauthorized scheduling fields.

node affinity and taints and tolerations articles cover scheduling mechanics.

Virtual clusters

Virtual clusters (vClusters and similar patterns) give each tenant a dedicated Kubernetes API namespace hierarchy while sharing worker nodes underneath:

• Per-tenant API abstraction—a controller presents a mostly normal kubectl experience
• Stronger object and namespace separation than a single shared namespace
• Underlying nodes and host administrators may still be shared
• Host-cluster admins retain etcd access, node SSH, and CNI visibility

Virtual clusters suit platform teams running many internal tenants on one operations footprint. They do not remove the need for host trust when tenants are mutually hostile—compare them to dedicated nodes or separate clusters for regulated data.

This article does not recommend a specific product; evaluate any virtual-cluster offering against your trust matrix and audit requirements.

Separate clusters

Split clusters when the blast radius of one API server or etcd store is unacceptable:

• Mutually hostile tenants (competitors, arbitrary customer code)
• Separate compliance regimes (PCI, HIPAA, sovereign data residency)
• Independent administrators with no shared root of trust
• Different upgrade cadences or Kubernetes versions
• Strong cloud-account or VPC separation requirements
• High blast-radius concerns from control-plane compromise

Separate clusters cost more to operate but provide the clearest administrative boundary. Use namespace multi-tenancy inside each cluster for sub-teams, not as a substitute when cluster separation is required.

Trust-level decision matrix

Re-evaluate when a tenant gains exec access, hosts untrusted containers, or stores regulated data—the model that worked for internal dev may fail for production customer code.

Validate isolation

Run negative tests after you design boundaries. On a kubeadm lab with Kubernetes 1.36, create tenant namespaces and prove cross-tenant access fails.

Create tenant namespaces:

export KUBECONFIG=/etc/kubernetes/admin.conf
kubectl create namespace tenant-a
kubectl create namespace tenant-b
kubectl create namespace tenant-restricted
kubectl create namespace tenant-quota

namespace/tenant-a created
namespace/tenant-b created
namespace/tenant-restricted created
namespace/tenant-quota created

Cross-namespace API access

Create a ServiceAccount in tenant-a with a namespaced Role that may read Pods only in that namespace:

kubectl create serviceaccount viewer-a -n tenant-a

kubectl create role pod-reader \
  -n tenant-a \
  --verb=get,list,watch \
  --resource=pods

kubectl create rolebinding viewer-a-pod-reader \
  -n tenant-a \
  --role=pod-reader \
  --serviceaccount=tenant-a:viewer-a

Verify API access:

kubectl auth can-i get pods -n tenant-a --as=system:serviceaccount:tenant-a:viewer-a

yes

kubectl auth can-i get pods -n tenant-b --as=system:serviceaccount:tenant-a:viewer-a

no

The same identity cannot list Pods across the namespace boundary—RBAC is doing its job for API reads.

Secret isolation

Create a Secret in tenant-b and confirm the tenant-a ServiceAccount cannot read it:

kubectl create secret generic tenant-b-secret \
  -n tenant-b \
  --from-literal=token=demo

kubectl get secret tenant-b-secret \
  -n tenant-b \
  --as=system:serviceaccount:tenant-a:viewer-a

Error from server (Forbidden): secrets "tenant-b-secret" is forbidden: User "system:serviceaccount:tenant-a:viewer-a" cannot get resource "secrets" in API group "" in the namespace "tenant-b"

Network isolation

With Calico (or any CNI that enforces NetworkPolicy), deploy test Pods and confirm cross-namespace connectivity before you apply a deny-all policy:

kubectl run web-a \
  -n tenant-a \
  --image=nginx:1.27-alpine \
  --restart=Never

kubectl run client-b \
  -n tenant-b \
  --image=busybox:1.36 \
  --restart=Never \
  -- sleep 3600

kubectl wait -n tenant-a --for=condition=Ready pod/web-a --timeout=90s
kubectl wait -n tenant-b --for=condition=Ready pod/client-b --timeout=90s

WEB_A_IP=$(kubectl get pod web-a -n tenant-a -o jsonpath='{.status.podIP}')

kubectl exec -n tenant-b client-b -- \
  wget -T 3 -qO- "http://${WEB_A_IP}"

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...

Apply a default-deny policy in tenant-a:

kubectl apply -f - <<'EOF'
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
  namespace: tenant-a
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress
EOF

Repeat the wget probe from client-b:

kubectl exec -n tenant-b client-b -- \
  wget -T 3 -qO- "http://${WEB_A_IP}"

wget: download timed out
command terminated with exit code 1

NetworkPolicy filtering is defined for TCP and UDP, and for SCTP when the CNI plugin supports SCTP NetworkPolicy enforcement. Behaviour for protocols such as ICMP is implementation-dependent. The wget timeout confirms the HTTP path is blocked after the deny-all policy is in place.

Privileged Pod creation

Label a namespace with Pod Security enforce=restricted for Kubernetes 1.36:

kubectl label namespace tenant-restricted \
  pod-security.kubernetes.io/enforce=restricted \
  pod-security.kubernetes.io/enforce-version=v1.36 \
  --overwrite

namespace/tenant-restricted labeled

Attempt a privileged Pod in that namespace:

kubectl apply -f - <<'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: privileged-test
  namespace: tenant-restricted
spec:
  containers:
  - name: privileged-test
    image: busybox:1.36
    command: ["sleep", "3600"]
    securityContext:
      privileged: true
EOF

Error from server (Forbidden): error when creating "STDIN": pods "privileged-test" is forbidden: violates PodSecurity "restricted:v1.36": privileged (container "privileged-test" must not set securityContext.privileged=true), allowPrivilegeEscalation != false (container "privileged-test" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "privileged-test" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "privileged-test" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "privileged-test" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

Pod Security enforcement rejects violating Pods during creation, before they reach a node.

Quota exhaustion

Use a separate namespace so earlier test Pods do not count against the quota:

kubectl create quota pod-quota \
  -n tenant-quota \
  --hard=pods=2

kubectl run quota-test-1 \
  -n tenant-quota \
  --image=busybox:1.36 \
  --restart=Never \
  -- sleep 3600

kubectl run quota-test-2 \
  -n tenant-quota \
  --image=busybox:1.36 \
  --restart=Never \
  -- sleep 3600

kubectl run quota-test-3 \
  -n tenant-quota \
  --image=busybox:1.36 \
  --restart=Never \
  -- sleep 3600

Error from server (Forbidden): pods "quota-test-3" is forbidden: exceeded quota: pod-quota, requested: pods=1, used: pods=2, limited: pods=2

A pods quota counts all non-terminal Pods already present in that namespace.

Extend validation to kubectl auth can-i for pods/exec, node label bypass attempts, and cloud metadata access from privileged Pods when your threat model includes those paths.

Delete lab namespaces when finished:

kubectl delete namespace tenant-a tenant-b tenant-restricted tenant-quota --ignore-not-found

Troubleshooting

What's Next

• Run Sandboxed Containers with RuntimeClass and gVisor
• Encrypt Kubernetes Pod Traffic with Cilium WireGuard
• Enable Istio Mutual TLS with PeerAuthentication

References

• Pod Security Standards — profile definitions
• Pod Security Admission — namespace labels
• Multi-tenancy — upstream overview
• Resource quotas — fairness limits
• Network policies — pod networking rules
• User namespaces — Pod UID mapping on the node
• API Priority and Fairness — API-server request fairness

Summary

You defined a tenant by trust—not just by namespace name—and compared isolation models from soft policy through dedicated nodes, sandboxed runtimes, virtual clusters, and separate control planes. Namespace multi-tenancy stacks RBAC, quotas, NetworkPolicy, and Pod Security Admission, but it shares one API server, one set of cluster admins, and one kernel on each node.

The decision matrix is the practical takeaway: cooperative internal teams start with namespaces and strong guardrails; customer code pushes you toward sandboxes or dedicated pools; hostile or regulated tenants belong on separate clusters. Validation is not optional—kubectl auth can-i, Secret access checks, default-deny network probes, PSA rejection of privileged Pods, and quota exhaustion tests confirm the boundaries you designed.

For CKS and platform engineering, treat this page as architecture glue: link outward for YAML and procedures, revisit trust assumptions when a tenant runs untrusted images or receives exec rights, and never present namespaces alone as hypervisor-grade isolation.