Kubernetes subPath Examples | MountPath vs subPath Explained

Introduction to Kubernetes SubPath

We recently explained how it is possible to update secrets and configmaps in Kubernetes without restarting any pods. One of the pitfalls of this approach was that when we specify a mount path in our deployment, and it already exists inside the container, all the existing files and directories inside that path become inaccessible. That doesn't mean that existing data is deleted, it simply ceases to exist for the time being.

This is the general behavior in Linux. When a filesystem is mounted into a non-empty directory, the directory then only contains the files from the newly mounted filesystem. The original files in that directory are inaccessible for as long as the filesystem remains mounted. In cases, when the directory contains crucial data, mounting a configmap or a secret in non-empty directory could break the functionality of the container.

To overcome this limitation, Kubernetes offers an alternative in the form of subPath. The official Kubernetes documentation defines this as follows:

Sometimes, it is useful to share one volume for multiple uses in a single pod. The volumeMounts.subPath property specifies a sub-path inside the referenced volume instead of its root.

The Kubernetes subPath property prevents overwriting of existing data by mounting a single file from the volume instead of mounting the whole volume. We'll explain this by mounting a configMap in an nginx deployment.

 

Known Limitation

At the time of writing this article, there is a known limitation when using Kubernetes subPath. ConfigMaps and secrets mounted using subPath cannot be updated dynamically. To update the configmap and secrets, new pods will need to be created.

 

Prerequisites

• Working K8s cluster
• Working knowledge of K8s and yaml language

 

Create a sample deployment (Optional)

We're first going to create a sample nginx deployment using a single replicaset as follows.

apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx

Once the deployment has been created, access the container and list the contents under the /etc/nginx directory. As can be seen, the /etc/nginx directory contains multiple files and a directory. Make a note of this and proceed to delete this deployment.

kubectl get pods

kubectl exec -it nginx-8f458dc5b-zkbdt bash

cd /etc/nginx/; ls -l

exit

 

Mount ConfigMap using Kubernetes mountPath

Let's create a simple nginx.conf file with the following content and mount it inside the container using a configmap using mountPath. There are multiple ways to access a configmap inside the container.

events {
	worker_connections 1024;
}

http {
	server {
		listen 80 default_server;
		server_name example.com;
		root /var/www;
	}
}

Create the configmap from the file.

kubectl create configmap nginx-cm --from-file=nginx.conf

 

Once the configmap has been created, create a new nginx deployment as follows. We're going to mount the sample nginx.conf file in /etc/nginx directory inside the container using a configmap.

apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx
        volumeMounts:
        - name: nginx-vol
          mountPath: /etc/nginx
      volumes:
      - name: nginx-vol
        configMap:
          name: nginx-cm

Once the deployment has been created, access the container and list the contents of /etc/nginx directory.

kubectl get pods

kubectl exec -it nginx-bd6ccf697-d99vq bash

cd /etc/nginx/; ls

cat nginx.conf

 

As you can see, only a single file i.e. nginx.conf, which contains the content of our configmap, is present in the /etc/nginx directory. All the files which we had previously seen when creating a sample deployment in the first example are not being shown.

This shows that whenever we mount in a non-empty directory, the existing data becomes inaccessible. We're now going to fix this by mounting our configMap using the subPath property. Delete the deployment and associated resources.

kubectl delete all -l app=nginx

 

Mount configMap using Kubernetes subPath

There is no need to change the configMap created in the last example. We'll just need to update our deployment file and make the following two changes:

• Change the mountPath from /etc/nginx to /etc/nginx/nginx.conf
• Add the Kubernetes subPath property under mountPath and set it's value to nginx.conf.

The value for Kubernetes subPath must match the path specified in mountPath. The updated deployment should be as follows:

apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx
        volumeMounts:
        - name: nginx-vol
          mountPath: /etc/nginx/nginx.conf
          subPath: nginx.conf
      volumes:
      - name: nginx-vol
        configMap:
          name: nginx-cm

Once the deployment has been created, access the container and list the contents under /etc/nginx.

kubectl get pods

kubectl exec -it nginx-65996f4c4d-lmcmj bash

cd /etc/nginx/; ls -l

As evident from the above output, all the files inside /etc/nginx including our configMap file, are being shown. This shows that KubernetessubPath property should be used when mounting a configMap in a non-empty directory inside the container. Otherwise, existing data will be overwritten.

 

mountPath vs subPath

Following are some of the differences between Kubernetes mountPath and subPath when using for ConfigMap and Secrets:

• The mountPath will overwrite existing directory with the configmap/secret. This is similar to mounting multiple directories on the same path. For example in Linux, if you have /mnt directory with some content and then you mount /dev/sda1 to /mnt then the exiting content of /mnt will not be visible. Similarly if you again mount /dev/sda2 to /mnt then the content of both /mnt and /dev/sda1 will be overwritten by content of /dev/sda2. But the content is not actually deleted, if you unmount /dev/sda2 then the content of /dev/sda1 will be visible again. The same way mountPath works in Kubernetes.
• The subPath on the other hand will just add your content on top of the existing directory so the existing content are not lost and both new mount and existing content can be accessed.
• The downside of using mountPath is that dynamic update of configmap or secret is not possible while the same is supported with subPath. So if you mount a configmap using subPath, then you can do some change to your secret on controller and the same will get refreshed on the container as well.

 

Summary

Kubernetes Subpaths prove useful when mounting in a non-empty directory inside the container. However, there's a limitation associated with this approach. A container using a configMap or a secret as a subpath volume mount cannot be updated dynamically. The same approach can be used for Kubernetes secrets as well. Although if you have an isolated directory then you can prefer to use mountPath in which case you will have the flexibility to dynamically update the configmap and secret content.