In the previous tutorial we learned about Authentication and Authorization in Kubernetes. With all of the authentication mechanisms we have learned, we need to craft a kubeconfig file that records the details of how we authenticate. kubectl uses this configuration file to determine where and how to issue requests to the API server. This file is typically located in your home directory under ~/.kube/config but may also be specified explicitly on the command line with the --kubeconfig parameter or by way of the KUBECONFIG environment variable.
You can check the tutorial where we installed our multi-node cluster using kubeadm, we had defined KUBECONFIG for root user and the same could be defined for normal user. Before we dive into the details of RBAC in Kubernetes, let us understand about kubeconfig.
Understanding kubeconfig
For someone who may not be familiar with a kubeconfig file, it is important to understand its three top-level structures: users, clusters, and contexts.
- With users we name a user and provide the mechanism by which he or she will authenticate to a cluster.
- The clusters attribute provides all of the data necessary to connect to a cluster. This, minimally, includes the IP or fully qualified domain name for the API server but may also include items like the CA bundle for a self-signed certificate.
- And contexts is where we associate users with clusters as a single named entity. The context serves as the means by which kubectl connects and authenticates to an API server.
Kubernetes Authentication
We know already that Kubernetes supports a number of different authentication providers, including:
- HTTP Basic Authentication (largely deprecated)
- x509 client certificates
- Static token files on the host
- Cloud authentication providers like Azure Active Directory and AWS Identity and Access Management (IAM)
- Authentication webhooks
In this tutorial we intend to use x508 client certificates to authenticate our user. Kubernetes has no User Objects. User account consists of an authorized certificate that is completed with some authorization as defined in RBAC.
Following are the brief steps required to create a user account:
- Create a private/public key pair
- Create a Certificate Signing Request
- Sign the Certificate
- Create kubernetes configuration file that uses these keys to access the cluster
- Create an RBAC role
- Create an RBAC role binding
Let's start working on these steps.
Step 1: Create User
First we will create a normal user "user1" which will be part of wheel group. I have already written another detailed tutorial on Linux users and groups
[root@controller ~]# useradd -G wheel user1
Verify the newly created user:
[root@controller ~]# id user1 uid=1004(user1) gid=1004(user1) groups=1004(user1),10(wheel)
Assign a password to this user:
[root@controller ~]# passwd user1
Step 2: Create certificates
You can follow my list of OpenSSL tutorials to understand the basics of generating certificates as here I will be very brief. First we need a private key. I am not using a very strict encryption and it is of just 2048 but size.
[root@controller ~]# openssl genrsa -out user1.key 2048
Generating RSA private key, 2048 bit long modulus (2 primes)
........................................................+++++
.........................+++++
e is 65537 (0x010001)
Next we create a Certificate Signing Request and it is important that you assign a Subject with the name of the user and the namespace which you plan to use for the user.
[root@controller ~]# openssl req -new -key user1.key -out user1.csr -subj "/CN=user1/O=dev"
Lastly create the certificate
[root@controller ~]# openssl x509 -req -in user1.csr -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -out user1.crt -days 365
Signature ok
subject=C = IN, ST = KARNATAKA, L = BENGALURU, O = GOLINUXCLOUD, OU = TEST, CN = controller, emailAddress = admin@golinuxcloud.com
Getting CA Private Key
Step 3: Create namespace (optional)
If you plan to use any existing namespace then you can ignore this step but I want to use a separate namespace to implement the RBAC policy so I will create one.
[root@controller ~]# kubectl create namespace dev
namespace/dev created
Step 4: Update Kubernetes Config file with User Credentials
Since we are using x509 certificates for authentcating the user, the same needs to be added in the kubernetes config file. Following is the default kubeconfig file:
[root@controller ~]# kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://192.168.43.48:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
Use the subcommand set-credentials as kubectl config set-credentials <CREDENTIAL_NAME> to add a credential into kubeconfig
[root@controller ~]# kubectl config set-credentials user1 --client-certificate=user1.crt --client-key=user1.key User "user1" set.
Next verify the kubernetes config file, it should also contain the credentials of the newly added user:
[root@controller ~]# kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://192.168.43.48:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
- name: user1
user:
client-certificate: /root/user1.crt
client-key: /root/user1.key
Step 5: Create security context for new user
A context defines which user to use with which cluster, but can also define the namespace that kubectl should use, when you don’t specify the namespace explicitly with the --namespace or -n option.
The following command is used to create a new context user1-context that ties together the cluster and the user you created:
[root@controller ~]# kubectl config set-context user1-context --cluster=kubernetes --namespace=dev --user=user1
Context "user1-context" created.
kubectl config get-clusters to get the cluster nameTo list the available contexts:
[root@controller ~]# kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* kubernetes-admin@kubernetes kubernetes kubernetes-admin
user1-context kubernetes user1 dev
Next verify if you permission to list pods under this context:
[root@controller ~]# kubectl --context=user1-context get pods
Error from server (Forbidden): pods is forbidden: User "user1" cannot list resource "pods" in API group "" in the namespace "dev"
As expected user1 is not allowed to access user1-context at the moment as we have not yet assigned a role and rolebinding to this user.
- A role is a set of abstract capabilities. For example, the
appdevrole might represent the ability to create Pods and services. - A role binding is an assignment of a role to one or more identities. Thus, binding the
appdevrole to the user identityaliceindicates thatAlicehas the ability to create Pods and services.
Example-1: Configure RBAC to define new role with "modify" permission
Create new role
We will create a new role "dev" based on the namespace where we want to apply this role. You can also choose any other name for the role. To get the KIND and apiVersion for Role we can explore api-resources:
[root@controller ~]# kubectl api-resources | grep -iE 'role|KIND'
NAME SHORTNAMES APIGROUP NAMESPACED KIND
clusterrolebindings rbac.authorization.k8s.io false ClusterRoleBinding
clusterroles rbac.authorization.k8s.io false ClusterRole
rolebindings rbac.authorization.k8s.io true RoleBinding
roles rbac.authorization.k8s.io true Role
Here we are interested in KIND Role, next let's get the apiVersion for this resource:
[root@controller ~]# kubectl explain Role | head -n 2
KIND: Role
VERSION: rbac.authorization.k8s.io/v1
So now we have both KIND and apiVersion of Role. Following is my YAML file to create dev role that gives an identity the ability to create and modify Pods, Deployments and ReplicaSets and can be applied to any of the three types of API Groups:
[root@controller ~]# cat dev-role.yml kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: namespace: dev name: dev rules: - apiGroups: ["", "extensions", "apps"] resources: ["deployments", "replicasets", "pods"] verbs: ["list", "get", "watch", "create", "update", "patch", "delete"]
Understanding Verbs for Kubernetes roles
Roles are defined in terms of both a resource (e.g., “Pods”) and a verb that describes an action that can be performed on that resource. The verbs correspond roughly to HTTP methods. The commonly used verbs in Kubernetes RBAC are listed in the following table:
| Verb | HTTP Method | Description |
|---|---|---|
| create | POST | Create a new resource. |
| delete | DELETE | Delete an existing resource. |
| get | GET | Get a resource. |
| list | LIST | List a collection of resources. |
| patch | PATCH | Modify an existing resource via a partial change. |
| update | PUT | Modify an existing resource via a complete object. |
| watch | GET | Watch for streaming updates to a resource. |
| proxy | GET | Connect to resource via a streaming WebSocket proxy. |
We will create this role:
[root@controller ~]# kubectl create -f dev-role.yml
role.rbac.authorization.k8s.io/dev created
Bind a user to the newly created role
Next we create RoleBinding to bind this Role to the user1. Following is our YAML file:
[root@controller ~]# cat user1-rolebind.yml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: dev-role-binding
namespace: dev
subjects:
- kind: User
name: user1
apiGroup: ""
roleRef:
kind: Role
name: dev
apiGroup: ""
Here we have deined the user to which this rolebinding must be applied to as user1.
Let's go ahead and create this RoleBinding:
[root@controller ~]# kubectl create -f user1-rolebind.yml
rolebinding.rbac.authorization.k8s.io/dev-role-binding created
Verify the RBAC Policy
Now that we have created a role with permission to create and modify different resources for user1, let's verify the same:
[root@controller ~]# kubectl --context=user1-context get pods
No resources found in dev namespace.
If you recall this command was throwing a "FORBIDDEN" error previously before applying the role and role binding but now we get a proper message so it means the RBAC is working as expected.
We can describe the role to get more information on the Verbs which are allowed for different resources:
[root@controller ~]# kubectl -n dev describe role dev Name: dev Labels: <none> Annotations: <none> PolicyRule: Resources Non-Resource URLs Resource Names Verbs --------- ----------------- -------------- ----- deployments [] [] [list get watch create update patch delete] pods [] [] [list get watch create update patch delete] replicasets [] [] [list get watch create update patch delete] deployments.apps [] [] [list get watch create update patch delete] pods.apps [] [] [list get watch create update patch delete] replicasets.apps [] [] [list get watch create update patch delete] deployments.extensions [] [] [list get watch create update patch delete] pods.extensions [] [] [list get watch create update patch delete] replicasets.extensions [] [] [list get watch create update patch delete]
Let's try to create a new deployment resource under user1-context to verify if user1 has enough permission:
[root@controller ~]# kubectl create deployment devnginx --image=nginx --context=user1-context
deployment.apps/devnginx created
As expected the deployment was successfully created but we can't find it under default namespace:
[root@controller ~]# kubectl get pods NAME READY STATUS RESTARTS AGE nginx-cm 1/1 Running 2 6d11h nginx-dev 1/1 Running 1 6d6h secret-tls-pod 1/1 Running 2 6d6h
That's because user1-context is applied for dev namespace so by default any resource would be created under dev namespace only:
[root@controller ~]# kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
devnginx-7bc9b989dc-v6zs9 0/1 ContainerCreating 0 8s
Testing Authorization with can-i
The first useful tool is the auth can-i command for kubectl. This tool is very useful for testing if a particular user can do a particular action. You can use can-i to validate configuration settings as you configure your cluster, or you can ask users to use the tool to validate their access when filing errors or bug reports.
In its simplest usage, the can-i command takes a verb and a resource.
[root@controller ~]# kubectl auth can-i create pods --context=user1-context yes [root@controller ~]# kubectl auth can-i create service --context=user1-context no
Since we have given permission to create Pods, Deployments and ReplicaSets only, the auth can-i gives a "no" for creating a service
Example-2: Configure RBAC to define new role with "view-only" permission
In this example we will create another role which will have permission to only view the available resources but not to create or modify. Let me create a new role as "view-only" with following content:
[root@controller ~]# cat view-only-role.yml kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: namespace: view-only name: view-only rules: - apiGroups: ["", "extensions", "apps"] resources: ["deployments", "replicasets", "pods"] verbs: ["list", "get", "watch"]
As you see I have given Verbs as list, get and watch which would only give viewing permission to the allowed resources part of the API group as mentioned above. I will apply these changed into a new namespace "view-only"
Now before we create this Role, we must make sure the provided namespace i.e. "view-only" exists, so let me go ahead and create one for our demonstration:
[root@controller ~]# kubectl create namespace view-only
namespace/view-only created
Next I will create this role:
[root@controller ~]# kubectl create -f view-only-role.yml
role.rbac.authorization.k8s.io/view-only created
Now we need to create a RoleBinding to bind this role to our user1
[root@controller ~]# cat view-only-rolebinding.yml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: view-only-role-binding
namespace: view-only
subjects:
- kind: User
name: user1
apiGroup: ""
roleRef:
kind: Role
name: view-only
apiGroup: ""
Let's create this RoleBinding:
[root@controller ~]# kubectl create -f view-only-rolebinding.yml
rolebinding.rbac.authorization.k8s.io/view-only-role-binding created
We are all set to create a new security context "viewonly-context" which can be used to give view only permission to user1
[root@controller ~]# kubectl config set-context viewonly-context --cluster=kubernetes --namespace=view-only --user=user1
Context "viewonly-context" created.
List the available contexts in the kubeconfig file:
[root@controller ~]# kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* kubernetes-admin@kubernetes kubernetes kubernetes-admin
user1-context kubernetes user1 dev
viewonly-context kubernetes user1 view-only
Next we can verify our viewonly-context by trying to list the available pods.
[root@controller ~]# kubectl --context=viewonly-context get pods
No resources found in view-only namespace.
Since the context has permission to list the resources, the command has executed successfully. We can get more details on the available verbs for this role using kubectl describe:
[root@controller ~]# kubectl -n view-only describe role view-only Name: view-only Labels: <none> Annotations: <none> PolicyRule: Resources Non-Resource URLs Resource Names Verbs --------- ----------------- -------------- ----- deployments [] [] [list get watch] pods [] [] [list get watch] replicasets [] [] [list get watch] deployments.apps [] [] [list get watch] pods.apps [] [] [list get watch] replicasets.apps [] [] [list get watch] deployments.extensions [] [] [list get watch] pods.extensions [] [] [list get watch] replicasets.extensions [] [] [list get watch]
Next let's try to create a new Deployment using this context:
[root@controller ~]# kubectl create deployment testnginx --image=nginx --context=viewonly-context
error: failed to create deployment: deployments.apps is forbidden: User "user1" cannot create resource "deployments" in API group "apps" in the namespace "view-only"
So this has failed with FORBIDDEN error as we had not given permission to create new resources.
We can also use auth can-i command to verify the user's privilege under viewonly-context:
[root@controller ~]# kubectl auth can-i create pods --context=viewonly-context no [root@controller ~]# kubectl auth can-i get pods --context=viewonly-context yes [root@controller ~]# kubectl auth can-i get service --context=viewonly-context no
So this context has privilege to list pods but has no privilege to create a pod or list service resource objects.
Deleting contexts
To clean up the list of contexts, you can either delete the entries from the kubeconfig file manually or use the following two commands:
[root@controller ~]# kubectl config delete-context user1-context deleted context user1-context from /etc/kubernetes/admin.conf [root@controller ~]# kubectl config delete-context viewonly-context deleted context viewonly-context from /etc/kubernetes/admin.conf
Deleting Role
Use the following command to delete a role along with the name of the namespace:
[root@controller ~]# kubectl delete role dev -n dev
role.rbac.authorization.k8s.io "dev" deleted
Deleting RoleBinding
Use the following command to delete a role binding along with the namespace under which the binding was created:
[root@controller ~]# kubectl delete rolebinding dev-role-binding -n dev
rolebinding.rbac.authorization.k8s.io "dev-role-binding" deleted
Conclusion
When you begin with a small cluster and a small team, it is sufficient to have every member of the team have equivalent access to the cluster. But as teams grow and products become more mission critical, limiting access to parts of the cluster is crucial. In a well-designed cluster, access is limited to the minimal set of people and capabilities needed to efficiently manage the applications in the cluster. Understanding how Kubernetes implements RBAC and how those capabilities can be used to control access to your cluster is important for both developers and cluster administrators.
