| Tested on | Rocky Linux 10.2 (Red Quartz) — kubeadm control plane node |
|---|---|
| Package | kubectl 1.36.3kubelet 1.36.3kubeadm 1.36.3 |
| Applies to | Ubuntu, Debian, Kali Linux, Linux Mint, Pop!_OS, Raspberry Pi OS, elementary OS, Zorin OS, Parrot OS, MX Linux, RHEL, Rocky Linux, AlmaLinux, Oracle Linux, CentOS Stream, Fedora Kubernetes cluster |
| Cert prep | CKS |
| Lab environment | Multi-node kubeadm cluster with containerd and SSH access to the control plane — install Kubernetes with kubeadm |
| Privilege | root on the control-plane node for manifest edits; cluster-admin kubectl for validation |
| Scope | Discover kube-apiserver settings from the static Pod manifest, assess network exposure and TLS, review anonymous access, authentication, authorization, admission, ServiceAccount issuer flags, request limits, and deprecated flags, then apply one change safely with post-change validation. Does not cover audit-policy design, full RBAC or OIDC setup, etcd hardening, general host-firewall tutorials, or managed-control-plane configuration. |
The API server is the front door to cluster state. On a kubeadm cluster it is defined by a static Pod manifest on the control-plane node, not a Deployment you edit with kubectl. This walkthrough records a baseline from /etc/kubernetes/manifests/kube-apiserver.yaml, reviews exposure and auth settings, applies one hardening flag (--profiling=false), and validates /readyz, kubectl, and node connectivity after the kubelet recreates the container.
Discover the effective configuration
On the control-plane node, list the static Pod manifest directory:
ls -l /etc/kubernetes/manifests/kube-apiserver.yaml-rw-------. 1 root root 3877 Jul 27 22:18 /etc/kubernetes/manifests/kube-apiserver.yamlThe kubelet mirrors this file into a Pod in kube-system. Extract the security-relevant flags into a baseline table before you change anything:
grep -E 'advertise-address|secure-port|authorization-mode|enable-admission-plugins|anonymous-auth|service-account-issuer|tls-cert-file|client-ca-file|profiling|max-requests|request-timeout' /etc/kubernetes/manifests/kube-apiserver.yamlkubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint: 192.168.56.108:6443
- --advertise-address=192.168.56.108
- --authorization-mode=Node,RBAC
- --client-ca-file=/etc/kubernetes/pki/ca.crt
- --enable-admission-plugins=NodeRestriction
- --secure-port=6443
- --service-account-issuer=https://kubernetes.default.svc.cluster.local
- --service-account-key-file=/etc/kubernetes/pki/sa.pub
- --service-account-signing-key-file=/etc/kubernetes/pki/sa.key
- --tls-cert-file=/etc/kubernetes/pki/apiserver.crtOn this lab cluster the baseline shows TLS on port 6443, Node,RBAC authorization, NodeRestriction admission, and a cluster-local ServiceAccount issuer. There is no --insecure-port flag and no explicit --anonymous-auth line, which means anonymous access uses the default (true).
Also note mounted policy and certificate paths in the full manifest:
/etc/kubernetes/pki— apiserver, client CA, ServiceAccount signing material- Request-header settings for aggregation (
front-proxy-ca.crt,X-Remote-Userheaders) - etcd client certificates under the same PKI tree
Copy the manifest outside the watched directory before any edit:
cp -a /etc/kubernetes/manifests/kube-apiserver.yaml /root/kube-apiserver.yaml.baseline-$(date +%Y%m%d)The backup lives outside /etc/kubernetes/manifests/ so the kubelet does not treat it as a second static Pod.
Restrict network exposure
The API server listens on the control-plane network interface because the static Pod uses hostNetwork: true. Confirm the secure listener:
ss -lntp | grep 6443LISTEN 0 4096 *:6443 *:* users:(("kube-apiserver",pid=1110268,fd=4))*:6443 means all interfaces on the node. That is normal for kubeadm; exposure control moves to host firewall, private networking, and who may route to the control-plane IP.
Review firewalld (or your host firewall) on the control plane:
firewall-cmd --list-allpublic (default, active)
services: cockpit dhcpv6-client http ntp ssh
ports: 1313/tcp 6443/tcp 2379-2380/tcp 10250/tcp 10259/tcp 10257/tcp 10256/tcp 30000-32767/tcp 4789/udp 5473/tcp 179/tcpPort 6443/tcp is open on this lab node. In production, narrow source addresses with rich rules or place the API behind a private load balancer. Do not block paths kubelets and controllers need:
- Worker nodes must reach
https://<control-plane>:6443 - Control-plane components talk to etcd on
127.0.0.1:2379(not through the public API) - Keep disaster-recovery break-glass access documented before you tighten rules
Record whether administrators reach the API by node IP, virtual IP, or load-balancer DNS. Certificate SANs must cover the name clients use — see the next section.
Verify TLS-only serving
Confirm only the secure port is in use. A default kubeadm manifest sets --secure-port=6443 and omits --insecure-port, which removes the obsolete insecure port entirely.
Inspect the serving certificate SANs:
openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -subject -dates -ext subjectAltNamesubject=CN=kube-apiserver
notBefore=Jul 27 16:43:02 2026 GMT
notAfter=Jul 27 16:48:02 2027 GMT
X509v3 Subject Alternative Name:
DNS:k8s-cp, DNS:kubernetes, DNS:kubernetes.default, DNS:kubernetes.default.svc, DNS:kubernetes.default.svc.cluster.local, IP Address:10.96.0.1, IP Address:192.168.56.108Clients connecting to https://192.168.56.108:6443 or the Kubernetes Service DNS names listed above validate cleanly when they trust the cluster CA. If you front the API with a load balancer, add its DNS name or IP to the apiserver certificate during renewal — see PKI and certificate renewal for the rotation workflow.
Verify TLS with a quick HTTPS probe:
curl -sk -o /dev/null -w "%{http_code}\n" https://127.0.0.1:6443/readyz200A 200 over HTTPS confirms the secure listener answers. There is no parallel HTTP port to disable on modern kubeadm builds.
Review anonymous authentication
Anonymous access is enabled by default when the manifest omits --anonymous-auth. Test what unauthenticated callers can reach:
curl -sk -o /dev/null -w "livez:%{http_code} " https://127.0.0.1:6443/livez
curl -sk -o /dev/null -w "readyz:%{http_code} " https://127.0.0.1:6443/readyz
curl -sk -o /dev/null -w "version:%{http_code} " https://127.0.0.1:6443/version
curl -sk -o /dev/null -w "api:%{http_code}\n" https://127.0.0.1:6443/apilivez:200 readyz:200 version:200 api:403Health and version endpoints respond without credentials. The core API returns 403 Forbidden for anonymous callers on this cluster, which is the expected split: probes can succeed while unauthenticated API listing stays blocked.
Disabling anonymous auth is tempting but risky without replanning probes. During lab testing, adding --anonymous-auth=false changed health checks to 401 while kubectl still worked from admin credentials:
livez:401 readyz:401 api:401The static Pod manifest uses HTTPS probes against /livez and /readyz without client certificates. Turning off anonymous auth can therefore break kubelet health checks or external load-balancer monitors. Once --anonymous-auth=false is set, requests do not become system:anonymous, so RBAC cannot restore unauthenticated probe access.
For Kubernetes 1.36, the preferred granular option is an AuthenticationConfiguration that permits anonymous authentication only for health paths while rejecting anonymous access to /version and other endpoints:
apiVersion: apiserver.config.k8s.io/v1
kind: AuthenticationConfiguration
anonymous:
enabled: true
conditions:
- path: /livez
- path: /readyz
- path: /healthzWhen this configuration controls anonymous authentication, --anonymous-auth must not also be set. Mount the file with --authentication-config and test every probe and monitor path before you rely on it in production.
Review authentication methods
Inventory what the manifest enables before you remove a method:
| Method | Evidence in manifest | Typical client |
|---|---|---|
| Client certificates | --client-ca-file=/etc/kubernetes/pki/ca.crt |
kubectl with admin kubeconfig, controllers |
| ServiceAccount tokens | --service-account-issuer=... plus signing key files |
In-cluster Pods, projected volumes |
| Bootstrap tokens | --enable-bootstrap-token-auth=true |
kubeadm join during node bootstrap |
| Aggregation / proxy headers | --requestheader-client-ca-file=... and X-Remote-User headers |
kube-aggregator, extension API servers |
OIDC and webhook token authentication are not configured on this lab cluster. Add them only with a tested identity provider — that installation path is out of scope here.
Confirm authenticated API access with the kubeadm administrator identity in admin.conf:
kubectl --kubeconfig=/etc/kubernetes/admin.conf get --raw /api{
"kind": "APIVersions",
"versions": [
"v1"
],
"serverAddressByClientCIDRs": [
{
"clientCIDR": "0.0.0.0/0",
"serverAddress": "192.168.56.108:6443"
}
]
}admin.conf contains the intended kubeadm administrator client certificate and proves authenticated API access remains functional. Do not use /etc/kubernetes/pki/apiserver-kubelet-client.crt for this test — that certificate is generated for the API server to authenticate to kubelets, not as a normal administrator identity for calling the API server. Remove unused authentication methods only after you prove no bootstrap job, user, or controller still depends on them.
Review authorization modes
The manifest should use explicit, restrictive modes:
grep authorization-mode /etc/kubernetes/manifests/kube-apiserver.yaml- --authorization-mode=Node,RBACNode lets kubelets perform node-scoped operations. RBAC governs everything else through Roles and ClusterRoles. Avoid AlwaysAllow in production.
Mode order matters when multiple authorizers are listed — the API server checks them in sequence. For Kubernetes 1.36, Node,RBAC is the kubeadm default and matches CIS guidance. Changing order or removing Node can break kubelet sync without improving human-user security.
Sanity-check an admin identity against RBAC from your workstation or the control plane:
export KUBECONFIG=/etc/kubernetes/admin.conf
kubectl auth can-i '*' '*' --all-namespacesyesUse kubectl auth can-i with less powerful kubeconfigs when you validate tenant RBAC — this article does not reproduce full Role design. See Kubernetes RBAC for that.
Review admission plugins
Admission runs after authentication and authorization on write requests. List enabled plugins:
grep enable-admission-plugins /etc/kubernetes/manifests/kube-apiserver.yaml- --enable-admission-plugins=NodeRestrictionNodeRestriction limits what kubelets can change on their own Node object and on Pods bound to that node — a core hardening plugin for multi-tenant clusters.
The manifest explicitly adds NodeRestriction; it does not replace the default admission-plugin set. Kubernetes 1.36 enables several admission controllers by default, including PodSecurity, ServiceAccount, ResourceQuota, LimitRanger, and the mutating and validating webhook plugins.
Confirm the effective default set from the running binary:
APISERVER_ID=$(crictl ps -q --name kube-apiserver | head -1)
crictl exec "$APISERVER_ID" kube-apiserver --help 2>&1 |
grep -F -A 1 ' --enable-admission-plugins strings'--enable-admission-plugins strings admission plugins that should be enabled in addition to default enabled ones (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PodGroupProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, ClusterTrustBundleAttest, CertificateSubjectRestriction, DefaultIngressClass, PodTopologyLabels, PodGroupWorkloadExists, NodeDeclaredFeatureValidator, JobValidation, PodResizeValidator, MutatingAdmissionPolicy, MutatingAdmissionWebhook, ValidatingAdmissionPolicy, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: ...The help text lists defaults separately from plugins you add with --enable-admission-plugins. Turning on additional webhooks or policy engines is a separate change category — add one plugin family at a time and rerun workload smoke tests. Audit logging configuration is intentionally excluded; it belongs in the logging and monitoring domain.
Validate ServiceAccount issuer settings
Projected ServiceAccount tokens depend on consistent issuer and signing key configuration:
grep service-account /etc/kubernetes/manifests/kube-apiserver.yaml- --service-account-issuer=https://kubernetes.default.svc.cluster.local
- --service-account-key-file=/etc/kubernetes/pki/sa.pub
- --service-account-signing-key-file=/etc/kubernetes/pki/sa.keyThe issuer URL should match what in-cluster validators expect. Changing issuer or keys without updating existing tokens breaks workloads that cache old JWTs. Before you edit these flags, read Kubernetes ServiceAccounts and plan a rolling restart of dependent controllers.
Confirm the API advertises the issuer in its discovery document:
kubectl get --raw /.well-known/openid-configuration | head -c 300{"issuer":"https://kubernetes.default.svc.cluster.local","jwks_uri":"https://kubernetes.default.svc.cluster.local/openid/v1/jwks","response_types_supported":["id_token"],"subject_types_supported":["public"],"id_token_signing_alg_values_supported":["RS256"]}The issuer field matches the manifest flag. For multi-issuer or external OIDC integration, align API flags with your identity design before you cut over production traffic.
Configure request and availability limits
The lab manifest does not override concurrency defaults. Read them from the running binary:
crictl exec $(crictl ps -q --name kube-apiserver | head -1) kube-apiserver --help 2>&1 | grep -E 'max-requests-inflight|max-mutating-requests-inflight|request-timeout'--max-mutating-requests-inflight int ... (default 200)
--max-requests-inflight int ... (default 400)
--request-timeout duration ... (default 1m0s)Treat these as starting points, not universal production values. Raising limits without load testing can mask overload; lowering them without measuring webhook latency can cause timeouts during admission storms.
When API Priority and Fairness is enabled (default on recent releases), total concurrency follows a different path than the legacy in-flight counters. Review server metrics and webhook timeout settings before you tune flags. Document the values you choose in your change record.
Remove deprecated or insecure flags
Audit the manifest against your Kubernetes minor version:
| Check | Lab result |
|---|---|
| Insecure port removed | No --insecure-port flag |
| Secure port only | --secure-port=6443 |
| Permissive authorization | Not present — Node,RBAC only |
| Profiling endpoints | Default is on; hardened below |
| Debug mounts | Only standard CA and PKI paths |
Profiling exposes /debug/pprof/ when --profiling defaults to true. Even with RBAC blocking anonymous API listing (403 in our curl tests), the profiling HTTP handler is a separate attack surface on the admin port.
Before you disable profiling, confirm the endpoint is currently exposed:
export KUBECONFIG=/etc/kubernetes/admin.conf
kubectl get --raw /debug/pprof/ >/dev/null &&
echo "Profiling endpoint is enabled"Profiling endpoint is enabledA successful response returns the pprof index HTML. The flag defaults to enabled when the manifest omits --profiling.
Apply changes safely
Change one control category at a time. This example disables the profiling HTTP endpoints.
Record the current container ID before you edit the manifest:
OLD_APISERVER_ID=$(crictl ps -q --name kube-apiserver | head -1)
echo "Old container: $OLD_APISERVER_ID"Edit the manifest to add the flag after an existing line so diffs stay readable:
python3 - <<'PY'
from pathlib import Path
p = Path("/etc/kubernetes/manifests/kube-apiserver.yaml")
text = p.read_text()
needle = " - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname\n"
insert = needle + " - --profiling=false\n"
if "--profiling=false" in text:
print("already set")
elif needle not in text:
raise SystemExit("expected insertion point not found; manifest was not changed")
else:
p.write_text(text.replace(needle, insert, 1))
print("updated manifest")
PYThe kubelet scans the static-Pod directory periodically, so a successful /readyz request alone does not prove the modified manifest was loaded. Wait for a new container ID, then wait until that replacement answers /readyz:
while true; do
NEW_APISERVER_ID=$(crictl ps -q --name kube-apiserver | head -1)
if [ -n "$NEW_APISERVER_ID" ] &&
[ "$NEW_APISERVER_ID" != "$OLD_APISERVER_ID" ]; then
echo "New container: $NEW_APISERVER_ID"
break
fi
sleep 2
done
export KUBECONFIG=/etc/kubernetes/admin.conf
until kubectl --request-timeout=5s get --raw='/readyz' \
>/dev/null 2>&1; do
sleep 2
done
echo "New API server container is ready"
kubectl get pod -n kube-system kube-apiserver-k8s-cpOld container: 690221ce84cc3a9089b498c7b57411230025f580a9ec3fbba36d8bf980fa2e44
New container: 1878202913ca389865ac6a8d21a14746cd8beb05a207e7d58a64feb3c320fa06
New API server container is ready
NAME READY STATUS RESTARTS AGE
kube-apiserver-k8s-cp 1/1 Running 0 5h6mThe container-ID change proves the kubelet recreated the static Pod from the edited manifest. The /readyz loop proves the replacement API server is ready to serve traffic.
Probe health endpoints without authentication:
curl -sk -o /dev/null -w "livez:%{http_code} " https://127.0.0.1:6443/livez
curl -sk -o /dev/null -w "readyz:%{http_code}\n" https://127.0.0.1:6443/readyzlivez:200 readyz:200Confirm administrators and nodes still reach the API:
kubectl get nodesNAME STATUS ROLES AGE VERSION
k8s-cp Ready control-plane 5h6m v1.36.3
worker01 Ready <none> 5h4m v1.36.3Verify the flag on the running process:
tr '\0' '\n' < /proc/$(pgrep -xo kube-apiserver)/cmdline | grep profiling--profiling=falseChecking process arguments proves the flag loaded. Correlate it with the behavior it changes — the profiling endpoint should no longer exist:
kubectl get --raw /debug/pprof/Error from server (NotFound): the server could not find the requested resourceA NotFound response confirms /debug/pprof/ was removed. Argument inspection alone does not prove the HTTP handler was disabled.
If anything fails, restore the backup:
cp /root/kube-apiserver.yaml.baseline-YYYYMMDD /etc/kubernetes/manifests/kube-apiserver.yamlReplace YYYYMMDD with your backup date, wait for recreation, and rerun the health checks above.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
connection refused on port 6443 after an edit |
Static Pod restarting or manifest YAML syntax error | Fix YAML; check crictl ps -a | grep kube-apiserver and crictl logs on the newest container ID |
livez returns 401 after auth hardening |
--anonymous-auth=false without authenticated probes |
Revert flag, authenticate probes, or use AuthenticationConfiguration with path-scoped anonymous access |
Nodes Ready but mirror Pod 0/1 |
Probe timing during recreation | Wait for startup probes; confirm curl to /readyz and kubelet logs |
| Workers cannot join after API change | Firewall or SAN mismatch on new endpoint | Open 6443 to worker subnets; renew apiserver cert with correct SANs |
| Admission webhook timeouts | New webhook without tuned timeouts | Roll back webhook configuration; adjust timeout and failure policy separately |
What's Next
- Kubernetes PKI, kubeconfig Users, CSR and Certificate Renewal
- Upgrade a Kubernetes Cluster with kubeadm
- Harden Linux Nodes for Kubernetes
References
- Kubernetes API server flags — upstream reference
- Authenticate Kubernetes — authentication overview
- Using Node Authorization — Node authorizer
- Admission controllers — plugin reference
- kubeadm API server documentation — static Pod layout
Summary
You audited the kube-apiserver static Pod on a kubeadm control plane: TLS on port 6443, Node,RBAC authorization, NodeRestriction admission, ServiceAccount issuer settings, and network exposure through host firewall rules. Anonymous callers could reach health and version endpoints but received 403 on /api, which is the pattern many clusters rely on for load-balancer probes.
The applied hardening change was deliberate and narrow — --profiling=false — with a backup taken outside the manifest directory, a container-ID wait followed by /readyz readiness, and validation of /livez, /readyz, kubectl get nodes, and /debug/pprof/ returning NotFound after the kubelet recreated the container. That workflow is the template for every other flag category: record a baseline, edit one concern, wait for readiness, correlate flags with behavior, and prove all clients still work.
The common misstep is disabling anonymous authentication without updating probes. Health checks returned 401 in testing while admin kubectl still functioned, which is a painful partial outage. Pair API server tightening with authentication and admission concepts, RBAC for identities, and periodic CIS benchmark scans so control-plane configuration does not drift unnoticed.

