Fix Kubernetes Pods Stuck in Pending or ContainerCreating

Fix Kubernetes Pods stuck in Pending or ContainerCreating using scheduling conditions, Events, PVCs, mounts, sandbox checks, image errors, and init logs.

Published

Updated

Read time 13 min read

Reviewed byDeepak Prasad

Kubernetes Pending and ContainerCreating Pod troubleshooting with scheduling events volume mounts and init containers
Tested on Rocky Linux 10.2 (Red Quartz) workstation
Package kubectl 1.36.3
Applies to Any host with kubectl configured; any Kubernetes cluster
Cert prep CKA · CKAD · CKS
Lab environment Multi-node kubeadm cluster with containerd — install Kubernetes with kubeadm
Privilege Normal user (no sudo required on the workstation)
Scope Pending versus ContainerCreating meaning, scheduled versus unscheduled decision, FailedScheduling causes, CPU and memory requests, selectors affinity and taints routing, unbound PVC signals, ContainerCreating kubelet events, FailedCreatePodSandBox and CNI routing, FailedMount and volume attachment, missing ConfigMap and Secret references, init container blocking, image-pull routing, verification sequence, and error-to-fix tables. Does not cover CNI installation, scheduler internals, CSI driver administration, StorageClass provisioning depth, node recovery, control-plane repair, full image-pull remediation, or CrashLoopBackOff after containers start.
Related guides kubectl logs, Events and describe
IMPORTANT
This guide fixes Pods that never reach a running application container—unscheduled Pending, ContainerCreating, Init:* statuses, and setup-time FailedMount events. It does not cover Fix Kubernetes CrashLoopBackOff after the image is pulled and the process exits, or full registry troubleshooting covered in Fix ImagePullBackOff and ErrImagePull. When you are unsure which STATUS row applies, start with the Kubernetes Pod troubleshooting workflow.

A Pod stuck before startup is a routing problem. Pending with no node means the scheduler never placed it. ContainerCreating with a node assigned means the kubelet is still preparing sandbox, network, volumes, or images. This walkthrough creates both patterns in pending-lab and reads conditions and events to pick the right branch.


Quick reference

Branch on node assignment first. Replace POD and NS with your Pod name and namespace.

Step Command What to check
1 kubectl get pod POD -n NS -o wide Empty NODE → scheduling; assigned NODE with ContainerCreating → kubelet setup
2 kubectl describe pod POD -n NS | grep -A6 '^Conditions:' PodScheduled=False → scheduler branch; True → sandbox, volumes, or images

Log and manifest filtering in examples uses grep for quick matches. | 3 | kubectl describe pod POD -n NS \| grep -A8 '^Events:' | FailedScheduling for CPU, taints, affinity, or unbound PVCs | | 4 | kubectl events -n NS --for pod/POD --types=Warning | FailedMount, FailedCreatePodSandBox, or image-pull errors on an assigned node | | 5 | kubectl logs POD -n NS -c INIT_CONTAINER | When STATUS begins with Init: — application containers have not started yet |


Pending vs ContainerCreating

Pending is a Pod phase. It covers the period before every container in the Pod is running—including time waiting for scheduling and time on a node while the kubelet prepares containers.

ContainerCreating is a kubectl STATUS value commonly seen while the kubelet prepares a scheduled Pod. Use PodReadyToStartContainers and Events to distinguish sandbox and network setup from later image, volume, and container-creation work.

In Kubernetes 1.36, PodReadyToStartContainers=True means the runtime sandbox and Pod networking were created successfully; image pulling and container creation happen afterward.

The exact blocker lives in conditions and events, not in the label alone:

Signal Branch
PodScheduled=False Scheduler constraints
PodScheduled=True, PodReadyToStartContainers=False Sandbox, runtime, or CNI setup
PodReadyToStartContainers=True plus FailedMount ConfigMap, Secret, PVC, or mount setup
ErrImagePull / ImagePullBackOff Image and registry workflow
Init:* Init-container state and logs

Prepare and Classify the Lab Failures

Create the test Pods

Create four deliberate failures: impossible node selector, excessive CPU request, unbound PVC, and missing ConfigMap mount.

bash
kubectl apply -f - <<'YAML'
apiVersion: v1
kind: Namespace
metadata:
  name: pending-lab
---
apiVersion: v1
kind: Pod
metadata:
  name: no-match-node
  namespace: pending-lab
spec:
  nodeSelector:
    disktype: ssd-does-not-exist
  containers:
  - name: app
    image: busybox:1.36
    command: ["sleep", "3600"]
---
apiVersion: v1
kind: Pod
metadata:
  name: huge-cpu
  namespace: pending-lab
spec:
  containers:
  - name: app
    image: busybox:1.36
    command: ["sleep", "3600"]
    resources:
      requests:
        cpu: "500"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data-pvc
  namespace: pending-lab
spec:
  accessModes: [ReadWriteOnce]
  resources:
    requests:
      storage: 1Gi
  storageClassName: does-not-exist-sc
---
apiVersion: v1
kind: Pod
metadata:
  name: pvc-wait
  namespace: pending-lab
spec:
  containers:
  - name: app
    image: busybox:1.36
    command: ["sleep", "3600"]
    volumeMounts:
    - name: data
      mountPath: /data
  volumes:
  - name: data
    persistentVolumeClaim:
      claimName: data-pvc
---
apiVersion: v1
kind: Pod
metadata:
  name: bad-mount
  namespace: pending-lab
spec:
  containers:
  - name: app
    image: busybox:1.36
    command: ["sleep", "3600"]
    volumeMounts:
    - name: cfg
      mountPath: /etc/app
  volumes:
  - name: cfg
    configMap:
      name: missing-configmap
YAML

The value cpu: "500" deliberately requests 500 whole CPU cores. It is not 500 millicores; that would be written as 500m.

Sample output:

output
namespace/pending-lab created
pod/no-match-node created
pod/huge-cpu created
persistentvolumeclaim/data-pvc created
pod/pvc-wait created
pod/bad-mount created

Wait until the unscheduled Pods report PodScheduled=False:

bash
for pod in no-match-node huge-cpu pvc-wait; do
  kubectl wait "pod/$pod" -n pending-lab --for='jsonpath={.status.conditions[?(@.type=="PodScheduled")].status}=False' --timeout=120s
done

Wait until bad-mount is assigned and enters container setup:

bash
kubectl wait pod/bad-mount -n pending-lab --for=jsonpath='{.spec.nodeName}' --timeout=120s
bash
kubectl wait pod/bad-mount -n pending-lab --for='jsonpath={.status.containerStatuses[0].state.waiting.reason}=ContainerCreating' --timeout=120s

kubectl wait supports both condition and JSONPath checks. PodScheduled=False identifies the scheduler branch, while an assigned node moves investigation to sandbox, network, image, and volume setup.

Check node assignment and Pod conditions

List Pods with node placement—the first fork in the workflow:

bash
kubectl get pods -n pending-lab -o wide

Sample output:

output
NAME            READY   STATUS              RESTARTS   AGE   IP       NODE       NOMINATED NODE   READINESS GATES
bad-mount       0/1     ContainerCreating   0          35s   <none>   worker01   <none>           <none>
huge-cpu        0/1     Pending             0          35s   <none>   <none>     <none>           <none>
no-match-node   0/1     Pending             0          35s   <none>   <none>     <none>           <none>
pvc-wait        0/1     Pending             0          35s   <none>   <none>     <none>           <none>
Result Troubleshooting branch
NODE column empty Scheduling problem—read FailedScheduling
NODE assigned, container not running Container setup—read kubelet events on that node

Confirm the scheduler condition on an unscheduled Pod:

bash
kubectl describe pod no-match-node -n pending-lab | grep -A6 '^Conditions:'

Sample output:

output
Conditions:
  Type           Status
  PodScheduled   False 
Volumes:
  kube-api-access-7w7sk:
    Type:                    Projected (a volume that contains injected data from multiple sources)

PodScheduled False with empty NODE means no branch of container setup has started yet.

Read scheduler and kubelet Events

Read FailedScheduling events first—they state why no node accepted the Pod:

bash
kubectl describe pod no-match-node -n pending-lab | grep -A8 '^Events:'

Sample output:

output
Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  35s   default-scheduler  0/2 nodes are available: 1 node(s) didn't match Pod's node affinity/selector, 1 node(s) had untolerated taint(s). no new claims to deallocate, preemption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling.

Common scheduling blockers:

  • Insufficient CPU or memory versus Pod requests
  • nodeSelector or required node affinity terms
  • Node taints without matching Pod tolerations
  • Nodes not Ready or marked unschedulable (cordoned)
  • Unbound immediate PVCs referenced by the Pod

Filter warning events for the high-CPU Pod:

bash
kubectl events -n pending-lab --for pod/huge-cpu --types=Warning

Sample output:

output
LAST SEEN   TYPE      REASON             OBJECT         MESSAGE
35s         Warning   FailedScheduling   Pod/huge-cpu   0/2 nodes are available: 1 Insufficient cpu, 1 node(s) had untolerated taint(s). no new claims to deallocate, preemption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling.

Insufficient cpu points at requests versus node allocatable capacity—not application limits after startup.


Fix Unscheduled Pending Pods

Insufficient CPU or memory

Compare Pod requests to what nodes can allocate:

bash
kubectl get nodes -o custom-columns='NAME:.metadata.name,ALLOCATABLE-CPU:.status.allocatable.cpu,ALLOCATABLE-MEMORY:.status.allocatable.memory,UNSCHEDULABLE:.spec.unschedulable'

If requests exceed allocatable CPU or memory on every candidate node, the scheduler keeps the Pod Pending. Fix by lowering requests, freeing workloads, adding nodes, or removing cordon/taint blockers that shrink the schedulable pool.

Request and limit semantics, QoS classes, and limit behaviour after the container runs are covered in How to limit Kubernetes resources (CPU & Memory). This article only uses scheduling events to spot oversized requests.

Node selector, affinity, and taint conflicts

When events mention node affinity, node selector, or untolerated taint, open the matching fields in the Pod spec. Scheduling tutorials for Kubernetes taints and tolerations and node affinity cover intentional placement; here the event text tells you which knob to adjust.

Admission failures when no Pod exists

ResourceQuota, LimitRange, and admission webhook checks happen during API admission. They can reject Pod creation, meaning there may be no Pending Pod or FailedScheduling event to inspect. A controller such as a ReplicaSet can instead report FailedCreate.

When a Deployment requests replicas but no Pod object exists, inspect the Deployment and ReplicaSet for FailedCreate, then check ResourceQuota, LimitRange, and admission-policy errors:

bash
kubectl describe deployment <deployment> -n <namespace>
bash
kubectl describe replicaset <replicaset> -n <namespace>
bash
kubectl get resourcequota,limitrange -n <namespace>

Fix Unbound PVC Scheduling

Inspect the PVC and StorageClass

When FailedScheduling mentions unbound immediate PersistentVolumeClaims, the Pod waits for PVC binding before placement.

bash
kubectl get pvc -n pending-lab

Sample output:

output
NAME       STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS        VOLUMEATTRIBUTESCLASS   AGE
data-pvc   Pending                                      does-not-exist-sc   <unset>                 35s
bash
kubectl describe pod pvc-wait -n pending-lab | grep -A6 '^Events:'

Sample output:

output
Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  35s   default-scheduler  0/2 nodes are available: pod has unbound immediate PersistentVolumeClaims.

Inspect the claim itself:

bash
kubectl describe pvc data-pvc -n pending-lab

Representative relevant output:

output
Status:        Pending
StorageClass:  does-not-exist-sc

Events:
  Type     Reason              From                         Message
  ----     ------              ----                         -------
  Warning  ProvisioningFailed  persistentvolume-controller  storageclass.storage.k8s.io "does-not-exist-sc" not found
bash
kubectl get storageclass

This lab requests a StorageClass that does not exist. Because storageClassName cannot normally be changed after PVC creation, delete the consuming Pod and the unbound PVC, update the manifest to use an existing StorageClass, and recreate both. Alternatively, provide a compatible static PV with the requested class. Provisioning, binding modes, and recovery workflows are in Kubernetes PersistentVolume and PVC.

Immediate versus WaitForFirstConsumer binding

Immediate binding happens when the claim is created; WaitForFirstConsumer delays binding or provisioning until a consuming Pod exists so scheduling topology can be considered.


Fix Container Setup Failures

Once NODE is set, kubelet events explain setup stalls. The lab bad-mount Pod is assigned but not running:

bash
kubectl get pod bad-mount -n pending-lab

Sample output:

output
NAME        READY   STATUS              RESTARTS   AGE
bad-mount   0/1     ContainerCreating   0          35s

Read kubelet warnings:

bash
kubectl describe pod bad-mount -n pending-lab | grep -A8 '^Events:'

Sample output:

output
Events:
  Type     Reason       Age               From               Message
  ----     ------       ----              ----               -------
  Normal   Scheduled    35s               default-scheduler  Successfully assigned pending-lab/bad-mount to worker01
  Warning  FailedMount  5s (x6 over 35s)  kubelet            MountVolume.SetUp failed for volume "cfg" : configmap "missing-configmap" not found

Pod sandbox and CNI errors

FailedCreatePodSandBox means the runtime could not create the Pod network sandbox. Symptoms include CNI plugin errors, IP allocation failure, or network namespace creation errors.

Determine scope:

  • One Pod on one node — node CNI agent or local config
  • Every Pod on one node — kubelet or CNI on that worker
  • Cluster-wide — control plane or CNI controller outage

CNI installation and node-level network repair are outside this article's scope. Use the event to classify the failure, then follow the cluster networking or node runbook.

Image preparation failures

When STATUS becomes ErrImagePull or ImagePullBackOff, the kubelet reached image pull and failed. That is no longer a scheduling problem.

Follow Fix ImagePullBackOff and ErrImagePull for tag, registry auth, TLS, and DNS fixes. Do not duplicate registry troubleshooting on this page.

Volume attachment and mount errors

Beyond missing ConfigMaps, volume setup failures include:

  • PVC not Bound while referenced as a volume
  • Wrong claimName, or a PVC that does not exist in the Pod's namespace
  • hostPath file versus directory mismatch
  • Invalid or missing subPath — see Kubernetes subPath examples for mount semantics
  • Volume already attached read-write to another node, causing a multi-attach or attachment failure
  • ReadWriteOnce volume needed by Pods scheduled on different nodes
  • ReadWriteOncePod claim already in use by another Pod
  • Driver-specific attachment limits or topology constraints
  • Volume still attaching—FailedAttachVolume on cloud disks

A Pod's PVC reference resolves within the same namespace. General volume types and mount patterns are in Kubernetes volumes. Match the event string to PVC state, Secret or ConfigMap existence, or attachment status before editing application code.

Missing ConfigMaps and Secrets

Required volume references must exist in the same namespace as the Pod:

  • Object name spelling matches the volume source
  • Keys exist for items mappings
  • Optional references are intentionally optional—required refs block startup

Create the missing object or fix the volume definition. For the bad-mount lab:

bash
kubectl create configmap missing-configmap -n pending-lab --from-literal=app.conf=enabled

Sample output:

output
configmap/missing-configmap created

The kubelet retries the mount automatically. Verify that the existing Pod becomes Ready:

bash
kubectl wait pod/bad-mount -n pending-lab --for=condition=Ready --timeout=120s

Sample output:

output
pod/bad-mount condition met
bash
kubectl get pod bad-mount -n pending-lab

Sample output:

output
NAME        READY   STATUS    RESTARTS   AGE
bad-mount   1/1     Running   0          1m

Debug Init Containers

When STATUS begins with Init:—for example Init:0/1, Init:Error, or Init:CrashLoopBackOff—application containers have not started. Create a failing init container:

bash
kubectl apply -f - <<'YAML'
apiVersion: v1
kind: Pod
metadata:
  name: init-fail
  namespace: pending-lab
spec:
  initContainers:
  - name: setup
    image: busybox:1.36
    command: ["sh", "-c", "echo init-start; exit 1"]
  containers:
  - name: app
    image: busybox:1.36
    command: ["sleep", "3600"]
YAML

Wait for the stable backoff state:

bash
kubectl wait pod/init-fail -n pending-lab --for='jsonpath={.status.initContainerStatuses[0].state.waiting.reason}=CrashLoopBackOff' --timeout=120s
bash
kubectl get pod init-fail -n pending-lab

Sample output:

output
NAME        READY   STATUS                    RESTARTS   AGE
init-fail   0/1     Init:CrashLoopBackOff     2          35s

Read both the latest and previous attempts:

bash
kubectl logs init-fail -n pending-lab -c setup
bash
kubectl logs init-fail -n pending-lab -c setup --previous

Sample output:

output
init-start

Init:Error means an init execution failed; Init:CrashLoopBackOff means it has failed repeatedly. Application containers do not begin until regular init containers complete successfully. Init ordering, restart policy, and shared volumes are covered in Kubernetes init containers.


Verify the Fix

A healthy startup sequence looks like this:

  1. Scheduler assigns a node (PodScheduled=True, NODE populated)
  2. Sandbox and network setup complete without FailedCreatePodSandBox
  3. Volumes mount without repeating FailedMount
  4. Image is present or pulls successfully
  5. Init containers complete (Init:N/N clears)
  6. Application containers reach Running
  7. Readiness probes pass and READY matches container count

Watch events transition from Scheduled to Pulled/Created/Started. For controller-owned Pods, confirm kubectl rollout status after template fixes.

The bad-mount recovery above demonstrates a complete diagnosis-and-fix workflow: FailedMount cleared after the ConfigMap was created and the Pod reached 1/1 Ready.


Troubleshoot Common Pending and ContainerCreating Errors

Symptom or event Likely cause Fix
Pending, no NODE Scheduler rejection Read FailedScheduling message
FailedScheduling CPU/memory requests, selectors, taints, unbound PVC Lower requests, fix selectors/tolerations, bind PVC
Controller requests replicas but no Pod is created Quota, LimitRange, or admission rejection Inspect Deployment/ReplicaSet FailedCreate events and the API error
unbound immediate PersistentVolumeClaims PVC phase, StorageClass, available PVs Fix StorageClass or provide compatible PV
ContainerCreating for minutes Sandbox, image, volume, or configuration setup stalled Read recent Events and route FailedCreatePodSandBox, image-pull, FailedMount, or FailedAttachVolume to the matching fix
FailedCreatePodSandBox CNI and sandbox setup on the assigned node Follow cluster networking or node runbook
FailedMount ConfigMap, Secret, PVC name, subPath Create object or fix volume reference
FailedAttachVolume Volume attachment, access mode, cloud provider Resolve attachment or topology constraint
Init:Error / Init:CrashLoopBackOff Init-container logs and exit code Fix init command, volume, or dependency
ErrImagePull / ImagePullBackOff Image reference and registry Follow image-pull guide
Init:0/N long duration First init waiting or failing Logs for init container at index 0
Scheduled after PVC binds Delayed volume binding Normal for some StorageClasses—watch PVC phase
Works on one node only Taints, zone labels, or local volume Compare node labels and volume topology

What's Next


References


Summary

Pods stuck before startup split into two branches. Empty NODE and PodScheduled=False mean scheduling—read FailedScheduling for CPU, selectors, taints, or unbound PVCs. Assigned node with ContainerCreating or Init:* means kubelet setup—read FailedMount, sandbox errors, init logs, or image-pull messages on that worker.

The pending-lab Pods demonstrated both sides: no-match-node and huge-cpu never received a node; pvc-wait stalled on a missing StorageClass; bad-mount landed on worker01, recovered after the ConfigMap was created; init-fail stopped at Init:CrashLoopBackOff with init-start in init logs. Match the event text to the fix category instead of deleting Pods at random.

Once the container runs, failures belong elsewhere—image pull depth in the ImagePullBackOff guide, exit codes and restarts in the CrashLoopBackOff guide. When STATUS is ambiguous, use the Pod troubleshooting workflow to pick the branch before diving into scheduling or mount details.


Frequently Asked Questions

1. What is the difference between Pending and ContainerCreating?

Pending is a Pod phase before all containers are running. ContainerCreating is a kubectl STATUS while the kubelet prepares sandbox, network, volumes, and images after the Pod is assigned to a node. Unscheduled Pods stay Pending with no NODE column; scheduled Pods stuck in setup often show ContainerCreating.

2. How do I know if a Pending Pod failed scheduling?

Check PodScheduled in kubectl describe. When PodScheduled is False, read FailedScheduling events for CPU, memory, selectors, taints, or unbound PVCs. When PodScheduled is True but STATUS is ContainerCreating, troubleshoot kubelet setup on the assigned node.

3. Why does my Pod say unbound immediate PersistentVolumeClaims?

The Pod references a PVC that is not Bound yet. Inspect the PVC phase, StorageClass, access modes, and available PVs. The scheduler will not place the Pod until immediate PVC claims are bound or the volume binding mode allows delayed binding.

4. Can a missing ConfigMap keep a Pod in ContainerCreating?

Yes. A required ConfigMap or Secret volume reference that does not exist produces FailedMount events while the kubelet retries setup. The container does not start until the object exists or the volume reference is corrected.

5. Does Init:Error mean the application container failed?

No. Init:Error or Init:CrashLoopBackOff means a regular init container failed before application containers started. Read init-container logs with kubectl logs -c init-name and fix the init command, volume, or dependency.

6. Should I delete a Pending Pod to fix scheduling?

Deleting alone does not fix FailedScheduling. Correct requests, selectors, tolerations, PVC binding, or node availability first. Delete only when a controller must recreate the Pod after its template is corrected.
Deepak Prasad

R&D Engineer

Founder of GoLinuxCloud with more than 15 years of expertise in Linux, Python, Go, Laravel, DevOps, Kubernetes, Git, Shell scripting, OpenShift, AWS, Networking, and Security. With extensive experience, he excels across development, DevOps, networking, and security, delivering robust and efficient solutions for diverse projects.

  • Go (programming language)
  • Python (programming language)
  • DevOps
  • Computer Security
  • Cloud Computing
  • Kubernetes
  • Linux
  • Ansible (software)