Set ulimit in Kubernetes Pods: open files, nproc, and process limits

If your application logs Too many open files, EMFILE, or Errno 24, the problem is usually the per-process file descriptor limit (ulimit -n / RLIMIT_NOFILE), not Kubernetes CPU or memory resources.limits. The same applies when you need to tune max user processes (nproc, RLIMIT_NPROC) or max file size (fsize, RLIMIT_FSIZE) inside a container—these are ulimit-style limits, the same family as ulimit on Linux on a VM.

On most clusters you still cannot declare nofile or nproc in a Pod manifest the way you declare CPU and memory. Containers inherit defaults from the container runtime (containerd, CRI-O), and those defaults vary by distro and version—sometimes very high, sometimes as low as 1024 open files. Running ulimit -n after kubectl exec only changes that exec shell, not your running app.

In this guide we show the pattern we use in production: wrap the container command with exec prlimit … -- (or ulimit in a shell entrypoint) so your application process starts with the limits you need, then verify with /proc/self/limits and small stress-test Pods you can apply on any cluster.

Tested on: kubectl client v1.36.1; API server/kubelet v1.35.0; kernel 6.14.0-37-generic.

We reproduced every example below on a test cluster with kubectl pointed at a namespace where you are allowed to create Pods. Use your usual kubeconfig—minikube, kind, k3s, EKS, GKE, or an on-prem cluster all work the same way for these manifests.

Why ulimit behaves differently in containers

If you are coming from bare metal or a VM, these habits do not carry over cleanly to Pods:

When we checked a fresh Pod with python:3.12-slim-bookworm, Max open files for PID 1 was already 1073741816—far above the 1024 default some articles warn about. Your cluster may differ (containerd version and node tuning matter), so always read /proc/1/limits on your workload before you change anything.

Map ulimit options to what you can set in a Pod

This table maps the ulimit flags you know from the shell to the prlimit names we use in entrypoints:

You can only raise a hard limit up to what the runtime already allows unless the platform grants CAP_SYS_RESOURCE or you run privileged (avoid in production unless policy allows it).

Step-by-step examples: set and prove ulimit limits in your Pods

Each subsection below is a standalone Pod manifest you can save and apply on your cluster with kubectl apply -f. After the Pod starts, read kubectl logs <pod-name> to see what the process actually hit—that is more reliable than ulimit -a inside a one-off exec session.

Baseline: check current ulimit and file descriptor limits in your Pod

Before you increase or decrease anything, we always inspect what the container runtime gave PID 1. In this example we start a minimal Pod whose only job is to print /proc/1/limits and then sleep:

apiVersion: v1
kind: Pod
metadata:
  name: ulimit-baseline
spec:
  restartPolicy: Never
  containers:
  - name: app
    image: python:3.12-slim-bookworm
    command: ["/bin/sh", "-c"]
    args:
    - |
      echo "=== PID 1 limits (/proc/1/limits) ==="
      cat /proc/1/limits
      sleep 3600

Apply it and read the startup logs:

kubectl apply -f ulimit-baseline.yaml
kubectl logs ulimit-baseline | head -20

On our test cluster the output looked like this (truncated):

Max processes             unlimited            unlimited            processes
Max open files            1073741816           1073741816           files
Max file size             unlimited            unlimited            bytes

As you can see, Max open files was already enormous on this cluster—so if you see EMFILE here, the leak or spike is almost certainly in the application, not a 1024 default. On other platforms (older containerd, Fargate, tight node tuning) you may see 1024 or 65535 instead—always trust /proc/1/limits over blog defaults.

Why kubectl exec ulimit does not change your application limits

A common mistake when debugging Too many open files is to kubectl exec into the Pod, run ulimit -n 65535, and expect the main process to pick it up. It will not—ulimit only affects the current shell.

We reproduced that with a Pod that starts a background Python process, then runs ulimit -n 32 in a subshell while the app keeps running:

apiVersion: v1
kind: Pod
metadata:
  name: ulimit-exec-mismatch
spec:
  restartPolicy: Never
  containers:
  - name: app
    image: python:3.12-slim-bookworm
    command: ["/bin/sh", "-c"]
    args:
    - |
      python3 -c "import time; time.sleep(3600)" &
      APP_PID=$!
      sleep 1
      echo "=== app PID limits before exec ulimit ==="
      grep 'Max open files' /proc/${APP_PID}/limits
      echo "=== subshell ulimit -n 32 does not change app PID ==="
      (ulimit -n 32; true)
      grep 'Max open files' /proc/${APP_PID}/limits
      sleep 3600

The logs tell the story:

=== app PID limits before exec ulimit ===
Max open files            1073741816           1073741816           files
=== subshell ulimit -n 32 does not change app PID ===
Max open files            1073741816           1073741816           files

The app PID never moved off 1073741816. Do not rely on post-start exec—or an InitContainer ulimit command—to fix EMFILE on a running workload.

Limiting max open files (nofile) to reproduce EMFILE / Too many open files

To prove enforcement, we first run a one-shot child with a very low nofile cap. This shows the kernel error your app would see, but note that PID 1 (the shell) still keeps runtime defaults unless you use exec prlimit:

prlimit --nofile=32:32 -- python3 -c "
import os
fds = []
try:
    for i in range(40):
        fds.append(os.open('/dev/null', os.O_RDONLY))
except OSError as e:
    print(f'failed after {len(fds)} opens: {e}')
"

When we ran that locally, Python stopped with:

failed after 29 opens: [Errno 24] Too many open files: '/dev/null'

As you can see, after about 29 successful opens the kernel returned Errno 24—that is the same EMFILE / Too many open files path databases and high-concurrency services hit in production.

For a long-running service, we wrap the real command with exec prlimit so the application becomes PID 1 with the limit baked in. In this Pod spec we set nofile to 32:32 on purpose (far too low for production) so the failure is obvious in logs:

apiVersion: v1
kind: Pod
metadata:
  name: ulimit-nofile-entrypoint
spec:
  restartPolicy: Never
  securityContext:
    runAsNonRoot: true
    runAsUser: 65534
    runAsGroup: 65534
    seccompProfile:
      type: RuntimeDefault
  containers:
  - name: app
    image: python:3.12-slim-bookworm
    env:
    - name: PYTHONUNBUFFERED
      value: "1"
    command: ["/bin/sh", "-c"]
    args:
    - |
      exec prlimit --nofile=32:32 -- python3 -c "
      import os, time
      print('=== app PID limits ===')
      for line in open('/proc/self/limits'):
          if 'Max open files' in line:
              print(line.rstrip())
      fds = []
      try:
          for i in range(40):
              fds.append(os.open('/dev/null', os.O_RDONLY))
      except OSError as e:
          print(f'failed after {len(fds)} opens: {e}')
      time.sleep(3600)
      "

After kubectl apply, the Pod logs showed both the enforced limit and the failure:

=== app PID limits ===
Max open files            32                   32                   files
failed after 29 opens: [Errno 24] Too many open files: '/dev/null'

You can double-check PID 1 from another terminal:

kubectl exec ulimit-nofile-entrypoint -- grep 'Max open files' /proc/1/limits

Max open files            32                   32                   files

For production you would swap 32:32 for something like 65535:65535 (as long as it stays at or below the runtime hard ceiling). The same command/args pattern drops straight into a Deployment.

Limiting max file size (fsize) inside a container

Some batch jobs need a hard cap on how large a single file may grow. Here we use exec prlimit --fsize=1024:1024 and attempt to write 2048 bytes—twice the soft limit:

apiVersion: v1
kind: Pod
metadata:
  name: ulimit-fsize-demo
spec:
  restartPolicy: Never
  securityContext:
    runAsNonRoot: true
    runAsUser: 65534
    runAsGroup: 65534
  containers:
  - name: app
    image: python:3.12-slim-bookworm
    env:
    - name: PYTHONUNBUFFERED
      value: "1"
    command: ["/bin/sh", "-c"]
    args:
    - |
      exec prlimit --fsize=1024:1024 -- python3 -c "
      import time
      print('=== app PID limits ===')
      for line in open('/proc/self/limits'):
          if 'Max file size' in line:
              print(line.rstrip())
      try:
          with open('/tmp/big.bin', 'wb') as fh:
              fh.write(b'x' * 2048)
          print('wrote 2048 bytes (unexpected)')
      except OSError as e:
          print(f'write blocked: {e}')
      time.sleep(3600)
      "

The logs confirmed RLIMIT_FSIZE and blocked the oversized write:

=== app PID limits ===
Max file size             1024                 1024                 bytes
write blocked: [Errno 27] File too large

Restricting max user processes (nproc) in a Pod

Thread-heavy or fork-heavy workloads can hit nproc before CPU limits matter. We capped nproc at 16:16 and tried to spawn thirty sleep children:

apiVersion: v1
kind: Pod
metadata:
  name: ulimit-nproc-demo
spec:
  restartPolicy: Never
  securityContext:
    runAsNonRoot: true
    runAsUser: 65534
    runAsGroup: 65534
  containers:
  - name: app
    image: python:3.12-slim-bookworm
    env:
    - name: PYTHONUNBUFFERED
      value: "1"
    command: ["/bin/sh", "-c"]
    args:
    - |
      exec prlimit --nproc=16:16 -- python3 -c "
      import subprocess, time
      print('=== app PID limits ===')
      for line in open('/proc/self/limits'):
          if 'Max processes' in line:
              print(line.rstrip())
      procs = []
      try:
          for i in range(30):
              procs.append(subprocess.Popen(['sleep', '300']))
          print(f'started {len(procs)} processes (unexpected)')
      except OSError as e:
          print(f'failed after {len(procs)} processes: {e}')
      for p in procs:
          p.terminate()
      time.sleep(3600)
      "

As you can see, spawn failures started well before thirty processes—the kernel reported Resource temporarily unavailable after seven children:

=== app PID limits ===
Max processes             16                   16                   processes
failed after 7 processes: [Errno 11] Resource temporarily unavailable

On some clusters a cgroup pids limit may bite before RLIMIT_NPROC; when in doubt, read /proc/self/limits and the pod cgroup files together.

Set ulimit in a Deployment with an entrypoint wrapper

For a real service we keep a small wrapper script in the image (or mount it from a ConfigMap). This is the production shape we recommend when you need to increase ulimit in a Kubernetes pod without touching node config:

#!/bin/sh
set -eu
# Tune SOFT:HARD for your workload; must be <= runtime hard limit unless you have CAP_SYS_RESOURCE.
exec prlimit --nofile=65535:65535 -- "$@"

Wire it into your Deployment like this—the container runs the wrapper, and the wrapper execs your binary so signals still reach the app:

spec:
  template:
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
      containers:
      - name: app
        image: your-registry/your-app:1.0
        command: ["/entrypoint.sh"]
        args: ["/usr/local/bin/your-app", "--config", "/config/app.yaml"]

If you prefer shell-only tuning without prlimit, the same idea works with ulimit -n immediately before exec:

ulimit -n 65535
exec /usr/local/bin/your-app "$@"

Either pattern is portable across Kubernetes versions and does not require privileged Pods.

Multiple processes: which entrypoint script to use

ulimit and prlimit apply to one process. Child processes inherit the parent’s limits at fork time—they do not pick up limits you set later in a sibling or in another container. How you wire the entrypoint depends on how many processes you run inside one container, and whether you use sidecars.

One main app that forks workers or threads

If your container runs a single binary (Java, Go, nginx, a Python app) and that process creates threads or child workers, wrap that binary with exec prlimit so it becomes PID 1 with the right limits. Every descendant inherits them:

#!/bin/sh
set -eu
exec prlimit --nofile=65535:65535 --nproc=4096:4096 -- /usr/local/bin/your-app "$@"

After deploy, one check on the app PID is enough: cat /proc/1/limits (when the app is PID 1) or cat /proc/<app-pid>/limits.

Shell entrypoint that starts several background processes

Legacy images sometimes use a shell script that launches more than one daemon in the same container. Set limits on the shell before you start any background job—children forked from that shell inherit what the shell already has:

#!/bin/sh
set -eu

# Apply once on the entrypoint shell; background children inherit these limits.
ulimit -n 65535
ulimit -u 4096

/usr/local/bin/worker-a &
/usr/local/bin/worker-b &
wait

If you need different limits per worker, run prlimit on each command instead of relying on one shared shell ulimit:

#!/bin/sh
set -eu

prlimit --nofile=65535:65535 -- /usr/local/bin/worker-a &
prlimit --nofile=131072:131072 -- /usr/local/bin/worker-b &
wait

Each worker gets its own limit set at start time. PID 1 remains the shell (runtime defaults on the shell itself usually do not matter unless the shell opens many files).

Supervisor or init wrapper (supervisord, tini, s6)

When a supervisor starts your processes, wrap the supervisor with exec prlimit, not each program inside its config—supervised children inherit from the supervisor when it forks them:

#!/bin/sh
set -eu
exec prlimit --nofile=65535:65535 --nproc=8192:8192 -- /usr/bin/supervisord -c /etc/supervisord.conf

If the supervisor re-execs or launches via an intermediate shell that does not inherit (unusual), set limits in the supervisor’s own program stanzas or use prlimit in each program’s command= line.

Sidecars and multiple containers in one Pod

Each container has its own PID 1 and cgroup. An entrypoint wrapper on the app container does not change limits on a logging or metrics sidecar. Repeat the same wrapper pattern on every container that needs tuned nofile or nproc:

containers:
- name: app
  command: ["/entrypoint.sh"]
  args: ["/usr/local/bin/app"]
- name: metrics-exporter
  command: ["/entrypoint.sh"]
  args: ["/usr/local/bin/exporter"]

Use the same /entrypoint.sh image layer or ConfigMap mount in both.

How to verify when more than one process matters

Do not assume PID 1 is the process hitting EMFILE. For multi-process containers, check the process that actually errors:

# List processes in the container
kubectl exec <pod> -c app -- ps aux

# Read limits for a specific PID (replace 42)
kubectl exec <pod> -c app -- sed -n '1,20p' /proc/42/limits

If limits differ between PIDs, trace which parent started the misconfigured process and move prlimit/ulimit to that parent’s start path.

Unprivileged pods and InitContainers

Does the entrypoint work on unprivileged pods?

Yes. You do not need privileged: true, root, or CAP_SYS_RESOURCE for the usual case: starting your app with exec prlimit or ulimit in the entrypoint. The step-by-step examples in this guide already run that way—we use runAsNonRoot: true, runAsUser: 65534, and seccompProfile: RuntimeDefault on the demo Pods.

On an unprivileged Pod you can:

• Lower limits freely (for example prlimit --nofile=32:32 to reproduce EMFILE in a test).
• Raise soft and hard limits up to the hard ceiling the container runtime already assigned when your process starts.

You cannot, without extra privilege or a platform change:

• Push the hard limit above that runtime ceiling—prlimit returns Operation not permitted.
• Expect node-wide defaults to change because you edited something in the Pod YAML—only the main container entrypoint (or each sidecar’s entrypoint) sets limits for that container’s processes.

Production-shaped fragment with a non-root, non-privileged Pod:

spec:
  securityContext:
    runAsNonRoot: true
    seccompProfile:
      type: RuntimeDefault
  containers:
  - name: app
    image: your-registry/your-app:1.0
    securityContext:
      allowPrivilegeEscalation: false
      capabilities:
        drop: ["ALL"]
      runAsUser: 1000
      runAsGroup: 1000
    command: ["/entrypoint.sh"]
    args: ["/usr/local/bin/your-app"]

Keep /entrypoint.sh as exec prlimit --nofile=65535:65535 -- "$@". If 65535 is above your cluster’s runtime hard cap, lower both numbers until prlimit succeeds, or ask the platform team to raise node/runtime defaults.

That is why this approach fits restricted and baseline Pod Security namespaces today—unlike KEP-5758 securityContext.ulimits, which (in alpha) is tied to privileged namespaces only.

Can an InitContainer set ulimit for the main container?

No. An InitContainer is a separate container: its own cgroup, its own PID 1, and its own /proc/1/limits. Running ulimit or prlimit there only affects processes inside that init container. When init finishes, the main container starts with runtime defaults unless its entrypoint applies limits.

This pattern does not fix EMFILE on your app:

initContainers:
- name: set-ulimit
  image: busybox
  command: ["sh", "-c", "ulimit -n 65535; echo done"]
containers:
- name: app
  image: your-app:1.0
  # main app still inherits runtime defaults—init ulimit never reached it

Making the InitContainer privileged does not help either—it still does not change another container’s process limits.

If init itself needs a higher nofile for a one-off setup task (large unpack, migration script), wrap that init command only:

initContainers:
- name: seed-data
  image: your-registry/tools:1.0
  command: ["/bin/sh", "-c"]
  args:
  - exec prlimit --nofile=8192:8192 -- /scripts/seed.sh

containers:
- name: app
  image: your-registry/your-app:1.0
  command: ["/entrypoint.sh"]
  args: ["/usr/local/bin/your-app"]

The long-running workload still needs /entrypoint.sh (or equivalent) on the main container. Forum tricks that write ulimit into a shared volume or .bashrc only work if the main container’s entrypoint actually sources that file—we recommend an explicit wrapper instead.

Modify or remove ulimit settings

There is nothing to kubectl apply for ulimit itself—only your Pod template command/args or image entrypoint changes.

What does not work

These approaches show up often in forum answers; we skip them because they do not reliably set limits on your application process:

• Mounted limits.conf only — does not alter /proc/1/limits without a full PAM login stack.
• ulimit in an InitContainer — separate container and cgroup; the main app never sees it.
• Privileged InitContainer or DaemonSet ulimit -n … — same isolation problem; does not retroactively change another container.
• kubectl exec ulimit — changes the exec shell only (see the exec-mismatch lab above).
• Editing containerd.service LimitNOFILE alone — that caps the containerd daemon, not a declarative per-Pod nofile limit (node-level runtime Rlimits in containerd config is a platform-wide default, not per workload).
• Raising hard above runtime ceiling without privilege — prlimit returns Operation not permitted; needs platform change or CAP_SYS_RESOURCE.

Future: native ulimits in Kubernetes (KEP-5758, alpha)

Upstream is adding a declarative path via KEP-5758: Per-container ulimits configuration. In Kubernetes 1.36 this is alpha, gated by ContainerUlimits, and the shape looks like this on the container securityContext:

containers:
- name: db-container
  image: mysql:latest
  securityContext:
    ulimits:
    - name: nofile
      soft: 65536
      hard: 65536
    - name: memlock
      soft: -1
      hard: -1

Treat that snippet as a preview, not something you can paste on every cluster today. Caveats we watch closely:

Is our prlimit entrypoint future-proof?

Yes—for the workloads this guide targets, the entrypoint pattern remains valid even after KEP-5758 graduates:

• It sets the same kernel RLIMIT values the native API will eventually set at container start.
• It works on unprivileged Pods in restricted namespaces where declarative ulimits may stay disallowed.
• It covers nproc and fsize, which are common production needs but outside the current alpha API set.
• When your platform team enables declarative ulimits, you can migrate privileged workloads to YAML and remove the wrapper—or keep the wrapper as a portable fallback across clusters and cloud vendors.

We plan to add a dedicated migration note once KEP-5758 reaches beta and ships on managed offerings (EKS, GKE, AKS). Until then, exec prlimit … -- is the approach we trust on real clusters.

Troubleshooting

Application still hits “Too many open files”

1. On a fresh Pod after deploy, run kubectl exec → cat /proc/1/limits.
2. Confirm the entrypoint uses exec prlimit (not a one-shot child process that exits).
3. Check sidecars—each container has its own PID 1 and its own limits.

prlimit: Operation not permitted

You tried to set soft/hard above the allowed hard limit. Lower the target or ask the platform team to adjust container runtime defaults (for example containerd Rlimits in config.toml on self-managed nodes).

Limits look fine but kubectl exec test behaves differently

Processes started by kubectl exec inherit exec/session defaults, not necessarily PID 1 limits. Test using the application PID or startup logs.

Confused with OOMKilled or CPU throttling

Those come from cgroup memory/CPU and resources:—out of scope here. For exit code 137, see Kubernetes OOMKilled.

Cleanup

When you are done testing, delete the demo Pods from your namespace:

kubectl delete pod ulimit-baseline ulimit-exec-mismatch ulimit-nofile-entrypoint \
  ulimit-fsize-demo ulimit-nproc-demo --ignore-not-found

Summary

Most Kubernetes clusters still do not offer a stable, declarative ulimit field for ordinary Pods. When you need to increase nofile, tune nproc, or cap file size, wrap your container command with exec prlimit or ulimit before exec, then verify with /proc/self/limits. For multiple processes in one container, set limits on the parent that forks them (entrypoint shell, main binary, or supervisor); give each sidecar its own wrapper. The step-by-step Pod examples show why kubectl exec ulimit and InitContainer tricks fail, and how low nofile caps reproduce EMFILE / Too many open files. KEP-5758 will add securityContext.ulimits for alpha clusters, but the entrypoint pattern stays portable and future-proof—especially for unprivileged namespaces and limits not yet in the alpha API. This guide intentionally avoids CPU/memory resources.limits; those are a separate control plane.

References

• ulimit on Linux (VM/bare metal)
• Kubernetes Pod
• Kubernetes OOMKilled (exit code 137)
• KEP-5758: Per-container ulimits configuration
• prlimit(1) — Linux manual page
• setrlimit(2)