Pod Restart Policies and Job Completion
Every other workload object exists to keep something running. A Job exists to run something until it succeeds and then stop - four completions two at a time, Pods that end up Completed rather than Running, and a CronJob that creates a fresh Job on every tick.
Kubernetes Fundamentals Guide 6 of 46 Beginner
- Kubernetes1.36.4
- Cluster4 nodes
- Runtimecontainerd 2.2.6
- CNICalico v3.32.1
- TimeAbout 14 min
- Reviewed23 August 2026
Written against the versions above. Job and CronJob are both in the `batch/v1` API group and have been stable for years. The `batch.kubernetes.io/job-name` label used here to select a Job's Pods replaced the older unprefixed `job-name` label in 1.27.
| Server Name | IP Address | OS | Roles | CPU | RAM | HDD |
|---|---|---|---|---|---|---|
| CKA1001 | 192.168.0.175 | Ubuntu 26.04 LTS | Control Plane Node | 2 Core | 4 GB | 50 GB |
| CKA1001-NODE01 | 192.168.0.176 | Ubuntu 26.04 LTS | Worker Node | 2 Core | 4 GB | 50 GB |
| CKA1001-NODE02 | 192.168.0.177 | Ubuntu 26.04 LTS | Worker Node | 2 Core | 4 GB | 50 GB |
| CKA1001-NODE03 | 192.168.0.178 | Ubuntu 26.04 LTS | Worker Node | 2 Core | 4 GB | 50 GB |
Before you start
- A cluster and kubectl. Nothing is installed; both objects are built in.
- This session runs a Job named
pithat computes 200 digits of pi in Perl, and a CronJob namedheartbeatthat prints a timestamp every minute.
-
Four completions, two at a time
A Job runs Pods until a set number of them succeed. This one asks for
completions: 4withparallelism: 2- four successful runs, no more than two at once:NAME STATUS COMPLETIONS DURATION AGE pi Complete 4/4 27s 27s4/2 completions/parallelism, succeeded=4Those two numbers do different jobs.
completionsis how much work there is;parallelismis how fast you are willing to do it. Raising parallelism does not change how much runs, only how long it takes - andbackoffLimit, set to 3 here, is how many failures the Job tolerates before it gives up entirely.This is the object to reach for when work has an end: a migration, a batch import, a report. A Deployment would restart it forever, because a Deployment's definition of healthy is *running*.
bash Example session kubectl wait --for=condition=Complete job/pi --timeout=300sjob.batch/pi condition metkubectl get job piNAME STATUS COMPLETIONS DURATION AGEpi Complete 4/4 27s 27skubectl get job pi -o jsonpath="{.spec.completions}/{.spec.parallelism} completions/parallelism, succeeded={.status.succeeded}{\"\n\"}"4/2 completions/parallelism, succeeded=4Expected resultA Job reporting Complete with 4 of 4 completions.
Success conditionYou can say what parallelism changes and what it does not.
-
The Pods stay, and that is deliberate
Look at what the Job left behind:
NAME READY STATUS RESTARTS AGE pi-5td9f 0/1 Completed 0 27s pi-bn9sg 0/1 Completed 0 3s pi-shckt 0/1 Completed 0 6sCompleted, not gone.0/1ready, because nothing is running in them any more - and they are still there. A Completed Pod is not a failure or a leak; it is the record. Its logs are still readable, which is the only reason you can ask a finished Job what it produced:3.14159265358979323846264338327950288419716939937510582097494459230781640628620899862803...This is where the disk goes on clusters that run a lot of batch work, so it is worth knowing the controls:
ttlSecondsAfterFinishedon the Job deletes it and its Pods a set time after finishing, and deleting the Job deletes its Pods with it. Nothing cleans up on its own by default.bash Example session kubectl get pods -l batch.kubernetes.io/job-name=piNAME READY STATUS RESTARTS AGEpi-5td9f 0/1 Completed 0 27spi-bn9sg 0/1 Completed 0 3spi-shckt 0/1 Completed 0 6skubectl logs -l batch.kubernetes.io/job-name=pi --tail=1 | head -23.14159265358979323846264338327950288419716939937510582097494459230781640628620899862803482534211706798214808651328230664709384460955058223172535940812848111745028410270193852110555964462294895493038203.1415926535897932384626433832795028841971693993751058209749445923078164062862089986280348253421170679821480865132823066470938446095505822317253594081284811174502841027019385211055596446229489549303820Expected resultCompleted Pods still listed, and their logs still readable.
Success conditionYou read output from a Pod that had already finished.
-
A CronJob makes a new Job every tick
A CronJob is a factory for Jobs.
schedule: "*/1 * * * *"- every minute:NAME SCHEDULE TIMEZONE SUSPEND ACTIVE LAST SCHEDULE AGE heartbeat */1 * * * * <none> False 0 <none> 0sA minute later there is a Job, with a name derived from the schedule tick:
NAME STATUS COMPLETIONS DURATION AGE heartbeat-29788229 Running 0/1 3s 3sand the CronJob now reports
ACTIVE 1and aLAST SCHEDULE.Three fields on a CronJob are worth setting on purpose, all present here.
concurrencyPolicy: Forbidskips a tick if the previous run is still going - without it a job that takes longer than its interval piles up.successfulJobsHistoryLimitandfailedJobsHistoryLimitcap how many finished Jobs are kept, which is the CronJob equivalent of the disk problem from the last step. AndTIMEZONEmeans the schedule runs in the cluster's time zone unless you settimeZoneexplicitly - a common source of jobs firing at an unexpected hour.bash Example session kubectl get cronjob heartbeatNAME SCHEDULE TIMEZONE SUSPEND ACTIVE LAST SCHEDULE AGEheartbeat */1 * * * * <none> False 0 <none> 0skubectl get jobsNAME STATUS COMPLETIONS DURATION AGEheartbeat-29788229 Running 0/1 3s 3sExpected resultA CronJob, then a Job created by it on the next minute boundary.
Success conditionYou can name what concurrencyPolicy prevents.
-
Run one now, without waiting for the clock
Waiting a minute to find out whether a CronJob works is a poor way to spend a minute.
--fromcopies the CronJob's template into a Job you trigger immediately:kubectl create job --from=cronjob/heartbeat heartbeat-manual2026-08-21T06:29:14ZThe same container, the same command, on demand. This is how you test a schedule's payload without touching the schedule, and it is the single most useful thing to know about CronJobs in practice.
The manual Job is an ordinary Job and is not counted against the CronJob's history limits, so remember to delete it. And
suspend: trueon the CronJob - theSUSPENDcolumn in the previous step - stops the schedule without deleting anything, which is what you want while debugging rather than removing the object.bash Example session kubectl create job --from=cronjob/heartbeat heartbeat-manualjob.batch/heartbeat-manual createdkubectl wait --for=condition=Complete job/heartbeat-manual --timeout=120sjob.batch/heartbeat-manual condition metkubectl logs job/heartbeat-manual2026-08-21T06:29:14ZExpected resultA Job created from the CronJob's template, completing immediately.
Success conditionYou ran the schedule's work without waiting for the schedule.
Troubleshooting
Completed Pods accumulate and fill the nodes' disks.
Why: Nothing removes finished Job Pods by default - they are kept as the record.
Fix:Set
ttlSecondsAfterFinishedon the Job, orsuccessfulJobsHistoryLimitandfailedJobsHistoryLimiton the CronJob. Deleting a Job deletes its Pods.A CronJob fires at the wrong hour.
Why: Without an explicit
timeZonethe schedule is interpreted in the cluster's time zone, not yours.kubectl get cronjobshows TIMEZONE as. Fix:Set
spec.timeZoneto an IANA name, and read the schedule back to confirm the column is no longer <none>.CronJob runs overlap and interfere with each other.
Why: The default concurrencyPolicy is Allow, so a run that outlasts its interval overlaps the next.
Fix:
concurrencyPolicy: Forbidto skip the tick, orReplaceto cancel the running one. Pick deliberately - Forbid silently drops work.A Job keeps creating Pods that fail.
Why: It retries up to
backoffLimitbefore marking itself Failed. Each attempt is a new Pod.Fix:
kubectl logs -l batch.kubernetes.io/job-name=<job>across all of them, and lower backoffLimit while debugging so it gives up quickly.