Hands-on Lab·Certified Kubernetes Administrator
Troubleshooting Failed Jobs
A Job that gives up, read from the object rather than guessed at - and the one field that decides whether every attempt leaves evidence behind or only the last one does.
Troubleshooting Guide 85 of 103 Intermediate
- Kubernetes1.36.4
- Cluster4 nodes
- Runtimecontainerd 2.2.6
- CNICalico v3.32.1
- TimeAbout 15 min
The job-name label is batch.kubernetes.io/job-name on current releases; older clusters use the unprefixed job-name.
| Server Name | IP Address | OS | Roles | CPU | RAM | HDD |
|---|---|---|---|---|---|---|
| CKA1001 | 192.168.0.175 | Ubuntu 26.04 LTS | Control Plane Node | 2 Core | 4 GB | 50 GB |
| CKA1001-NODE01 | 192.168.0.176 | Ubuntu 26.04 LTS | Worker Node | 2 Core | 4 GB | 50 GB |
| CKA1001-NODE02 | 192.168.0.177 | Ubuntu 26.04 LTS | Worker Node | 2 Core | 4 GB | 50 GB |
| CKA1001-NODE03 | 192.168.0.178 | Ubuntu 26.04 LTS | Worker Node | 2 Core | 4 GB | 50 GB |
This guide includes
Use this when a Job has given up. This matters because a Job stops on purpose after backoffLimit attempts, and with restartPolicy: Never every attempt leaves a differently named Pod - so collecting the logs needs a selector rather than a name.
- writing a Job that cannot succeed, with an explicit
backoffLimit - reading from the Job object itself why it stopped, rather than guessing at it
- collecting logs across every attempt using the job-name label selector
- running the same failure with
restartPolicy: OnFailureand seeing what changes
Before you start
- guide 30 - what a Job is for, before this covers what it does when it cannot finish.
-
A Job that cannot succeed
-
Why it stopped
-
Getting the logs when every attempt has its own Pod
-
The same failure with OnFailure