Container Logs with kubectl logs

Troubleshooting / Guide 77 of 103 / Beginner

• Kubernetes1.36.4
• Runtimecontainerd 2.2.6
• Cluster4 nodes
• CNICalico v3.32.1
• TimeAbout 25 min
• Reviewed21 August 2026

Written against the versions above. Log retention is the node's, not Kubernetes'. Nothing here survives Pod deletion.

Before you start

• The Pods guide, for multi-container Pods.
• The CrashLoopBackOff guide, for --previous and its limits.
• A scratch namespace.

1. 1

   Two containers, and the one kubectl picks for you

   The Pod has web (nginx) and sidecar (a loop printing a counter). Two log streams, and kubectl logs has to choose.

   With no -c it takes the first container in the spec and, helpfully, says so:

   Defaulted container "web" out of: web, sidecar

   That line goes to stderr, so it is easy to lose when piping. Worth knowing, because otherwise you read nginx's startup messages while looking for your application's output and conclude the application logged nothing.

   -c sidecar picks explicitly. The container name is what appears in the Pod spec, not the image name.

   A useful detail about ordering: since the default is the first container, the container you most want logs from is worth putting first in the spec. It also determines what kubectl exec and kubectl attach default to.

   $kubectl get pod app -n dbgNAME   READY   STATUS    RESTARTS   AGEapp    2/2     Running   0          1s$kubectl logs app -n dbg 2>&1 | tail -32026/08/21 09:37:57 [notice] 1#1: start worker processes2026/08/21 09:37:57 [notice] 1#1: start worker process 292026/08/21 09:37:57 [notice] 1#1: start worker process 30$kubectl logs app -n dbg -c sidecar --tail=3sidecar tick 1$kubectl exec app -n dbg -- hostnameDefaulted container "web" out of: web, sidecarapp

   Expected resultnginx output from the default container, one tick from the sidecar, and the Defaulted container notice shown here via exec where it is not filtered away by the pipe. 2/2 in the READY column is the reminder that there are two streams to choose between.

   Success conditionYou can read either container's logs by name.

2. 2

   All containers at once, with prefixes

   --all-containers=true --prefix=true is the combination worth memorising. It merges every container's output and labels each line with its source:

   [pod/app/web] ...
   [pod/app/sidecar] sidecar tick 1

   That is how you see a request arriving at a proxy container and the application handling it, in one view, without guessing at timing.

   --prefix is what makes it usable. Without it the streams are interleaved with no indication of which line came from where, which for two containers logging similar things is worse than reading them separately.

   Note --tail=4 applied per container, not to the merged output: four nginx lines plus the sidecar's one. Every --tail and --since in this guide is per container.

   The same works across Pods with -l. kubectl logs -l app=demo -c sidecar reads that container from every Pod matching the label, which is the fastest way to see what all replicas of a Deployment are saying. Two limits to know: it caps at five Pods by default (raise with --max-log-requests), and it does not follow new Pods appearing after you started.

   $kubectl logs app -n dbg --all-containers=true --prefix=true --tail=4[pod/app/web] 2026/08/21 09:37:57 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1024:524288[pod/app/web] 2026/08/21 09:37:57 [notice] 1#1: start worker processes[pod/app/web] 2026/08/21 09:37:57 [notice] 1#1: start worker process 29[pod/app/web] 2026/08/21 09:37:57 [notice] 1#1: start worker process 30[pod/app/sidecar] sidecar tick 1$kubectl logs -n dbg -l app=demo -c sidecar --tail=2 --prefix=true[pod/app/sidecar] sidecar tick 2[pod/app/sidecar] sidecar tick 3

   Expected resultPrefixed lines from both containers, then the label-selected form. One Pod matches here, so the label version shows one prefix; on a Deployment with several replicas each would be named.

   Success conditionPrefixed output identifies which container produced each line.

3. 3

   Narrowing by time

   --since takes a duration and returns only what was written inside it. --since=10s after a 12-second wait gives ticks 2 and 3, not tick 1.

   This is the flag that matters on a real system. A container that has been running for a week has a log you cannot read, and the question is almost always "what happened in the last few minutes". --since=5m answers it.

   There is also --since-time for an absolute RFC3339 timestamp, which is what you want when correlating against an incident report.

   --timestamps=true prefixes each line with the time the runtime received it, in nanosecond RFC3339. Two reasons to use it:

   • Many applications log without timestamps, or in local time, or in a format that does not sort. The runtime's timestamp is consistent, UTC, and sortable regardless.
   • When comparing two containers' logs, only a common clock makes the ordering meaningful.

   The timestamps here also reveal the sidecar's interval precisely: 09:38:02.395439 and 09:38:07.395575, five seconds apart to within a fraction of a millisecond, which is the sleep 5 in its loop.

   Two more flags worth knowing, not shown here because they do not terminate: -f follows the stream like tail -f, and combining -f with --tail=20 starts from recent history rather than replaying everything.

   $sleep 12; kubectl logs app -n dbg -c sidecar --since=10ssidecar tick 2sidecar tick 3$kubectl logs app -n dbg -c sidecar --timestamps=true --tail=22026-08-21T09:38:02.395439433Z sidecar tick 22026-08-21T09:38:07.395575353Z sidecar tick 3

   Expected resultTick 1 excluded by the time window, and nanosecond timestamps on the same two lines. Note the timestamps are the runtime's, so they are trustworthy even for an application that prints none.

   Success condition--since=10s returns fewer lines than the container has produced.

Troubleshooting

• You are reading logs and the application's output is not there.

  Why: A multi-container Pod defaulted to the first container, which is not yours.

  Fix:Look for Defaulted container "x" out of: x, y on stderr; a pipe hides it. Use -c <name>, or --all-containers --prefix to see everything at once. kubectl get pod <name> -o jsonpath='{.spec.containers[*].name}' lists the names.

• kubectl logs returns nothing at all, with no error.

  Why: The container wrote nothing to stdout or stderr. Usually it logs to a file inside the container instead.

  Fix:Kubernetes only collects the container's standard streams. Check for a log file: kubectl exec <pod> -- ls -la /var/log/, and read it with kubectl exec <pod> -- cat /var/log/app.log. The real fix is to configure the application to log to stdout, which is what containerised applications are expected to do.

• Logs from before a restart are gone.

  Why: kubectl logs shows the current container. Rotation and the runtime's retention govern the rest.

  Fix:--previous reads the immediately preceding container, and only while its files remain, which is not long after several fast restarts. There is no --previous=2. For anything you need later, ship logs off the node.

• A container's logs stop partway through and it is still running.

  Why: Log rotation. The kubelet rotates at 10Mi per container by default and keeps a limited number of files, so a chatty container overwrites its own history quickly.

  Fix:--since will not recover what was rotated away. Reduce the volume, or raise containerLogMaxSize and containerLogMaxFiles in the kubelet configuration if the node has disk to spare. This is another argument for external log collection.

• kubectl logs -l misses some Pods.

  Why: It reads at most five Pods by default.

  Fix:Raise it with --max-log-requests=20, and note each request is a separate connection to a kubelet, so a large number is genuinely expensive. For a Deployment with many replicas, sampling a few is usually the better approach.

• kubectl logs fails with a timeout while kubectl get works.

  Why: Logs are served by the kubelet on port 10250 and proxied by the API server. get only touches etcd.

  Fix:This narrows the fault precisely: the control plane is fine and the path to that node's kubelet is not. Check the node is reachable on 10250 from the control plane and that its kubelet is running. kubectl exec and port-forward fail the same way for the same reason.

Official sources

• Logging architecture
• kubectl logs reference
• Debug running Pods
• Log rotation settings
• Sidecar containers