CertGrid CertGrid
Hands-on Lab·Kubernetes and Cloud Native Associate

nodeSelector and Scheduling Constraints

Two Pods with an identical nodeSelector pointing at the same node. One runs, one stays Pending forever. The difference is three lines the node itself put in the way - and the distinction between what a Pod asks for and what a node accepts is most of what scheduling is.

Scheduling and Placement Guide 14 of 46 Beginner

Written against the versions above. Labels, nodeSelector, affinity and taints are core Kubernetes and behave the same on any conformant cluster. The control plane's own node-role.kubernetes.io/control-plane taint is a kubeadm convention - managed services usually hide the control plane from you entirely.

One control plane and three workers. The control plane carries a taint from the day it was installed, which is why it never runs your Pods.
Server NameIP AddressOSRolesCPURAMHDD
CKA1001192.168.0.175Ubuntu 26.04 LTSControl Plane Node2 Core4 GB50 GB
CKA1001-NODE01192.168.0.176Ubuntu 26.04 LTSWorker Node2 Core4 GB50 GB
CKA1001-NODE02192.168.0.177Ubuntu 26.04 LTSWorker Node2 Core4 GB50 GB
CKA1001-NODE03192.168.0.178Ubuntu 26.04 LTSWorker Node2 Core4 GB50 GB

Before you start

  1. Nodes are labelled, and selectors are how you talk about them

    Scheduling starts with labels, because that is the only vocabulary the scheduler has for describing a node. Put two on a worker:

    node/cka1001-node01 labeled

    Now the same selector syntax that works on Pods works on nodes. Equality, a column view, set membership, and existence:

    kubectl get nodes -l disk=ssd            # just node01
kubectl get nodes -L disk,tier            # all nodes, labels as columns
kubectl get nodes -l "disk in (ssd,nvme)"  # set membership
kubectl get nodes -l "!disk"               # the three WITHOUT the label

    -l filters, -L displays. The !disk form is the one worth remembering: selectors can ask for the absence of a label, which is how you find the nodes a placement rule will not match. Every node already carries labels nobody set - kubernetes.io/hostname, kubernetes.io/arch, kubernetes.io/os - and those are usable in exactly the same way.

    bash Example session 
    kubectl label node cka1001-node01 disk=ssd tier=fastnode/cka1001-node01 labeledkubectl get nodes -l disk=ssdNAME             STATUS   ROLES    AGE    VERSIONcka1001-node01   Ready    <none>   150m   v1.36.4kubectl get nodes -L disk,tierNAME             STATUS   ROLES           AGE    VERSION   DISK   TIERcka1001          Ready    control-plane   151m   v1.36.4          cka1001-node01   Ready    <none>          150m   v1.36.4   ssd    fastcka1001-node02   Ready    <none>          150m   v1.36.4          cka1001-node03   Ready    <none>          85m    v1.36.4          kubectl get nodes -l "!disk"NAME             STATUS   ROLES           AGE    VERSIONcka1001          Ready    control-plane   151m   v1.36.4cka1001-node02   Ready    <none>          150m   v1.36.4cka1001-node03   Ready    <none>          85m    v1.36.4

    Expected resultOne node carries the new labels; three do not, and a selector can find either set.

    Success conditionYou can filter nodes by a label and by the absence of one.

  2. nodeSelector is a requirement; affinity can be a preference

    A Deployment with nodeSelector: {disk: ssd} and three replicas. All three land on the one node that has the label:

          3 cka1001-node01

    nodeSelector is absolute. If no node matches, the Pods do not go anywhere - they stay Pending indefinitely, which is one of the two Pending causes worth recognising on sight.

    Node affinity says the same kind of thing with a dial. This second Deployment uses preferredDuringSchedulingIgnoredDuringExecution with weight 100, so the scheduler *tries* for the ssd node and settles for anything if it cannot. Four replicas:

          4 cka1001-node01

    Same outcome, and that is the point worth sitting with - an identical result proves nothing about which mechanism you used. The preference was satisfiable here because node01 had room. Fill that node and the preferred version would spread to the others while the nodeSelector version would go Pending. The difference only shows up under pressure, which is exactly when you need to know which you wrote.

    That long field name is worth decoding: it applies DuringScheduling and is IgnoredDuringExecution - so a Pod already running is never moved because a label changed.

    bash Example session 
    kubectl get pods -l app=pinned -o jsonpath="{range .items[*]}{.spec.nodeName}{\"\n\"}{end}" | sort | uniq -c      3 cka1001-node01kubectl get pods -l app=affinity-demo -o jsonpath="{range .items[*]}{.spec.nodeName}{\"\n\"}{end}" | sort | uniq -c      4 cka1001-node01

    Expected resultBoth Deployments place every replica on the labelled node.

    Success conditionYou can say what would differ between the two if that node were full.

  3. A taint is the node's own refusal

    Everything so far is the Pod choosing. A taint is the other direction: the node refusing Pods that have not explicitly accepted it.

    node/cka1001-node02 tainted

    Note what the taint list shows - the control plane has carried one since installation:

    cka1001          node-role.kubernetes.io/control-plane
cka1001-node02   dedicated

    That is the real answer to "why do my Pods never run on the control plane". Not a special rule - just a taint, the same mechanism you can apply yourself.

    Now the pair this guide is named after. Two Pods, the same nodeSelector: {kubernetes.io/hostname: cka1001-node02}, differing only in that the second carries a toleration:

    intolerant   Pending   <none>
tolerant     Running   cka1001-node02

    The event on the failed one counts it out loud:

    0/4 nodes are available: 2 node(s) didn't match Pod's node affinity/selector,
2 node(s) had untolerated taint(s).

    Two rejected for the wrong hostname, two for taints - the control plane and the one just tainted. Its selector pointed at a node that would have taken it on every count except one. Asking for a node is not the same as being accepted by it.

    bash Example session 
    kubectl taint node cka1001-node02 dedicated=batch:NoSchedulenode/cka1001-node02 taintedkubectl get nodes -o custom-columns=NAME:.metadata.name,TAINTS:.spec.taints[*].keyNAME             TAINTScka1001          node-role.kubernetes.io/control-planecka1001-node01   <none>cka1001-node02   dedicatedcka1001-node03   <none>kubectl get pod intolerantNAME         READY   STATUS    RESTARTS   AGEintolerant   0/1     Pending   0          12skubectl get events --field-selector involvedObject.name=intolerant,reason=FailedScheduling | tail -2LAST SEEN   TYPE      REASON             OBJECT           MESSAGE12s         Warning   FailedScheduling   pod/intolerant   0/4 nodes are available: 2 node(s) didn't match Pod's node affinity/selector, 2 node(s) had untolerated taint(s). no new claims to deallocate, preemption: 0/4 nodes are available: 4 Preemption is not helpful for scheduling.kubectl wait --for=condition=Ready pod/tolerant --timeout=180spod/tolerant condition metkubectl get pods intolerant tolerant -o custom-columns=NAME:.metadata.name,STATUS:.status.phase,NODE:.spec.nodeNameNAME         STATUS    NODEintolerant   Pending   <none>tolerant     Running   cka1001-node02

    Expected resultIdentical selectors, one Pod Pending and one Running, with the taint named in the event.

    Success conditionYou can explain the Pending Pod without looking at its image or its logs.

  4. Priority decides who gets evicted when there is no room

    The last piece KCNA expects you to recognise. A PriorityClass is a named number, and a Pod that references one is scheduled ahead of lower-priority Pods - and can cause them to be evicted to make room:

    NAME        VALUE     GLOBAL-DEFAULT   AGE   PREEMPTIONPOLICY
demo-low    100       false            0s    PreemptLowerPriority
demo-high   1000000   false            0s    PreemptLowerPriority

    globalDefault: false on both, so nothing changes for Pods that do not ask. PreemptLowerPriority is the default policy: this priority may evict things below it. The alternative, Never, jumps the scheduling queue without evicting anyone.

    Clusters ship with their own classes - system-cluster-critical at two billion - which is how the control plane's own workloads outrank anything you create. The numbers are meaningless in isolation and only matter relative to each other, which is why picking round numbers far apart is the convention.

    bash Example session 
    kubectl get priorityclass demo-low demo-highNAME        VALUE     GLOBAL-DEFAULT   AGE   PREEMPTIONPOLICYdemo-low    100       false            0s    PreemptLowerPrioritydemo-high   1000000   false            0s    PreemptLowerPriority

    Expected resultTwo PriorityClasses, neither of them a global default.

    Success conditionYou can say what PreemptLowerPriority permits.

Troubleshooting

Official sources

← All KCNA guides