HowToCodePage

Scheduling: affinity, taints and tolerations

Steer pods onto the right nodes with selectors, affinity, taints and topology spread, and take a node out of service without dropping traffic.

How the scheduler chooses

The scheduler runs two phases: filtering, which removes nodes that cannot run the pod, and scoring, which ranks the survivors. Requests, node selectors, affinity and taints all participate in filtering; spread and balance influence scoring.

spec:
  nodeSelector:                      # simplest: exact label match
    disktype: ssd
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:   # hard rule
        nodeSelectorTerms:
          - matchExpressions:
              - key: topology.kubernetes.io/zone
                operator: In
                values: ["eu-west-1a", "eu-west-1b"]
      preferredDuringSchedulingIgnoredDuringExecution:  # soft rule
        - weight: 80
          preference:
            matchExpressions:
              - key: node-pool
                operator: In
                values: ["general"]
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          podAffinityTerm:
            labelSelector:
              matchLabels:
                app: api
            topologyKey: kubernetes.io/hostname   # spread api pods across nodes
  containers:
    - name: api
      image: ghcr.io/example/api:2.1.0
MechanismMeaningBehaviour when unmet
nodeSelectorExact label matchPod stays Pending
Node affinity, requiredExpression over node labelsPod stays Pending
Node affinity, preferredScored preferenceIgnored if nothing matches
Pod affinityCo-locate with labeled podsPending if required
Pod anti-affinityAvoid labeled pods in a topologyPending if required and unavoidable
Taint and tolerationNode repels pods that do not tolerate itPod is not scheduled there

Required pod anti-affinity is the classic self-inflicted Pending pod: with three replicas and a two-node cluster, the third has nowhere to go. Use preferred unless you genuinely need the guarantee.

Taints and tolerations

A taint marks a node as reserved, and only pods with a matching toleration are allowed there. Taints keep workloads off nodes; affinity pulls workloads onto them. Using both gives you a pool that is reserved and correctly targeted.

kubectl taint nodes node-1 dedicated=gpu:NoSchedule
kubectl taint nodes node-1 dedicated=gpu:NoSchedule-     # remove
kubectl describe node node-1 | grep -A4 Taints
kubectl label nodes node-1 node-pool=gpu
spec:
  nodeSelector:
    node-pool: gpu
  tolerations:
    - key: dedicated
      operator: Equal
      value: gpu
      effect: NoSchedule
  containers:
    - name: trainer
      image: ghcr.io/example/trainer:0.9.1
EffectMeaning
NoScheduleNew pods without a toleration are not placed here; existing pods stay
PreferNoScheduleThe scheduler tries to avoid the node but may use it
NoExecuteExisting pods without a toleration are evicted; tolerationSeconds delays it
  • operator: Exists with no key tolerates every taint on the node — useful for a DaemonSet, dangerous for an application.
  • Kubernetes adds built-in taints such as node.kubernetes.io/not-ready and node.kubernetes.io/unreachable, which is what makes pods reschedule after a node failure.
  • A taint does not attract anything. Without the matching node selector or affinity, a tolerating pod may still land on an ordinary node.
  • Adding a taint does not evict running pods unless the effect is NoExecute.

Topology spread, priority and node maintenance

Hard anti-affinity guarantees at most one pod per node. Topology spread is the more flexible tool: it distributes pods evenly across zones or nodes and tolerates imbalance within a limit.

spec:
  topologySpreadConstraints:
    - maxSkew: 1
      topologyKey: topology.kubernetes.io/zone
      whenUnsatisfiable: DoNotSchedule     # or ScheduleAnyway
      labelSelector:
        matchLabels:
          app: api
  priorityClassName: business-critical
---
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: business-critical
value: 100000
preemptionPolicy: PreemptLowerPriority
description: "Customer-facing services; may evict batch work"
kubectl cordon node-1                       # stop new pods being scheduled
kubectl drain node-1 --ignore-daemonsets --delete-emptydir-data
kubectl uncordon node-1
kubectl get pods -A -o wide --field-selector spec.nodeName=node-1
kubectl get pdb -A                           # what drain must respect
  • maxSkew: 1 with DoNotSchedule means no zone may have more than one extra pod. It is stricter than it sounds: with two zones and three replicas the third pod can be unschedulable.
  • minDomains prevents a low-zone count from making the constraint trivially satisfiable.
  • Priority classes decide who survives pressure. A high value can preempt lower-priority pods, which is powerful and worth reviewing deliberately.
  • drain evicts pods one at a time and waits for PodDisruptionBudgets. Without a PDB it will happily take every replica of a service down together.
  • --ignore-daemonsets is required because DaemonSet pods are not evictable, and --delete-emptydir-data acknowledges that scratch data will be lost.
⚠️
A drain is a controlled outage only if the workloads can move. If every replica requires a persistent volume that exists in one zone, or if no PDB protects a single-replica Deployment, cordon and drain convert maintenance into an incident. Check the PDBs and the volume topology first.

FAQ

My pod is Pending with '0/5 nodes are available'. How do I read that?
The event lists every node with its rejection reason: insufficient CPU or memory, taint not tolerated, node selector unmatched, or affinity rules unsatisfied. That single line is the whole diagnosis.
Taints or node affinity?
Use both. The taint reserves the node so nothing unsuited lands there by accident, and the affinity makes sure the intended pods actually choose it rather than drifting to a general pool.

Resource requests, limits and autoscaling Labels, selectors, namespaces and annotations

Last refreshed 2026-09-18.