Two Kubernetes mechanisms run a pod “on a node” without you scheduling it explicitly, and they are constantly confused: static pods and DaemonSets. Both produce pods that appear on specific nodes, but they are managed by completely different components and serve different purposes. Static pods are how kubeadm runs the control plane itself; DaemonSets are how you run agents (logging, monitoring, networking) on every node. This tutorial explains both, shows how to create, inspect and update each, and gives a clear decision rule.

Prerequisites: a cluster built with kubeadm (see Install Kubernetes with kubeadm), SSH access to at least one node, and basic Pod knowledge. Tested on Kubernetes 1.33.

Side-by-side comparison

	Static pod	DaemonSet
Managed by	The kubelet on one node, from a manifest file on disk	The DaemonSet controller in kube-controller-manager, through the API server
Defined where	/etc/kubernetes/manifests/*.yaml on the node	A DaemonSet object in the API (kubectl apply)
Scope	Exactly that node	Every node (or a subset chosen by selectors/tolerations)
Needs the API server	No – works even if the control plane is down	Yes
Visible with kubectl	Yes, as a read-only mirror pod named <name>-<node>	Yes, normal pods
kubectl delete	Pod comes back immediately (kubelet recreates it)	Pod comes back (controller recreates it)
Updates	Edit the file; kubelet restarts the pod	RollingUpdate or OnDelete strategy
Typical use	kube-apiserver, etcd, controller-manager, scheduler	kube-proxy, CNI agents, node exporters, log shippers, storage drivers

Part 1 – Static pods

Where they come from

Every kubelet watches a directory defined by staticPodPath in its configuration (kubeadm sets /etc/kubernetes/manifests). Any pod manifest placed there is started by the kubelet directly, with no scheduler involved. This solves the bootstrapping paradox: the API server cannot schedule itself, so kubeadm writes it as a static pod.

# On a control-plane node
grep staticPodPath /var/lib/kubelet/config.yaml
# staticPodPath: /etc/kubernetes/manifests

ls /etc/kubernetes/manifests/
# etcd.yaml  kube-apiserver.yaml  kube-controller-manager.yaml  kube-scheduler.yaml

# From any machine with kubectl: mirror pods carry the node name as suffix
kubectl get pods -n kube-system -o wide | grep -E 'apiserver|etcd'
# kube-apiserver-cp1   1/1   Running   ...   cp1
# etcd-cp1             1/1   Running   ...   cp1

Create your own static pod

SSH to a worker node and drop a manifest into the directory. Nothing else is needed.

ssh w1
sudo tee /etc/kubernetes/manifests/node-banner.yaml > /dev/null <<'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: node-banner
  namespace: default
  labels:
    app: node-banner
spec:
  containers:
    - name: web
      image: nginx:1.27-alpine
      ports:
        - containerPort: 80
          hostPort: 8080          # reachable on the node's IP
      resources:
        limits:
          memory: 64Mi
EOF

# The kubelet notices the file within seconds
sudo crictl ps | grep node-banner
curl -s http://localhost:8080 | head -3

# Back on your workstation
kubectl get pods -o wide
# NAME              READY   STATUS    NODE
# node-banner-w1    1/1     Running   w1

# Deleting the mirror pod does nothing lasting
kubectl delete pod node-banner-w1
kubectl get pods            # node-banner-w1 is back

# The only way to remove it is to remove the file on the node
ssh w1 sudo rm /etc/kubernetes/manifests/node-banner.yaml

Try to kubectl edit the mirror pod: the API refuses meaningful changes, because the source of truth is the file. To change a static pod you edit the manifest; the kubelet detects the change and recreates the container.

Exam tip (CKA): when asked to “create a pod that survives even if the control plane is unavailable” or to troubleshoot a broken kube-apiserver, think static pods and look in /etc/kubernetes/manifests. A typo in kube-apiserver.yaml makes kubectl stop working entirely; fix the file and watch crictl ps / journalctl -u kubelet.

Part 2 – DaemonSets

One pod per node, managed by the API

A DaemonSet guarantees that a copy of a pod runs on every eligible node. When a node joins, the controller adds a pod; when it leaves, the pod is garbage-collected. Since Kubernetes 1.12 the default scheduler places DaemonSet pods (through node affinity), so they respect resources, taints and priorities like any other pod.

# node-exporter.yaml – a typical monitoring agent
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter
  namespace: monitoring
  labels:
    app: node-exporter
spec:
  selector:
    matchLabels:
      app: node-exporter
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
  template:
    metadata:
      labels:
        app: node-exporter
    spec:
      hostNetwork: true
      hostPID: true
      tolerations:
        - key: node-role.kubernetes.io/control-plane   # also run on control-plane nodes
          operator: Exists
          effect: NoSchedule
      containers:
        - name: node-exporter
          image: quay.io/prometheus/node-exporter:v1.9.1
          args:
            - --path.rootfs=/host
          ports:
            - containerPort: 9100
              name: metrics
          volumeMounts:
            - name: rootfs
              mountPath: /host
              readOnly: true
          resources:
            requests: { cpu: 50m, memory: 64Mi }
            limits:   { memory: 128Mi }
      volumes:
        - name: rootfs
          hostPath:
            path: /

kubectl create namespace monitoring
kubectl apply -f node-exporter.yaml

kubectl get daemonset -n monitoring
# NAME            DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
# node-exporter   3         3         3       3            3           <none>          20s

kubectl get pods -n monitoring -o wide     # one pod on cp1, w1, w2
curl -s http://192.168.56.11:9100/metrics | head -5

Running on a subset of nodes

Use nodeSelector or node affinity to limit where the DaemonSet runs, and tolerations to allow it onto tainted nodes (control plane, GPU nodes, spot nodes).

spec:
  template:
    spec:
      nodeSelector:
        disktype: ssd                      # only nodes labelled disktype=ssd
      tolerations:
        - key: dedicated
          operator: Equal
          value: gpu
          effect: NoSchedule

kubectl label node w2 disktype=ssd
kubectl get pods -n monitoring -o wide     # pod only on w2 after the change
kubectl label node w2 disktype-             # remove label → pod is evicted

Updating a DaemonSet

kubectl set image daemonset/node-exporter -n monitoring node-exporter=quay.io/prometheus/node-exporter:v1.9.2
kubectl rollout status daemonset/node-exporter -n monitoring
kubectl rollout history daemonset/node-exporter -n monitoring
kubectl rollout undo daemonset/node-exporter -n monitoring     # roll back if needed

With RollingUpdate (default), pods are replaced node by node, respecting maxUnavailable (or maxSurge if you allow two copies briefly). With OnDelete, nothing happens until you delete each old pod yourself, which is useful for storage drivers where you want to control the order.

Why kube-proxy is a DaemonSet but kube-apiserver is a static pod

Run kubectl get ds -n kube-system and you will see kube-proxy and the Calico node agent: they need the API to exist, run on every node and benefit from rolling updates. The control-plane components, in contrast, must start before the API exists and must keep running even if the API is broken. Each mechanism sits exactly where its dependency allows.

Decision rule

• You are bootstrapping or repairing the control plane, or need something that must run without the API → static pod.
• You need an agent on every node (or every node of a type), with declarative management and rolling updates → DaemonSet.
• You need N replicas anywhere in the cluster → neither; use a Deployment.
• In application code, you should almost never create static pods; treat them as a kubelet/kubeadm implementation detail.

Troubleshooting

• Static pod file ignored – wrong directory, invalid YAML, or a missing namespace. Check journalctl -u kubelet | grep -i static.
• DaemonSet shows DESIRED 2 on a 3-node cluster – one node is tainted (usually the control plane); add a toleration or accept it.
• DaemonSet pods Pending – insufficient resources on the node, or a hostPort already in use by a static pod or another DaemonSet.
• Rolling update stuck – maxUnavailable too small combined with a pod that never becomes Ready; inspect kubectl describe pod.

Clean up

kubectl delete -f node-exporter.yaml
kubectl delete namespace monitoring

Key takeaways

• Static pods are files in /etc/kubernetes/manifests, run by the kubelet, shown as mirror pods, and used for the control plane.
• DaemonSets are API objects that keep one pod per (selected) node with rolling updates.
• Deleting a pod of either kind is pointless; change the file or the DaemonSet spec.
• Tolerations and nodeSelectors decide which nodes a DaemonSet covers.

Next tutorial

Next: ConfigMaps and Secrets. Official docs: Create static Pods, DaemonSet.