CertGrid CertGrid
Hands-on Lab·Certified Kubernetes Security Specialist

NetworkPolicy for Cluster-Level Access Control

The first competency in the first domain, and the task most likely to appear in front of you: take a namespace that can reach everything and leave it able to reach one service. This guide does it in four steps and measures the cluster after each one, including the step where everything breaks.

Cluster Setup Guide 4 of 40 Intermediate

Written against the versions above. Enforcement is the CNI's job - these clusters run Cilium 1.18.1. A NetworkPolicy on a cluster whose CNI ignores them is accepted by the API and does nothing at all.

The cka6001 cluster: one control plane and 2 schedulable workers, on Cilium.
Server NameIP AddressOSRolesCPURAMHDD
CKA6001192.168.0.46Ubuntu 26.04 LTSControl Plane Node (tainted NoSchedule)2 Core4 GB50 GB
CKA6001-NODE01192.168.0.47Ubuntu 26.04 LTSWorker Node2 Core4 GB50 GB
CKA6001-NODE02192.168.0.48Ubuntu 26.04 LTSWorker Node2 Core4 GB50 GB

Before you start

  1. The baseline, before anything is restricted

    A client in one namespace, a service in another, and no policy anywhere.

    bash Example session
    kubectl create ns cks-app >/dev/null 2>&1; kubectl create ns cks-client >/dev/null 2>&1; kubectl -n cks-app run web --image=docker.io/library/busybox:1.37 --labels=app=web --restart=Never --command -- sh -c 'echo APP-BACKEND-OK > /tmp/index.html; httpd -f -h /tmp -p 8080' >/dev/null; kubectl -n cks-app expose pod web --port=8080 --name=web >/dev/null; kubectl -n cks-client run client --image=docker.io/library/busybox:1.37 --restart=Never --command -- sh -c 'sleep 900' >/dev/null; kubectl -n cks-app wait --for=condition=Ready pod/web --timeout=150s >/dev/null; kubectl -n cks-client wait --for=condition=Ready pod/client --timeout=150s >/dev/null; echo "--- both up, and nothing is restricting anything yet"--- both up, and nothing is restricting anything yetecho -n "DNS:  "; kubectl -n cks-client exec client -- sh -c 'nslookup web.cks-app.svc.cluster.local 2>&1 | grep -E "^Address|timed out" | tail -1'; echo -n "HTTP: "; kubectl -n cks-client exec client -- sh -c 'if wget -O /tmp/o -T 6 http://web.cks-app.svc.cluster.local:8080/ 2>/tmp/e; then cat /tmp/o; else grep -m1 "wget:" /tmp/e; fi'; echo "--- the baseline: the name resolves to the Service ClusterIP, and the request is answered. Every step below is measured against these two lines"DNS:  Address: 10.111.167.47HTTP: APP-BACKEND-OK--- the baseline: the name resolves to the Service ClusterIP, and the request is answered. Every step below is measured against these two lines

    Expected resultDNS: Address: 10.111.167.47 and HTTP: APP-BACKEND-OK.

    Success conditionYou have two lines to measure every later step against.

  2. Default-deny, and the thing it breaks first

    The four-line policy that closes a namespace, and the immediate consequence.

    bash Example session
    printf 'apiVersion: networking.k8s.io/v1\nkind: NetworkPolicy\nmetadata:\n  name: default-deny\n  namespace: cks-client\nspec:\n  podSelector: {}\n  policyTypes: ["Ingress","Egress"]\n' | kubectl apply -f - 2>&1 | tail -1; sleep 8; echo -n "DNS:  "; kubectl -n cks-client exec client -- sh -c 'nslookup web.cks-app.svc.cluster.local 2>&1 | grep -E "^Address|timed out" | tail -1'; echo "--- DNS is the first casualty of an egress deny, and it is the mistake everyone makes once"networkpolicy.networking.k8s.io/default-deny createdDNS:  ;; connection timed out; no servers could be reached--- DNS is the first casualty of an egress deny, and it is the mistake everyone makes onceecho -n "HTTP straight to the Pod IP: "; W=$(kubectl -n cks-app get pod web -o jsonpath='{.status.podIP}'); kubectl -n cks-client exec client -- sh -c "if wget -O /tmp/o -T 6 http://$W:8080/ 2>/tmp/e; then cat /tmp/o; else grep -m1 'wget:' /tmp/e; fi"; echo "--- and the connection itself is denied too, so this is not only a name-resolution problem"HTTP straight to the Pod IP: wget: download timed out--- and the connection itself is denied too, so this is not only a name-resolution problem

    Expected resultDNS: ;; connection timed out; no servers could be reached, and the direct request to the Pod IP returns wget: download timed out.

    Success conditionYou know exactly what a closed namespace looks like from inside it.

  3. Allow DNS, and only DNS

    The rule every default-deny needs next.

    bash Example session
    printf 'apiVersion: networking.k8s.io/v1\nkind: NetworkPolicy\nmetadata:\n  name: allow-dns\n  namespace: cks-client\nspec:\n  podSelector: {}\n  policyTypes: ["Egress"]\n  egress:\n  - to:\n    - namespaceSelector:\n        matchLabels:\n          kubernetes.io/metadata.name: kube-system\n      podSelector:\n        matchLabels:\n          k8s-app: kube-dns\n    ports:\n    - {protocol: UDP, port: 53}\n    - {protocol: TCP, port: 53}\n' | kubectl apply -f - 2>&1 | tail -1; sleep 8; echo -n "DNS:  "; kubectl -n cks-client exec client -- sh -c 'nslookup web.cks-app.svc.cluster.local 2>&1 | grep -E "^Address|timed out" | tail -1'; echo -n "HTTP: "; kubectl -n cks-client exec client -- sh -c 'if wget -O /tmp/o -T 6 http://web.cks-app.svc.cluster.local:8080/ 2>/tmp/e; then cat /tmp/o; else grep -m1 "wget:" /tmp/e; fi'; echo "--- the name resolves again and the request still fails. Two separate permissions, and DNS is the one nobody remembers"networkpolicy.networking.k8s.io/allow-dns createdDNS:  Address: 10.111.167.47HTTP: wget: download timed out--- the name resolves again and the request still fails. Two separate permissions, and DNS is the one nobody remembers

    Expected resultThe name resolves again - and HTTP: wget: download timed out still.

    Success conditionYou can restore name resolution without opening anything else.

  4. Allow the one thing this client is for

    One more rule, scoped by label and port.

    bash Example session
    printf 'apiVersion: networking.k8s.io/v1\nkind: NetworkPolicy\nmetadata:\n  name: allow-web\n  namespace: cks-client\nspec:\n  podSelector: {}\n  policyTypes: ["Egress"]\n  egress:\n  - to:\n    - namespaceSelector:\n        matchLabels:\n          kubernetes.io/metadata.name: cks-app\n      podSelector:\n        matchLabels:\n          app: web\n    ports:\n    - {protocol: TCP, port: 8080}\n' | kubectl apply -f - 2>&1 | tail -1; sleep 8; echo -n "HTTP: "; kubectl -n cks-client exec client -- sh -c 'if wget -O /tmp/o -T 6 http://web.cks-app.svc.cluster.local:8080/ 2>/tmp/e; then cat /tmp/o; else grep -m1 "wget:" /tmp/e; fi'; echo "--- allowed, by label, to exactly one workload on exactly one port"networkpolicy.networking.k8s.io/allow-web createdHTTP: APP-BACKEND-OK--- allowed, by label, to exactly one workload on exactly one portkubectl -n cks-client get networkpolicy --no-headers; echo "--- three objects, and the rules are ADDITIVE: a connection is allowed if ANY policy allows it, and denied if none does"allow-dns      <none>   23sallow-web      <none>   8sdefault-deny   <none>   42s--- three objects, and the rules are ADDITIVE: a connection is allowed if ANY policy allows it, and denied if none does

    Expected resultHTTP: APP-BACKEND-OK, and three policies in the namespace - allow-dns, allow-web, default-deny.

    Success conditionA namespace that can reach DNS and exactly one workload.

  5. The ingress half, and the assumption it breaks

    Protect the service itself, then test from two directions.

    bash Example session
    printf 'apiVersion: networking.k8s.io/v1\nkind: NetworkPolicy\nmetadata:\n  name: web-ingress-from-client-ns\n  namespace: cks-app\nspec:\n  podSelector:\n    matchLabels:\n      app: web\n  policyTypes: ["Ingress"]\n  ingress:\n  - from:\n    - namespaceSelector:\n        matchLabels:\n          kubernetes.io/metadata.name: cks-client\n    ports:\n    - {protocol: TCP, port: 8080}\n' | kubectl apply -f - 2>&1 | tail -1; sleep 8; echo -n "from cks-client:     "; kubectl -n cks-client exec client -- sh -c 'if wget -O /tmp/o -T 6 http://web.cks-app.svc.cluster.local:8080/ 2>/tmp/e; then cat /tmp/o; else grep -m1 "wget:" /tmp/e; fi'networkpolicy.networking.k8s.io/web-ingress-from-client-ns createdfrom cks-client:     APP-BACKEND-OKkubectl -n cks-app run nosy --image=docker.io/library/busybox:1.37 --restart=Never --command -- sh -c 'sleep 300' >/dev/null; kubectl -n cks-app wait --for=condition=Ready pod/nosy --timeout=150s >/dev/null; W=$(kubectl -n cks-app get pod web -o jsonpath='{.status.podIP}'); echo -n "from inside cks-app: "; kubectl -n cks-app exec nosy -- sh -c "if wget -O /tmp/o -T 6 http://$W:8080/ 2>/tmp/e; then cat /tmp/o; else grep -m1 'wget:' /tmp/e; fi"; echo "--- the client namespace is allowed and a Pod in the SAME namespace is not. An ingress rule does not treat local traffic as trusted"from inside cks-app: wget: download timed out--- the client namespace is allowed and a Pod in the SAME namespace is not. An ingress rule does not treat local traffic as trusted

    Expected resultfrom cks-client: APP-BACKEND-OK and from inside cks-app: wget: download timed out.

    Success conditionYou can restrict a workload without depending on its clients behaving.

Troubleshooting

Official sources