Calico Internals and Policy Enforcement

Services and Networking / Guide 64 of 103 / Advanced

• Kubernetes1.36.4
• Cluster4 nodes
• Runtimecontainerd 2.2.6
• CNICalico v3.32.1
• TimeAbout 40 min

An operator-installed Calico with VXLANCrossSubnet encapsulation. A manifest install or a BGP-mode install differs in the details, though the IPAM and routing model is the same.

• Calicov3.32.1
• EncapsulationVXLANCrossSubnet

This guide includes

Use this when Pod networking has to stop being magic - during an outage, or before you are willing to trust a policy. This matters because every piece is inspectable on the node: an IPAM block, a veth and route per Pod, and the iptables chains Felix writes.

• identifying the Calico workloads and which of them runs on every node
• finding that Calico allocates from its own IP pool rather than the node's podCIDR
• reading the CNI conflist the kubelet uses, which knows nothing about Calico
• counting one veth and one host route per Pod, and the VXLAN device CrossSubnet leaves unused
• counting the cali- chains Felix writes into iptables to enforce policy

Before you start

• The Kubernetes network model guide.
• The CNI and Pod networking guide, which covers what a CNI plugin is asked to do.
• Shell access to a node, since half of this is read with ip rather than kubectl.

1. 1

   The pieces, and which of them is on every node

2. 2

   IPAM: Calico does not use the node's podCIDR

3. 3

   What the kubelet reads, and what it runs

4. 4

   One veth and one host route per Pod

5. 5

   CrossSubnet: the VXLAN device that is not being used

6. 6

   Felix, and the filter it writes

Official sources

• Calico documentation
• Calico IP address management
• Calico overlay networking
• Configure MTU
• Network plugins