Replacing Weave Net with Calico in a Kubernetes Cluster

This guide explains how to replace Weave Net with Calico as the CNI (Container Network Interface) in a running Kubernetes cluster. Calico offers improved network policy support and better scalability.

Prerequisites

• Administrative access to the Kubernetes cluster

• kubectl configured and authenticated

• Backup of all critical workloads and configurations (changing CNI can disrupt networking)

Step-by-Step Instructions

1. Verify Weave Net is in Use

Check that Weave Net is currently running:

bash

kubectl get pods -n kube-system | grep weave

You should see Weave Net pods listed.

2. Drain and Cordon Nodes

Prevent scheduling during migration:

bash

kubectl get nodes -o name | xargs -I {} kubectl cordon {}

Then drain each node:

bash

kubectl drain <node-name> --ignore-daemonsets --delete-emptydir-data

Replace <node-name> with your actual node names.

3. Remove Weave Net

Delete the Weave Net resources:

bash

kubectl delete -f https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s.yaml

Or, if using a custom manifest:

bash

kubectl delete -f <weave-net-manifest.yaml>

Confirm removal:

bash

kubectl get pods -n kube-system | grep weave

4. Clean Up Weave Net CNI Configuration

On each node, SSH in and run:

bash

rm -f /etc/cni/net.d/10-weave.conf 
rm -f /opt/cni/bin/weave* 
ip link delete weave 
systemctl restart kubelet

5. Install Calico

Install Calico using the official manifest:

bash

kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml

For custom settings (e.g., BGP or CIDR changes), download and edit the manifest:

bash

curl -O https://docs.projectcalico.org/manifests/calico.yaml 
# Edit the file as needed 
kubectl apply -f calico.yaml

6. Verify Calico Installation

Ensure Calico pods are running:

bash

kubectl get pods -n kube-system | grep calico

Confirm Calico CNI configuration exists:

bash

ls /etc/cni/net.d/

You should see something like 10-calico.conflist.

7. Uncordon Nodes

Re-enable scheduling:

bash

kubectl get nodes -o name | xargs -I {} kubectl uncordon {}

8. Restart Workloads

Recreate pods to use the new CNI:

bash

kubectl delete pod -n <namespace> --all

Monitor progress:

bash

kubectl get pods -A

9. Verify Networking

Run basic connectivity tests:

bash

kubectl run -i --tty --rm test-pod --image=busybox -- sh 
# Inside the pod 
ping <other-pod-ip>

Check Calico network policies (if configured):

bash

kubectl get networkpolicies -A

Optional: Configure Calico Network Policies

Example: Allow traffic only from specific pods.

yaml

apiVersion: networking.k8s.io/v1 
kind: NetworkPolicy 
metadata: 
  name: allow-specific 
  namespace: default 
spec: 
  podSelector: 
    matchLabels: 
      app: my-app 
  policyTypes: - 
    - Ingress 
  ingress: 
    - from: 
      - podSelector: 
        matchLabels: 
          app: allowed-app 
    ports: 
      - protocol: TCP 
        port: 80

Apply it:

bash

kubectl apply -f <policy.yaml>

Notes

• Downtime: Expect brief networking disruptions. Perform during a maintenance window.

• IP Pool: Calico’s CALICO_IPV4POOL_CIDR should match your cluster's pod subnet.

Check it:

bash

kubectl get cm kubeadm-config -n kube-system -o yaml | grep podSubnet

• Custom Config: Modify the Calico manifest if your cluster uses BGP, custom MTU, etc.

• Rollback: Reapply the Weave Net manifest and repeat cleanup if needed.

Troubleshooting

Pods Stuck in Terminating

bash

kubectl delete pod <pod-name> -n <namespace> --force --grace-period=0

Networking Issues

Check Calico logs:

bash

kubectl logs -n kube-system -l k8s-app=calico-node

CNI Conflicts

Ensure all Weave Net files are removed from /etc/cni/net.d/.

Resources

• Calico Documentation

• Weave Net GitHub

Switch to the calico operator:

https://docs.tigera.io/calico/latest/getting-started/kubernetes/self-managed-onprem/onpremises

Install k8s on windows and join it to the cluster, then enable calico:

https://docs.tigera.io/calico/latest/getting-started/kubernetes/windows-calico/operator

Get token:

kubeadm token list

Get discovery hash:

openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -
outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'