Kubernetes Network Policies for Zero Trust Security

When I first deployed default-deny network policies across my homelab cluster, I immediately broke everything. The API could not reach PostgreSQL, the workers could not pull from NATS, and even DNS resolution stopped working because I had forgotten to allow egress to kube-system on port 53. It took me an entire evening with kubectl logs and Hubble flow traces to understand the cascade of failures. That painful experience taught me to always deploy allow policies before deny policies, and to keep a validation script handy. This article is the playbook I built from those hard-won lessons.

Introduction

In a flat network, a compromised pod can attack any other pod in the cluster. Network Policies act as a firewall for pods, enabling a Zero Trust architecture where communication is denied by default and explicitly allowed only where necessary. This guide demonstrates how to implement a defense-in-depth strategy using Kubernetes Network Policies, ensuring that services can only communicate with their valid dependencies.

[Network Policies] — Kubernetes , 2024-03-01

Architecture Overview

Network Policies enforce a whitelist model. All traffic is blocked unless explicitly allowed.

flowchart TB
    Traefik["🌐 Ingress<br/>(Traefik)"] --> Web["💻 Web"]
    Web --> API["🛡️ API"]
    
    API --> DB[("🗄️ Database")]
    API --> Cache[("🧊 Redis")]
    
    subgraph Attack["🚫 Blocked Path"]
        Attacker["😈 Compromised<br/>Web Pod"] -.-> DB
    end

    classDef primary fill:#7c3aed,color:#fff
    classDef secondary fill:#06b6d4,color:#fff
    classDef db fill:#f43f5e,color:#fff
    classDef warning fill:#fbbf24,color:#000

    class Web,API primary
    class Traefik secondary
    class DB,Cache db
    class Attacker warning

Implementation

Default Deny Policies

Deny All Ingress

# core-infra/security/default-deny-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
  namespace: myapp-staging
spec:
  podSelector: {}
  policyTypes:
    - Ingress

Deny All Egress

# core-infra/security/default-deny-egress.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-egress
  namespace: myapp-staging
spec:
  podSelector: {}
  policyTypes:
    - Egress

Combined Default Deny

# core-infra/security/default-deny-all.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: myapp-staging
spec:
  podSelector: {}
  policyTypes:
    - Ingress
    - Egress

[NIST SP 800-207: Zero Trust Architecture] — Scott Rose et al. , 2020-08-11

Allow Policies

API to Database

# apps/myapp-api/network-policy.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: myapp-api-allow
  namespace: myapp-staging
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: myapp-api
  policyTypes:
    - Ingress
    - Egress
  ingress:
    # Allow from Traefik ingress
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: traefik-system
          podSelector:
            matchLabels:
              app.kubernetes.io/name: traefik
      ports:
        - protocol: TCP
          port: 8080
  egress:
    # Allow to PostgreSQL
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: data-layer
          podSelector:
            matchLabels:
              app.kubernetes.io/name: postgresql
      ports:
        - protocol: TCP
          port: 5432
    # Allow to NATS
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: data-layer
          podSelector:
            matchLabels:
              app.kubernetes.io/name: nats
      ports:
        - protocol: TCP
          port: 4222
    # Allow DNS
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53

Worker to Services

# apps/myapp-workers/network-policy.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: myapp-workers-allow
  namespace: myapp-staging
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: myapp-workers
  policyTypes:
    - Egress
  egress:
    # PostgreSQL
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: data-layer
          podSelector:
            matchLabels:
              app.kubernetes.io/name: postgresql
      ports:
        - protocol: TCP
          port: 5432
    # NATS
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: data-layer
          podSelector:
            matchLabels:
              app.kubernetes.io/name: nats
      ports:
        - protocol: TCP
          port: 4222
    # Qdrant
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: data-layer
          podSelector:
            matchLabels:
              app.kubernetes.io/name: qdrant
      ports:
        - protocol: TCP
          port: 6333
        - protocol: TCP
          port: 6334
    # MinIO
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: data-layer
          podSelector:
            matchLabels:
              app.kubernetes.io/name: minio
      ports:
        - protocol: TCP
          port: 9000
    # Ollama AI
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: ai
          podSelector:
            matchLabels:
              app.kubernetes.io/name: ollama
      ports:
        - protocol: TCP
          port: 11434
    # DNS
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
      ports:
        - protocol: UDP
          port: 53

Web to API

# apps/my-web-client/network-policy.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: my-web-client-allow
  namespace: myapp-staging
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: my-web-client
  policyTypes:
    - Ingress
    - Egress
  ingress:
    # Allow from Traefik
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: traefik-system
      ports:
        - protocol: TCP
          port: 8080
  egress:
    # Allow to API
    - to:
        - podSelector:
            matchLabels:
              app.kubernetes.io/name: myapp-api
      ports:
        - protocol: TCP
          port: 8080
    # Allow to NATS (real-time updates)
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: data-layer
          podSelector:
            matchLabels:
              app.kubernetes.io/name: nats
      ports:
        - protocol: TCP
          port: 4222
    # DNS
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
      ports:
        - protocol: UDP
          port: 53

Database Layer Policies

PostgreSQL Access Control

# infrastructure/data-layer/postgresql-network-policy.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: postgresql-allow
  namespace: data-layer
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: postgresql
  policyTypes:
    - Ingress
  ingress:
    # Allow from staging API and workers
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: myapp-staging
          podSelector:
            matchExpressions:
              - key: app.kubernetes.io/name
                operator: In
                values:
                  - myapp-api
                  - myapp-workers
                  - my-web-client
      ports:
        - protocol: TCP
          port: 5432
    # Allow from production
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: myapp-prod
          podSelector:
            matchExpressions:
              - key: app.kubernetes.io/name
                operator: In
                values:
                  - myapp-api
                  - myapp-workers
                  - my-web-client
      ports:
        - protocol: TCP
          port: 5432
    # Allow from CNPG operator
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: cnpg-system
      ports:
        - protocol: TCP
          port: 5432

NATS Access Control

# infrastructure/data-layer/nats-network-policy.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: nats-allow
  namespace: data-layer
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: nats
  policyTypes:
    - Ingress
  ingress:
    # Client connections
    - from:
        - namespaceSelector:
            matchExpressions:
              - key: kubernetes.io/metadata.name
                operator: In
                values:
                  - myapp-staging
                  - myapp-prod
      ports:
        - protocol: TCP
          port: 4222
    # Cluster communication (between NATS pods)
    - from:
        - podSelector:
            matchLabels:
              app.kubernetes.io/name: nats
      ports:
        - protocol: TCP
          port: 6222
    # Leafnode connections
    - from:
        - podSelector:
            matchLabels:
              app.kubernetes.io/name: nats
      ports:
        - protocol: TCP
          port: 7422

CIDR-Based Rules

External API Access

# Allow egress to external APIs
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-external-apis
  namespace: myapp-staging
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: myapp-workers
  policyTypes:
    - Egress
  egress:
    # Allow HTTPS to external APIs
    - to:
        - ipBlock:
            cidr: 0.0.0.0/0
            except:
              - 10.0.0.0/8
              - 172.16.0.0/12
              - 192.168.0.0/16
      ports:
        - protocol: TCP
          port: 443

Cilium Enhanced Policies

[Cilium Network Policy Documentation] — Cilium , 2024-05-15

Layer 7 HTTP Policy

# Cilium L7 policy for API path filtering
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: myapp-api-l7
  namespace: myapp-staging
spec:
  endpointSelector:
    matchLabels:
      app.kubernetes.io/name: myapp-api
  ingress:
    - fromEndpoints:
        - matchLabels:
            app.kubernetes.io/name: my-web-client
      toPorts:
        - ports:
            - port: "8080"
              protocol: TCP
          rules:
            http:
              - method: "GET"
                path: "/api/documents.*"
              - method: "POST"
                path: "/api/documents"
              - method: "GET"
                path: "/health.*"

DNS Policy

# Cilium DNS egress policy
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-specific-dns
  namespace: myapp-staging
spec:
  endpointSelector:
    matchLabels:
      app.kubernetes.io/name: myapp-workers
  egress:
    - toEndpoints:
        - matchLabels:
            k8s:io.kubernetes.pod.namespace: kube-system
            k8s-app: kube-dns
      toPorts:
        - ports:
            - port: "53"
              protocol: UDP
          rules:
            dns:
              - matchPattern: "*.svc.cluster.local"
              - matchName: "api.openai.com"

Monitoring Network Policies

Policy Audit with Cilium

# Enable Hubble for network flow visibility
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: audit-deny
  namespace: myapp-staging
  annotations:
    io.cilium.proxy-visibility: "<Egress/80/TCP/HTTP>,<Ingress/8080/TCP/HTTP>"
spec:
  endpointSelector: {}
  enableDefaultDeny:
    ingress: true
    egress: true

Validation Script

#!/bin/bash
# Test network policy enforcement

# Test allowed connection
kubectl run test-client --rm -i --restart=Never \
  --image=busybox \
  -n myapp-staging \
  --labels="app.kubernetes.io/name=myapp-api" \
  -- wget -qO- --timeout=5 \
  http://postgresql.data-layer.svc.cluster.local:5432 || echo "Connection blocked (expected)"

# Test denied connection
kubectl run test-client --rm -i --restart=Never \
  --image=busybox \
  -n myapp-staging \
  --labels="app.kubernetes.io/name=test-pod" \
  -- wget -qO- --timeout=5 \
  http://postgresql.data-layer.svc.cluster.local:5432 && echo "Connection allowed (unexpected!)" || echo "Connection blocked (expected)"

Testing Strategy

[Kubernetes Network Policy Recipes] — Ahmet Alp Balkan , 2023-11-20

Policy Test Matrix

Source	Destination	Port	Expected
myapp-api	postgresql	5432	Allow
myapp-api	nats	4222	Allow
myapp-api	qdrant	6333	Deny
myapp-workers	qdrant	6333	Allow
my-web-client	myapp-api	8080	Allow
my-web-client	postgresql	5432	Deny
traefik	myapp-api	8080	Allow

Conclusion

Network Policies are a fundamental building block of Kubernetes security. By starting with a default-deny posture and whitelisting required traffic, you significantly reduce the attack surface of your cluster, turning it from a soft-shell target into a hardened fortress.

Implementing these policies in my own homelab felt like going from leaving the front door wide open to installing a proper lock on every room. The upfront investment in mapping out which services genuinely need to talk to each other forced me to understand my own architecture more deeply than I had before. Every time I add a new service now, the first thing I write is its network policy — it has become as natural as writing the Deployment manifest itself.

[Network Policies - Kubernetes Documentation] — Kubernetes , 2024-03-01

Next Steps

Enable Hubble UI for real-time network flow visualization and policy debugging.
Implement CiliumClusterwideNetworkPolicy for policies that span multiple namespaces.
Explore NetworkPolicy editor tools like Cilium Editor to visually design and validate policies before deployment.

Introduction

Architecture Overview

Implementation

Default Deny Policies

Deny All Ingress

Deny All Egress

Combined Default Deny

Allow Policies

API to Database

Worker to Services

Web to API

Database Layer Policies

PostgreSQL Access Control

NATS Access Control

CIDR-Based Rules

External API Access

Cilium Enhanced Policies

Layer 7 HTTP Policy

DNS Policy

Monitoring Network Policies

Policy Audit with Cilium

Validation Script

Testing Strategy

Policy Test Matrix

Conclusion

Next Steps

Further Reading