Kustomize Overlays for Multi-Environment Deployments

When I first started managing multiple environments in my homelab cluster, I made the mistake that everyone warns you about: I copied entire YAML files for staging and production, then tried to keep them in sync manually. Within a week, a resource limit change I made in staging never made it to prod, and my API pod got OOM-killed during a load test. Kustomize overlays solved this by letting me define a single base and override only what differs per environment. The initial restructuring of my repository took an evening, but it paid for itself the very next time I needed to add a new environment variable across all environments with a single edit to the base.

Introduction

Managing Kubernetes configurations across multiple environments (Dev, Staging, Prod) can quickly become a maintenance nightmare if you’re duplicating YAML files. Kustomize offers a template-free way to customize application configuration, preserving the base structure while allowing for environment-specific “overlays” to modify behavior. This guide details how to structure a multi-environment deployment using standard Kustomize patterns.

[Kustomize - Kubernetes native configuration management] — Kubernetes SIG CLI , 2024-07-01

Architecture Overview

The Kustomize workflow separates the “Base” definition (shared across all environments) from “Overlays” (environment-specific modifications). When built, Kustomize merges the layers to produce the final manifests.

flowchart LR
    Base["📄 Base Configuration<br/>(Deployment, Service)"] --> Build("⚙️ Kustomize Build")
    Overlaydev["📝 Dev Overlay<br/>(Patches, Config)"] --> Build
    Overlayprod["📝 Prod Overlay<br/>(Patches, HPA)"] --> Build
    Build --> Output["📦 Rendered Manifests"]

    classDef primary fill:#7c3aed,color:#fff
    classDef secondary fill:#06b6d4,color:#fff
    classDef db fill:#f43f5e,color:#fff
    classDef warning fill:#fbbf24,color:#000

    class Build primary
    class Overlaydev,Overlayprod secondary
    class Base,Output db

Implementation

Directory Structure

apps/
└── myapp-api/
    ├── base/
    │   ├── kustomization.yaml
    │   ├── deployment.yaml
    │   ├── service.yaml
    │   └── serviceaccount.yaml
    └── overlays/
        ├── dev/
        │   ├── kustomization.yaml
        │   └── deployment-patch.yaml
        ├── staging/
        │   ├── kustomization.yaml
        │   ├── deployment-patch.yaml
        │   └── hpa.yaml
        └── prod/
            ├── kustomization.yaml
            ├── deployment-patch.yaml
            ├── hpa.yaml
            └── pdb.yaml

Base Configuration

[Kustomize Glossary - Base] — Kubernetes SIG CLI , 2024-07-01

Base Kustomization

# apps/myapp-api/base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

commonLabels:
  app.kubernetes.io/name: myapp-api
  app.kubernetes.io/component: api

resources:
  - deployment.yaml
  - service.yaml
  - serviceaccount.yaml

images:
  - name: myapp-api
    newName: registry.example.com/myapp/myapp-api
    newTag: latest

Base Deployment

# apps/myapp-api/base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-api
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: myapp-api
  template:
    metadata:
      labels:
        app.kubernetes.io/name: myapp-api
    spec:
      serviceAccountName: myapp-api
      containers:
        - name: api
          image: myapp-api
          ports:
            - containerPort: 8080
              name: http
          env:
            - name: ASPNETCORE_ENVIRONMENT
              value: Production
            - name: ASPNETCORE_URLS
              value: http://+:8080
          envFrom:
            - secretRef:
                name: myapp-api-secrets
          resources:
            requests:
              cpu: 100m
              memory: 256Mi
            limits:
              cpu: 1000m
              memory: 1Gi
          readinessProbe:
            httpGet:
              path: /health/ready
              port: 8080
            initialDelaySeconds: 10
            periodSeconds: 5
          livenessProbe:
            httpGet:
              path: /health/live
              port: 8080
            initialDelaySeconds: 30
            periodSeconds: 10

Base Service

# apps/myapp-api/base/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: myapp-api
spec:
  selector:
    app.kubernetes.io/name: myapp-api
  ports:
    - port: 80
      targetPort: 8080
      name: http

Environment Overlays

Staging Overlay

# apps/myapp-api/overlays/staging/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: myapp-staging

namePrefix: staging-

commonLabels:
  app.kubernetes.io/instance: staging

resources:
  - ../../base
  - hpa.yaml
  - externalsecret.yaml
  - ingressroute.yaml

patches:
  - path: deployment-patch.yaml
    target:
      kind: Deployment
      name: myapp-api

images:
  - name: myapp-api
    newName: registry.example.com/myapp/myapp-api
    newTag: 20260101-abc1234  # Updated by CI/CD

configMapGenerator:
  - name: myapp-api-config
    literals:
      - Environment=staging
      - LogLevel=Debug
      - Otlp__Endpoint=http://signoz-otel-collector.observability.svc.cluster.local:4317

Staging Deployment Patch

# apps/myapp-api/overlays/staging/deployment-patch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-api
spec:
  replicas: 2
  template:
    spec:
      containers:
        - name: api
          env:
            - name: Environment
              value: staging
          resources:
            requests:
              cpu: 200m
              memory: 512Mi
            limits:
              cpu: 1000m
              memory: 1Gi

Staging HPA

# apps/myapp-api/overlays/staging/hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: myapp-api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: myapp-api
  minReplicas: 2
  maxReplicas: 5
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80

Production Overlay

# apps/myapp-api/overlays/prod/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: myapp-prod

namePrefix: prod-

commonLabels:
  app.kubernetes.io/instance: prod

resources:
  - ../../base
  - hpa.yaml
  - pdb.yaml
  - externalsecret.yaml
  - ingressroute.yaml

patches:
  - path: deployment-patch.yaml
    target:
      kind: Deployment
      name: myapp-api

images:
  - name: myapp-api
    newName: registry.example.com/myapp/myapp-api
    newTag: 20260101-def5678

configMapGenerator:
  - name: myapp-api-config
    literals:
      - Environment=prod
      - LogLevel=Information
      - Otlp__Endpoint=http://signoz-otel-collector.observability.svc.cluster.local:4317

replicas:
  - name: myapp-api
    count: 3

Production Deployment Patch

# apps/myapp-api/overlays/prod/deployment-patch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-api
spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    spec:
      containers:
        - name: api
          env:
            - name: Environment
              value: prod
          resources:
            requests:
              cpu: 500m
              memory: 1Gi
            limits:
              cpu: 2000m
              memory: 2Gi
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: kubernetes.io/hostname
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              app.kubernetes.io/name: myapp-api

Pod Disruption Budget

# apps/myapp-api/overlays/prod/pdb.yaml
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: myapp-api
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app.kubernetes.io/name: myapp-api

Strategic Merge Patches

[Kustomize - Patches (Strategic Merge)] — Kubernetes SIG CLI , 2024-07-01

JSON Patch

# apps/myapp-api/overlays/staging/json-patch.yaml
- op: add
  path: /spec/template/spec/containers/0/env/-
  value:
    name: FEATURE_FLAG_NEW_SEARCH
    value: "true"

- op: replace
  path: /spec/template/spec/containers/0/resources/requests/memory
  value: 768Mi

Using JSON Patch

# apps/myapp-api/overlays/staging/kustomization.yaml
patches:
  - path: json-patch.yaml
    target:
      kind: Deployment
      name: myapp-api

ConfigMap and Secret Generators

ConfigMap Generator

# apps/myapp-api/overlays/staging/kustomization.yaml
configMapGenerator:
  - name: myapp-api-config
    behavior: merge
    files:
      - appsettings.json=config/appsettings.staging.json
    literals:
      - NATS_SUBJECT_PREFIX=staging

Secret Generator (for dev only)

# apps/myapp-api/overlays/dev/kustomization.yaml
secretGenerator:
  - name: myapp-api-dev-secrets
    literals:
      - DATABASE_PASSWORD=devpassword
    options:
      disableNameSuffixHash: true

Flux Integration

[Flux - Kustomize Controller] — Flux Project , 2024-10-01

Kustomization Resource

# clusters/my-cluster/apps/myapp-api-staging.yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: myapp-api-staging
  namespace: flux-system
spec:
  interval: 10m
  path: ./apps/myapp-api/overlays/staging
  prune: true
  sourceRef:
    kind: GitRepository
    name: my-infra
  targetNamespace: myapp-staging
  healthChecks:
    - apiVersion: apps/v1
      kind: Deployment
      name: staging-myapp-api
      namespace: myapp-staging
  dependsOn:
    - name: external-secrets
    - name: data-layer

Validation

Build and Preview

# Preview staging configuration
kustomize build apps/myapp-api/overlays/staging

# Validate without applying
kubectl apply --dry-run=client -k apps/myapp-api/overlays/staging

# Diff against cluster state
kubectl diff -k apps/myapp-api/overlays/staging

Conclusion

Using Kustomize overlays allows us to maintain a clean, DRY (Don’t Repeat Yourself) configuration base while accommodating the specific needs of each environment. This approach aligns perfectly with GitOps principles, providing a declarative source of truth that is both flexible and maintainable.

Kustomize structure benefits:

Feature	Benefit
Base + Overlays	DRY configuration management
Patches	Environment-specific modifications
Generators	ConfigMaps and Secrets from files/literals
Name Prefixing	Clear resource namespacing
Flux Integration	GitOps-native deployment

Combined with Flux, Kustomize enables declarative, version-controlled Kubernetes deployments.

[Kustomize Documentation] — Kubernetes SIG CLI