The problem everyone hits
You’ve got a Kubernetes cluster. Now you need to describe what should run in it. You write some YAML, apply it, it works.
Then you need a second environment. Or a second service. Or someone else joins the project and asks “how do I add an app to this?” and you don’t have a good answer.
This is the manifest management problem, and there are five common solutions — ranging from “this works until it doesn’t” to “this is what production platforms actually look like.”
Approach 1: Raw manifests
The starting point for almost everyone. Write a YAML file, kubectl apply -f, done.
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
namespace: myapp
spec:
replicas: 1
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: myapp:v1.2.3
Where it works: one service, one environment, learning Kubernetes. The feedback loop is immediate — write YAML, see what happens.
Where it breaks:
- No templating. Want to change the image tag across ten services? Ten files, ten edits, ten chances to get it wrong.
- Live state leaks in. If you export existing resources with
kubectl get -o yaml, you getresourceVersion,generation,creationTimestamp, andmanagedFieldsin the output. Commit that to Git and you’ve created a permanent source of conflicts — ArgoCD compares what’s in Git against what’s in the cluster, sees stale version counters, and the diff never clears. - Copy-paste hell. A Deployment, a Service, an IngressRoute, a ServiceAccount, a NetworkPolicy — five files per app. Add a new app, copy five files, change the names, forget to update one. This is how environments drift apart silently.
The fix for the live-state problem is: only commit desired state. Strip every field that Kubernetes manages internally back to its clean spec. It’s tedious and easy to forget, which is exactly why people move on from raw manifests.
Approach 2: Kustomize
Kustomize is built into kubectl (kubectl apply -k) and natively supported by ArgoCD. The idea: you have a base/ with your raw manifests, and overlays that patch on top of them for different environments.
app/
├── base/
│ ├── deployment.yaml
│ ├── service.yaml
│ └── kustomization.yaml
└── overlays/
├── staging/
│ ├── kustomization.yaml # patches replicas to 1, image to :staging
└── production/
└── kustomization.yaml # patches replicas to 3, image to :v1.2.3
# overlays/production/kustomization.yaml
resources:
- ../../base
patches:
- patch: |-
- op: replace
path: /spec/replicas
value: 3
target:
kind: Deployment
Where it works: multi-environment setups where the difference between environments is mostly configuration values, not structure. Kustomize is good at this — you write the base once and patch only what differs.
Where it breaks:
- No real parameterization. Kustomize patches are surgical edits, not templates. If your base structure needs to vary (different resource shapes per environment, conditional blocks), you’re fighting the tool.
- Patching deep structures is ugly. JSON patches on nested YAML are verbose and hard to read. You end up writing more patch YAML than it would take to just copy the file.
- Still repetitive across apps. Each app still gets its own base directory. You’re not abstracting the shared patterns across apps, only the differences between environments of the same app.
Kustomize is a significant step up from raw manifests for multi-environment setups. For complex templating or platform-level abstractions, it runs out of power quickly.
Approach 3: Helm
Helm adds real templating. Charts are parameterized bundles — templates with variables, conditionals, and loops — and values files supply the parameters.
# templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Values.name }}
namespace: {{ .Release.Namespace }}
spec:
replicas: {{ .Values.replicas | default 1 }}
template:
spec:
containers:
- name: {{ .Values.name }}
image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
{{- if .Values.resources }}
resources: {{ .Values.resources | toYaml | nindent 12 }}
{{- end }}
# values-production.yaml
name: myapp
replicas: 3
image:
repository: myorg/myapp
tag: v1.2.3
Helm renders the templates at deploy time. What lands in the cluster is clean rendered YAML — no internal state, no conflicts.
Where it works: almost everywhere. The Helm Hub has charts for most common software already. For custom apps, writing a chart once and parameterizing per-environment is straightforwardly better than copying YAML.
Where it breaks:
- Chart authoring is verbose. Writing a Helm chart from scratch involves a lot of Go templating boilerplate. For a simple app, it can feel like more scaffolding than application.
- Debugging rendered output is annoying.
helm templateis your friend, but errors in templates produce unhelpful messages. The indentation rules (nindent,indent,toYaml) have sharp edges. - Values files still pile up. If every app has its own values file and there’s no shared structure between them, you’re back to copy-paste but now in YAML-that-configures-YAML.
Helm is the right tool for most Kubernetes deployments. The ecosystem support alone (upstream charts for Postgres, Redis, Vault, every CNCF project) makes it the pragmatic default.
Approach 4: Jsonnet / CUE
For teams that need programmatic config generation — actual code, not templates — Jsonnet and CUE are the serious alternatives.
// deployment.jsonnet
local k = import "k.libsonnet";
local deployment(name, image, replicas=1) =
k.apps.v1.deployment.new(name, replicas, [
k.core.v1.container.new(name, image)
]);
{
"deployment.yaml": deployment("myapp", "myorg/myapp:v1.2.3", replicas=3)
}
Where it works: large platforms where configuration is genuinely complex — many environments, many apps, deep interdependencies. Jsonnet lets you write real functions, share libraries, compose abstractions properly.
Where it breaks:
- Steep learning curve. Jsonnet is a full language. CUE even more so — it has types, schemas, and a constraint system that takes time to internalise.
- Small community. Excellent tooling, but you’re solving problems that have fewer Stack Overflow answers.
- Overkill for most setups. If you’re not managing hundreds of services across multiple clusters, Helm is simpler and has everything you need.
Jsonnet is used seriously at Google-scale infrastructure teams and in some CNCF projects. For a homelab or a small-to-medium platform, it’s the right answer to a question you probably aren’t asking yet.
Approach 5: App-of-apps with generated Application CRDs
This is the ArgoCD-native meta-layer. Instead of managing manifests, you manage Application resources — and potentially use a chart or tool to generate those too.
A naive version: commit a folder of Application YAML files to Git, one per service. ArgoCD watches the folder and deploys each app.
A more sophisticated version: one “root app” that points to a chart, which generates all the other Application resources dynamically from a single config file.
Where it works: at the platform level, not the individual app level. App-of-apps is how you manage what ArgoCD manages, not how you write the service manifests themselves. Combined with Helm, it gives you centralized control over the entire cluster’s structure.
Where it breaks:
- Manual
ApplicationCRDs are painful. If you’re maintaining a folder of hand-writtenApplicationYAML files — one per service — you’ve traded manifest copy-paste for Application copy-paste. Each app needs its own CRD with its repo URL, path, sync policy, project reference. - Sync ordering matters. The root app must exist before children can sync. Get the wave ordering wrong and apps try to deploy before their namespaces exist.
How this homelab compares
My setup sits at the far end of approach 5, using Helm throughout.
There’s a single applications.yml file that describes every service in the cluster. A root Helm chart reads it and generates all the ArgoCD Application and AppProject CRDs automatically. Adding a service means adding an entry to that file — not touching five different places across five different files.
# applications.yml — this is the entire service catalog
- namespace: web-vaultwarden
networkPolicies:
profile: web-app
applications:
- applicationCode: web-vaultwarden
path: helm-charts/extra-objects
autoSync: true
That one entry generates: a Namespace, an ArgoCD AppProject, an ArgoCD Application, a set of Cilium NetworkPolicies (deny-all with ingress from Traefik and DNS/HTTPS egress), and a ServiceAccount. Nothing is written by hand.
The actual service manifests live in an extra-objects chart — a thin wrapper that renders raw YAML from values files. No templating in the service manifests themselves (they’re simple enough not to need it), but the infrastructure scaffolding around each app is entirely generated.
The result: every service gets the same operational properties. Same GitOps workflow, same secret management, same network isolation, same TLS termination. The platform work was done once. Adding a new app is writing manifests for the app’s specific behavior, not recreating the scaffolding.
The honest spectrum
| Approach | Templating | Abstraction | Ecosystem | Complexity |
|---|---|---|---|---|
| Raw manifests | None | None | None | Low |
| Kustomize | Patches only | Overlays | Medium | Low-medium |
| Helm | Full | Per-chart | Large | Medium |
| Jsonnet/CUE | Full + typed | Libraries | Small | High |
| App-of-apps | Depends | Platform-level | ArgoCD-native | High |
Most setups should start at Helm. Kustomize if you’re multi-environment and comfortable with patching. App-of-apps when you’re managing the platform layer, not individual services. Jsonnet/CUE when you know you’ve outgrown Helm — which is a specific and relatively rare problem to have.
Raw manifests are fine for learning. They’re the wrong answer for anything you intend to maintain.
More on how the homelab is structured: My Homelab Runs on GitOps.
