GitOps demands that every piece of desired state lives in a Git repository — but Git is not a secret store. Committing a database password, an API key, or a TLS private key in plaintext to a repository is one of the most common and consequential security mistakes in modern infrastructure. Even in a private repository, the blast radius of an accidental exposure includes every developer with clone access, every CI runner, every fork, and every entry in the reflog. At big-tech scale, thousands of engineers touch the same repositories — a single plaintext secret commit is a permanent liability.
This lesson covers the three dominant patterns used in production GitOps environments to solve this problem: Sealed Secrets, SOPS, and External Secrets Operator. Each solves a different part of the problem space. By the end, you will know which tool fits which context and how to implement each one correctly.
The core tension: GitOps says "put everything in Git." Security says "never put secrets in Git." The solution is not to break one of these rules — it is to ensure that what goes into Git is an encrypted or referenced representation of the secret, not the secret itself. The cluster, and only the cluster, can decrypt or retrieve the real value.
Why Kubernetes Secrets Alone Are Not Enough
A Kubernetes Secret object is base64-encoded, not encrypted. Anyone who can read the etcd database or run kubectl get secret my-db-creds -o yaml with the right RBAC permissions gets the raw value. If you commit a Secret manifest to Git, you have committed the secret in plaintext with a thin disguise. This is the starting point that every secrets-in-GitOps tool exists to fix.
Option 1: Sealed Secrets (Bitnami)
Sealed Secrets is a Kubernetes controller and CLI tool from Bitnami. The model is asymmetric encryption baked into the cluster itself. The controller holds a private key inside the cluster; you use the corresponding public key (fetched via CLI) to encrypt a regular Kubernetes Secret into a SealedSecret custom resource. The SealedSecret is safe to commit to Git — only the controller holding the private key can decrypt it.
Install the controller via Helm:
# Add the Sealed Secrets Helm repo and install the controller
helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets
helm repo update
helm install sealed-secrets sealed-secrets/sealed-secrets \
--namespace kube-system \
--set fullnameOverride=sealed-secrets-controller
# Install the kubeseal CLI (macOS example; Linux: download from GitHub releases)
brew install kubeseal
Once the controller is running, seal a secret with kubeseal:
# Step 1 – create a regular (unencrypted) Secret manifest in a temp file
kubectl create secret generic db-credentials \
--from-literal=username=appuser \
--from-literal=password='S3cur3P@ssw0rd!' \
--namespace=production \
--dry-run=client \
-o yaml > /tmp/db-credentials.yaml
# Step 2 – seal it (fetches the public key from the controller automatically)
kubeseal \
--controller-name=sealed-secrets-controller \
--controller-namespace=kube-system \
--format yaml \
< /tmp/db-credentials.yaml \
> gitops-config/secrets/db-credentials-sealed.yaml
# The sealed file is SAFE to commit. Remove the plaintext temp file.
rm /tmp/db-credentials.yaml
# Step 3 – commit and push
git add gitops-config/secrets/db-credentials-sealed.yaml
git commit -m "feat: add sealed db-credentials for production"
git push
The resulting SealedSecret manifest looks like this (ciphertext abbreviated):
When ArgoCD or Flux applies this manifest, the Sealed Secrets controller intercepts it, decrypts the values using its private key, and creates a standard Kubernetes Secret in the same namespace. Your Pods mount the Secret normally — they never know about the sealing layer.
Scope your sealed secrets correctly. By default, kubeseal produces a namespace-scoped sealed secret that can only be decrypted in the declared namespace. Use --scope cluster-wide only if you genuinely need a secret accessible in multiple namespaces — it is a broader blast radius if the sealed file leaks.
Controller key rotation is a production responsibility. The Sealed Secrets controller generates a new sealing key every 30 days by default. Old keys are kept for decryption but new secrets are sealed with the latest key. If you delete the controller without backing up its keys (stored as Secrets in kube-system), you permanently lose the ability to decrypt existing SealedSecrets. Back up the key Secret: kubectl get secret -n kube-system -l sealedsecrets.bitnami.com/sealed-secrets-key -o yaml > sealed-secrets-master-key-backup.yaml — and store this backup in a secure vault, not in Git.
Option 2: SOPS (Mozilla) — Encrypted Files in Git
SOPS (Secrets OPerationS) is a file-level encryption tool from Mozilla. Instead of a Kubernetes-specific controller, SOPS works at the file system level: it encrypts the values in a YAML, JSON, or ENV file while leaving the keys readable. The encryption backend can be AWS KMS, GCP KMS, Azure Key Vault, HashiCorp Vault, or age (a modern, simple encryption tool). In practice, age is the default for teams without a cloud KMS, and AWS/GCP KMS is preferred in cloud environments because key access is tied to IAM — no long-lived key material to manage.
Encrypt a Kubernetes Secret with SOPS + age:
# Install age and sops
brew install age sops # macOS
# apt install age sops # Debian/Ubuntu
# Generate an age key pair (do this once per operator/CI identity)
age-keygen -o age-key.txt
# Public key: age1qldzds...
# Private key stored in age-key.txt -- NEVER COMMIT THIS FILE
# Add age public key to .sops.yaml at the repo root -- this IS committed
cat > .sops.yaml << 'EOF'
creation_rules:
- path_regex: secrets/.*\.yaml$
age: age1qldzds6xnekn3yjcxgkuq6y8w6g5hkzxm... # replace with your real pubkey
EOF
# Create your plaintext secret file temporarily
cat > /tmp/db-secret.yaml << 'EOF'
apiVersion: v1
kind: Secret
metadata:
name: db-credentials
namespace: production
type: Opaque
stringData:
username: appuser
password: S3cur3P@ssw0rd!
EOF
# Encrypt with sops -- outputs the encrypted version
SOPS_AGE_KEY_FILE=age-key.txt sops --encrypt /tmp/db-secret.yaml \
> gitops-config/secrets/db-secret.yaml
# Verify the keys are readable (values are encrypted):
# apiVersion: v1
# kind: Secret
# ...
# stringData:
# username: ENC[AES256_GCM,data:4wd...,type:str]
# password: ENC[AES256_GCM,data:rB2...,type:str]
# Commit the encrypted file -- safe to push
git add .sops.yaml gitops-config/secrets/db-secret.yaml
git commit -m "feat: add SOPS-encrypted db credentials"
git push
# On the cluster side (Flux with SOPS decryption support):
# Create a Kubernetes Secret holding the age private key
kubectl create secret generic sops-age \
--namespace=flux-system \
--from-file=age.agekey=age-key.txt
# Reference it in your Flux Kustomization resource:
# decryption:
# provider: sops
# secretRef:
# name: sops-age
Three approaches to GitOps secrets management — Sealed Secrets (in-cluster key), SOPS (encrypted values in Git), and External Secrets Operator (reference only in Git).
Option 3: External Secrets Operator (ESO)
ESO takes a different philosophical approach: secrets are never stored in Git, even encrypted. Instead, you commit a lightweight ExternalSecret custom resource that describes where to find the secret (AWS Secrets Manager path, HashiCorp Vault path, GCP Secret Manager name, etc.) and how to map it to a Kubernetes Secret. The ESO controller, running in the cluster, polls the external vault and synthesizes the Kubernetes Secret on the fly. When the secret rotates in the vault, ESO picks up the new value automatically within the configured refresh interval.
# Install ESO via Helm
helm repo add external-secrets https://charts.external-secrets.io
helm repo update
helm install external-secrets external-secrets/external-secrets \
--namespace external-secrets \
--create-namespace \
--set installCRDs=true
# Create a SecretStore that points to AWS Secrets Manager.
# The controller uses an IAM role bound to a ServiceAccount (IRSA / EKS Pod Identity).
cat > gitops-config/secrets/cluster-secretstore.yaml << 'EOF'
apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
name: aws-secretsmanager
spec:
provider:
aws:
service: SecretsManager
region: us-east-1
auth:
jwt:
serviceAccountRef:
name: external-secrets-sa
namespace: external-secrets
EOF
# Create an ExternalSecret that maps a Secrets Manager path to a k8s Secret
cat > gitops-config/secrets/db-external-secret.yaml << 'EOF'
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: db-credentials
namespace: production
spec:
refreshInterval: 1h # how often ESO polls the vault
secretStoreRef:
name: aws-secretsmanager
kind: ClusterSecretStore
target:
name: db-credentials # name of the k8s Secret to create
creationPolicy: Owner
data:
- secretKey: username # key in the k8s Secret
remoteRef:
key: prod/db/credentials # path in Secrets Manager
property: username # JSON field inside the secret
- secretKey: password
remoteRef:
key: prod/db/credentials
property: password
EOF
# Both files are plain YAML with NO secret values -- safe to commit
git add gitops-config/secrets/
git commit -m "feat: add ExternalSecret for db credentials (ESO)"
git push
Choosing the Right Tool
At big-tech scale the three tools are not mutually exclusive — many organizations use ESO for cloud-managed secrets (AWS SM, GCP SM, Vault) and SOPS or Sealed Secrets for infrastructure bootstrap secrets that must exist before the vault is reachable:
Sealed Secrets — Best for self-hosted clusters with no cloud KMS, small teams, and simple secret lifecycles. Low operational overhead. Weakness: secrets are locked to the cluster that holds the sealing key; key loss is catastrophic.
SOPS — Best when you want encrypted secrets stored in Git with full version history of secret changes, and you have a KMS (AWS KMS, age) you trust. Works with both ArgoCD (via Helm Secrets plugin or decrypt init container) and Flux (native SOPS support). Most transparent audit trail — every secret change is a git diff (of ciphertext).
External Secrets Operator — Best for cloud-native organizations with an existing secret manager. Secrets rotate automatically, access is governed by IAM policy (not file permissions), and the blast radius of a compromised Git repo is zero — the repo contains only references, not encrypted values. The overhead is that you need a secret manager running and properly configured.
Production pattern at large organizations: Use ESO as the primary secrets pattern for application secrets (database creds, API keys, TLS certs). Use SOPS to encrypt the ESO controller's own bootstrap credentials (the IAM key or Vault token it needs to authenticate to the secret manager on first boot) — those cannot be fetched from the vault because the vault connection is not yet established. This two-layer approach gives you automatic rotation, zero plaintext in Git, and a fully GitOps-managed bootstrap path.
Audit your Git history before adopting any of these tools. If a plaintext secret was ever committed — even in a commit that was later reverted — it is still in the reflog and in any clone. Run a tool like truffleHog or git-secrets against your entire history. If you find one, rotate the secret immediately, then use BFG Repo Cleaner or git filter-repo to purge the history and force-push. Treat the secret as compromised until you confirm rotation.
The next lesson moves to environment promotion — how changes flow from dev through staging to production in a GitOps pipeline, including automated image tag updates and promotion gates.