Module 9 of 16

Secrets Management & Machine Identity

Vault, dynamic secrets, certificate rotation, and replacing secret sprawl with workload identity

3.5 hours2 labsFree

Start here

Learning objectives

  • Integrate HashiCorp Vault with Kubernetes
  • Implement dynamic secrets and automatic rotation
  • Replace static credentials with workload identity
  • Design a secrets management strategy for production

Before

  • API keys in env vars
  • Shared database passwords
  • Long-lived certificates
  • Vault tokens as secrets
  • Manual rotation (or never)

After

  • SVID-based authentication
  • Dynamic per-service credentials
  • Auto-rotated short-lived certs
  • Identity-based Vault auth
  • Automatic rotation always
SECRET SPRAWL vs WORKLOAD IDENTITYSecret Sprawl (Before)API keys in env varsDatabase passwords in ConfigMapsLong-lived certificates on diskVault tokens (another secret!)Secrets in Git historyEvery secret is a breach vectorManual rotation (or never)Workload Identity (After)SPIFFE SVIDs (auto-issued)Dynamic DB credentials (Vault)Short-lived certs (auto-rotated)SVID auth to Vault (no tokens)Zero secrets in code or configIdentity replaces secretsAutomatic rotation (always)

Secret sprawl is one of the most dangerous problems in cloud-native security. API keys in environment variables, database passwords in ConfigMaps, certificates that never rotate, and Vault tokens that need their own distribution mechanism. Each secret is a breach vector. Workload identity replaces most of these with cryptographic proof that needs no distribution.

Kubernetes Secrets: The Problem

Kubernetes Secrets are base64-encoded (not encrypted) by default. They are accessible to anyone with RBAC access to the namespace. They persist in etcd and must be encrypted at rest explicitly. They have no automatic rotation mechanism.

HashiCorp Vault Integration

Vault provides encrypted secret storage, dynamic credentials (database passwords generated on demand and automatically revoked), PKI certificates, and transit encryption. The key integration: workloads authenticate to Vault using their SPIFFE SVID instead of static Vault tokens.

Dynamic Secrets

# Instead of: static database password shared across all services
# Vault generates: unique short-lived credentials per service
# Each credential is automatically revoked after TTL expires

# Vault dynamic database credential:
vault read database/creds/readonly
# Key             Value
# lease_id        database/creds/readonly/abc123
# lease_duration  1h
# username        v-svc-readonly-xyz123  (unique per request!)
# password        A1b2C3-random-xyz      (auto-revoked in 1 hour)

Replacing Secrets with Workload Identity

The ultimate goal: eliminate static secrets entirely. Services authenticate with their SVID (automatic, short-lived, no distribution needed). Vault issues dynamic credentials based on verified identity. Certificates are managed by SPIRE (automatic rotation). The only secret that remains is the Vault unseal key — protected by cloud KMS.

Real world

Where this shows up

  • Eliminating static database passwords with Vault dynamic secrets
  • SPIFFE-based Vault authentication (no token distribution)
  • Certificate rotation for internal PKI
  • Encrypting application data with Vault transit

Common mistakes

What usually breaks

  • Storing Vault tokens as Kubernetes Secrets (replaces one secret problem with another)
  • Not encrypting Kubernetes Secrets at rest in etcd
  • Using static database passwords shared across all services
  • Not implementing credential rotation — "it works fine" until the breach

Key terms

Vocabulary used in this module

Secret Sprawl

Uncontrolled proliferation of credentials across systems and configs

Dynamic Secrets

Credentials generated on-demand with automatic expiration

Vault

HashiCorp secret management platform

Transit Encryption

Vault engine for encrypting/decrypting data without storing it

Labs

Hands-on labs

35 minIntermediate

Integrate Vault with Kubernetes

Deploy Vault and configure Kubernetes authentication.

  1. Deploy Vault in dev mode on Kubernetes
  2. Enable Kubernetes auth backend
  3. Create a policy for a service account
  4. Retrieve secrets from a pod using the Vault agent
View lab on GitHub
30 minIntermediate

Dynamic Secret Rotation

Configure Vault to issue short-lived database credentials.

  1. Deploy PostgreSQL
  2. Configure Vault database secrets engine
  3. Generate dynamic credentials with 1-hour TTL
  4. Verify automatic revocation after TTL expires
View lab on GitHub

Recap

Key takeaways

  • Secret sprawl is a top cloud-native security risk — every static secret is a breach vector
  • Vault provides encrypted storage, dynamic credentials, and PKI — but needs its own auth
  • SPIFFE SVIDs replace Vault tokens — workloads authenticate to Vault with their identity
  • Dynamic secrets (short-lived, auto-revoked) are always better than static passwords
  • The goal: zero static secrets in your infrastructure

Related resources

Keep learning across CodersSecret