Distributed Security & Zero Trust

The classical security model assumed a trusted internal network behind a firewall. That assumption broke the moment one application talked to another over the internet, and it broke entirely with cloud-native architectures where workloads spin up and down across clusters, regions, and clouds in seconds. Zero Trust is the response: do not trust any caller based on network position; verify identity, posture, and policy on every request.

This module is the load-bearing security wall of distributed-systems engineering. After this you should be able to design how every internal API call authenticates, authorises, and audits itself, even across cluster and cloud boundaries.

Zero Trust in One Sentence

“Never trust, always verify, assume breach.” That is the operational summary. The architectural translation: every caller has a cryptographically verifiable identity; every authorization decision uses that identity plus context; every channel is encrypted; every action is logged; and the system is designed so a compromised component does not give the attacker the keys to the kingdom.

mTLS — The Secure Channel

Mutual TLS is the foundation: both client and server present certificates and verify each other's identity. Unlike server-side TLS (where only the server is identified), mTLS gives you bidirectional cryptographic identity on every connection.

The catch: mTLS is hard at scale because of credential management. Long-lived certificates leak, get committed to git, and never rotate. Short-lived certificates require an identity issuance system. That system is what SPIFFE/SPIRE provides.

SPIFFE / SPIRE

SPIFFE (Secure Production Identity Framework For Everyone) is a CNCF specification defining a universal format for workload identity:

• SPIFFE ID: a URI like spiffe://example.com/ns/orders/sa/orders-api that uniquely names a workload.
• SVID (SPIFFE Verifiable Identity Document): a cryptographic document (X.509 certificate or JWT) that proves the holder owns the SPIFFE ID. Short-lived (minutes to an hour) and auto-rotated.
• Workload API: a Unix-socket API workloads use to fetch their current SVID. No application code touches secrets directly.

SPIRE is the reference implementation: a SPIRE Server issues SVIDs after a SPIRE Agent attests the workload via selectors (Kubernetes namespace, ServiceAccount, container image hash, etc.). The result: every workload has a unique cryptographic identity, automatically issued and rotated, with no shared secrets.

The free Mastering SPIFFE & SPIRE course goes 13 modules deep on this topic. This module gives you the architectural picture; that course gives you the deployment.

Authorization with OPA

Authentication answers “who is calling?”; authorization answers “is this caller allowed to do this?”. OPA (Open Policy Agent) is the CNCF-graduated policy engine that lets you express authz rules as code (in the Rego language), evaluate them at admission time (Kubernetes admission webhook, Kyverno) or at request time (Envoy ext_authz, application middleware).

Sample Rego rule: “a workload from spiffe://example.com/ns/billing/sa/charger may call POST /charges if its tenant_id matches the charge's tenant_id”. The rule lives in version control, runs in CI, ships independently of application code.

API Security

For external API security — user authentication, token formats, OAuth, JWT — the patterns are different. Module 9 of the Cloud Native Security Engineering course covers these. The API Attack & Defense Simulator is the hands-on exercise.

For service-to-service inside your infrastructure: SPIFFE workload identity + mTLS + OPA authz is the production architecture. For human-to-API: OAuth + JWT + scope-based policy. The two patterns coexist; do not blur them.

Federation Across Trust Domains

Multi-cluster and multi-cloud distributed systems need workloads in one cluster to authenticate workloads in another. SPIFFE federation is the mechanism: each trust domain (cluster) exposes its trust bundle via a bundle endpoint; federated peers fetch and trust each other's bundles. SVIDs issued in one cluster are verifiable in another.

This is how you build cross-cluster service-to-service security without VPNs, shared secrets, or per-cluster identity sprawl. The Zero Trust Network Builder simulator walks through SPIFFE federation scenarios in production form.

Operational Practice

• Issue SVIDs valid for 1 hour or less; rotate automatically; never let credentials accumulate validity beyond what an attacker could exploit.
• Authorization decisions log every allow/deny with the principal's SPIFFE ID; this is your audit trail.
• Default-deny at the policy layer; explicit allow rules for known patterns; everything else rejected.
• Treat the workload identity provider (SPIRE) as a tier-0 dependency; HA cluster, backups, tested restoration.

mTLS Handshake Sequence

Self-Check Quiz

1. You issue SVIDs valid for 24 hours. The security team objects. Why? (Answer: the longer the validity, the larger the blast radius if a credential leaks. Industry default for SPIFFE SVIDs is 1 hour with 30-min rotation. Short-lived = self-healing.)
2. Your OPA policy denies a request. The application returns 500. What is wrong? (Answer: should return 403. 500 is “something broke”; 403 is “policy denied”. The distinction matters for triage.)
3. How do you authorise “only orders-service can call payments-service” in OPA? (Answer: input.peer.spiffe_id == "spiffe://example.com/ns/orders/sa/orders-svc" — or use a path-prefix match for groups of allowed callers.)
4. SPIFFE federation between two clusters fails 24 hours after rotation. What happened? (Answer: stale trust bundle. The federation peer needs to refresh from the bundle endpoint regularly. Static bundle copies always fail this way.)
5. Your service mesh (Istio) provides automatic mTLS. Do you still need SPIFFE? (Answer: Istio uses SPIFFE-style identity internally; explicit SPIFFE/SPIRE is needed for non-mesh workloads, federation across clusters, or richer authz.)

For implementation depth, take the free Mastering SPIFFE & SPIRE course. Reference the glossary on key primitives: SPIFFE, SPIRE, SVID, mTLS, workload identity, Zero Trust, OPA, and service mesh. The SPIFFE/SPIRE cheatsheet, OPA / Rego cheatsheet, and API Security cheatsheet are the operational quick references. Practice with the Zero Trust Network Builder.