Module 8: Distributed Security & Zero Trust Slides

Slide Outline

1. Distributed Security & Zero Trust - How modern distributed systems authenticate workload-to-workload — mTLS, SPIFFE/SPIRE, OPA, and the Zero Trust patterns that replace network-perimeter security.
2. Learning Objectives - 5 outcomes for this module
3. Why This Module Matters - This is the differentiator module of this course. Most distributed-systems training treats security as a separate topic
4. Before vs After - The operational shift this module teaches
5. Zero Trust in One Sentence - Lesson section from the full module
6. mTLS - The Secure Channel - Lesson section from the full module
7. SPIFFE / SPIRE - Lesson section from the full module
8. Authorization with OPA - Lesson section from the full module
9. API Security - Lesson section from the full module
10. Federation Across Trust Domains - Lesson section from the full module
11. Operational Practice - Lesson section from the full module
12. mTLS Handshake Sequence - Lesson section from the full module
13. Real-World Use Cases - Bloomberg, Pinterest, Anthem, and Yahoo all run SPIRE in production for service identity at scale., Netflix uses an internal SPIFFE-style identity system across thousands of services.
14. Common Mistakes to Avoid - 4 mistakes covered
15. Production Notes - 4 practical notes
16. Security Risks to Watch - 4 risks covered
17. Hands-On Labs - 3 hands-on labs
18. Key Takeaways - 5 points to remember

Learning Objectives

• Explain Zero Trust as an architectural principle, not a product
• Bootstrap mTLS between services with short-lived, automatically-rotated credentials
• Use SPIFFE/SPIRE to issue cryptographic workload identity at scale
• Enforce authorization with OPA / Rego at admission and at request time
• Federate trust across clusters and clouds without leaking secrets

Why This Module Matters

This is the differentiator module of this course. Most distributed-systems training treats security as a separate topic added at the end. In real production engineering, security is woven into every architectural decision — the choice between shared secrets and SPIFFE workload identity is the same scale of architectural choice as the choice between monolith and microservices. Engineers who internalise this model design systems that scale and stay secure together. Engineers who do not end up retrofitting security after the first incident.

Production Notes

• Issue SVIDs valid for 1 hour or less; rotate automatically. Long-lived credentials are accumulated risk.
• Default-deny at the policy layer; explicit allow rules; everything else rejected.
• Treat SPIRE Server as tier-0: HA, KMS-backed encryption at rest, tested restoration runbook.
• Log every authz decision with the principal's SPIFFE ID. That log is your audit trail.

Common Mistakes

• Long-lived (24h+) certificates as a “safety margin”. The opposite is true — longer = larger blast radius if leaked.
• OPA policies returning HTTP 500 on deny instead of 403. Triage gets confused; production stays on fire.
• Substring matching on SPIFFE IDs (<code>strings.Contains(id, &quot;orders&quot;)</code>) instead of structured comparison. Trivial to bypass.
• Static trust-bundle copies for federation. Become stale at the next CA rotation.

Key Takeaways

• Zero Trust is an architectural principle: never trust caller location, always verify identity
• mTLS gives bidirectional cryptographic identity; SPIFFE/SPIRE makes it scalable
• Workload identity replaces shared secrets and long-lived credentials
• OPA / Rego puts authorization policy into version control and CI
• Federation extends Zero Trust across clusters and clouds without identity sprawl

Hands-On Labs

1. Lab 8.1 — mTLS Between Two Services with SPIFFE

   Deploy two services on Kubernetes; bootstrap mTLS using SPIRE-issued SVIDs.

   120 minutes - Intermediate

   • Install SPIRE on kind cluster
   • Register workloads with SPIRE selectors
   • Implement mTLS server using go-spiffe
   • Verify peer identity on every connection

   View lab files on GitHub

2. Lab 8.2 — OPA Authorization at Envoy

   Add OPA ext_authz to Envoy; enforce SPIFFE-ID-based access policy.

   90 minutes - Advanced

   • Deploy Envoy + OPA sidecar pattern
   • Write Rego policy: only orders-api can call payments-api
   • Send authorized and unauthorized calls; verify deny path

   View lab files on GitHub

3. Lab 8.3 — SPIFFE Federation Across Two Clusters

   Stand up two kind clusters; federate trust; have a workload in cluster A authenticate to a workload in cluster B.

   120 minutes - Advanced

   • Stand up two kind clusters
   • Install SPIRE in each with distinct trust domains
   • Configure bundle endpoint exchange
   • Cross-cluster mTLS verified by SPIFFE ID

   View lab files on GitHub