Production Design: Security, Performance, and Scale

Performance Model

JWT validation is usually fast after JWKS keys are cached. External authorization adds a network hop on the request path. That means auth service latency becomes user-facing latency. Use short timeouts, horizontal scaling, connection reuse, circuit breakers, and careful caching where policy allows it.

Production Resilience Example

This cluster snippet shows the types of controls you should think about for the central auth service: timeouts, circuit breakers, outlier detection, and stable Kubernetes service discovery.

clusters:
- name: central_authz_service
  type: STRICT_DNS
  connect_timeout: 300ms
  circuit_breakers:
    thresholds:
    - priority: DEFAULT
      max_connections: 500
      max_pending_requests: 1000
      max_requests: 2000
  outlier_detection:
    consecutive_5xx: 5
    interval: 5s
    base_ejection_time: 30s
  load_assignment:
    cluster_name: central_authz_service
    endpoints:
    - lb_endpoints:
      - endpoint:
          address:
            socket_address:
              address: authz.platform.svc.cluster.local
              port_value: 9000

Fail-Closed vs Fail-Open

Security-sensitive routes should usually fail closed: if Envoy cannot validate identity or ask the auth service, deny the request. Some read-only low-risk routes may have a documented degraded mode, but it must be explicit. Never let a timeout silently become broad access.

Scaling the Auth Layer

• Run multiple Envoy replicas across nodes and zones.
• Run multiple auth service replicas with autoscaling and health checks.
• Cache JWKS and simple policy decisions carefully.
• Keep IdP calls out of the hot path after login when possible.
• Use metrics for allow rate, deny rate, error rate, decision latency, token failures, and route-level spikes.

Advantages

• Consistent auth behavior across products.
• Better audit logs and incident response.
• Less duplicated token parsing and SSO code.
• Easier migration from static secrets to short-lived tokens and federated credentials.

Disadvantages

• Central auth becomes critical path infrastructure.
• Bad central policy can break many products at once.
• Teams need clear ownership for route policy, product policy, and data policy.
• Performance work is required when every request asks for a decision.

Production Rollout Plan

1. Inventory products, routes, token types, current auth logic, and risk level.
2. Start with one low-risk product behind Envoy in report-only or shadow mode.
3. Enable JWT validation for API routes with explicit issuer and audience.
4. Add ext_authz for routes that need central policy or browser SSO decisions.
5. Strip and rewrite trusted identity headers.
6. Add dashboards and alerts before migrating high-risk products.
7. Document emergency rollback and key-rotation procedures.