Security without visibility is flying blind. You need to know: who accessed what, when, whether SVID rotation is healthy, how many policy violations occurred, and whether any runtime alerts were triggered. Security observability connects all your security controls into a unified monitoring layer.
Kubernetes Audit Logging
Audit logs capture every API request: who made it, what was requested, and what was the result. Critical for compliance (SOC 2, PCI-DSS) and incident investigation.
# Audit policy: log all authentication failures and secret access
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: Metadata
resources:
- group: ""
resources: ["secrets"]
verbs: ["get", "list", "watch"]
- level: RequestResponse
users: ["system:anonymous"]
- level: Metadata
verbs: ["create", "delete", "patch"]
OpenTelemetry for Security
OpenTelemetry collects traces, metrics, and logs from all services. For security, this means: correlating authentication events with service behavior, tracking SVID rotation health, monitoring policy decision latency, and tracing request flows across mTLS boundaries.
Security Dashboards
Essential Grafana dashboards: authentication failures over time, SVID rotation success rate, policy violations by namespace, runtime alerts (Falco/Tetragon), certificate expiry countdown, and API server audit log analysis.