Module 10 of 16

Runtime Security & Threat Detection

Falco, Tetragon, eBPF - detecting container escapes, unauthorized access, and runtime threats

3.5 hours2 labsFree

Start here

Learning objectives

  • Understand runtime threat categories in Kubernetes
  • Deploy Falco for syscall-based threat detection
  • Use Tetragon for eBPF-based enforcement
  • Build incident response procedures for runtime events
RUNTIME THREAT DETECTION PIPELINEThreat Eventshell in containerFalco / TetragoneBPF + syscall rulesAlertSlack / PagerDutyResponsekill / isolateCommon Runtime Threats DetectedShell spawned in containerSensitive file read (/etc/shadow)Unexpected outbound connectionPrivilege escalation attemptCrypto-mining processContainer escape via nsenterUnauthorized kubectl execBinary modified at runtime

Identity prevents unauthorized access. Network policies restrict traffic. But what about threats INSIDE an authorized workload? A compromised container running a cryptominer, an attacker spawning a shell, malware reading sensitive files - these are runtime threats that require real-time detection and response.

Falco: Syscall-Based Detection

Falco is a CNCF graduated project that monitors Linux syscalls and alerts on suspicious behavior. It runs as a DaemonSet and watches every container on the node.

# Falco rule: detect shell spawned in container
- rule: Terminal shell in container
  desc: A shell was spawned in a container
  condition: >
    spawned_process and container and
    proc.name in (bash, sh, zsh, dash)
  output: >
    Shell spawned in container
    (user=%user.name container=%container.name
     image=%container.image.repository
     pod=%k8s.pod.name ns=%k8s.ns.name)
  priority: WARNING

Tetragon: eBPF Runtime Enforcement

Tetragon goes beyond detection - it can block malicious actions in real-time using eBPF. While Falco alerts you to a shell spawn, Tetragon can prevent the shell from executing.

eBPF: The Foundation

eBPF (extended Berkeley Packet Filter) runs verified programs in the Linux kernel without loading a traditional kernel module. It gives runtime security tools low-level visibility and enforcement hooks, with overhead that depends on the probes, event volume, and userspace processing and should be measured on the target workload.

Incident Response

Detection without response is just monitoring. The response pipeline should: alert the security team (Slack/PagerDuty), capture forensic data (pod logs, network connections), isolate the pod (NetworkPolicy deny-all), preserve evidence (do not delete the pod), and investigate root cause.

Real world

Where this shows up

  • Detecting cryptominers in production containers
  • Container escape attempt alerting
  • Runtime threat detection for compliance (PCI-DSS)
  • Automated incident response for security events

Common mistakes

What usually breaks

  • Running Falco without custom rules (defaults miss many threats)
  • Not integrating alerts with incident response workflows
  • Using Tetragon enforcement rules without thorough testing (can block legitimate operations)
  • Not preserving forensic evidence when responding to incidents

Key terms

Vocabulary used in this module

Falco

CNCF runtime security tool that monitors syscalls for suspicious behavior

Tetragon

Cilium eBPF-based runtime enforcement engine

eBPF

Extended Berkeley Packet Filter - runs programs in the kernel sandbox

Syscall

System call - interface between user programs and the kernel

Labs

Hands-on labs

30 minIntermediate

Detect Container Escape Attempts with Falco

Deploy Falco and trigger security alerts.

  1. Install Falco as a DaemonSet
  2. Spawn a shell inside a container
  3. Read /etc/shadow from inside a container
  4. Observe Falco alerts for both events
View lab on GitHub
30 minAdvanced

Runtime Enforcement with Tetragon

Block malicious actions at the kernel level.

  1. Install Tetragon
  2. Create a TracingPolicy that blocks shell execution
  3. Attempt to spawn a shell - observe block
  4. View Tetragon event logs
View lab on GitHub

Recap

Key takeaways

  • Runtime security detects threats INSIDE authorized workloads - the last line of defense
  • Falco monitors syscalls and alerts on suspicious behavior (CNCF graduated)
  • Tetragon uses eBPF to BLOCK malicious actions, not just detect
  • eBPF enables kernel-level observability with near-zero overhead
  • Detection without response is just monitoring - build the full incident pipeline

Related resources

Keep learning across CodersSecret