Production Reference

Runtime Security Cheatsheet (Falco / eBPF)

Detection-engineering reference for Linux runtime security. Falco rule syntax, syscall reference, eBPF observability with Tetragon, and the patterns that catch real attacks while keeping noise out of your alert pipeline.

Command-firstProduction notesSecurity warningsHardened patterns

Falco rule structure

3 commands

- macro: container
  condition: container.id != host

Macros are reusable named conditions. Most rules start with `evt.type=execve and container` to scope to container syscalls.

Production note: Falco ships with a comprehensive set of macros in /etc/falco/falco_rules.yaml. Override them in falco_rules.local.yaml — never edit the upstream file.

- list: shell_binaries
  items: [bash, sh, csh, zsh, ksh]

Lists are reusable name sets. Reference with `proc.name in (shell_binaries)`.

Production note: Use lists for image registries, allowed processes, sensitive paths — anything that's a finite enumeration.

- rule: Terminal shell in container
  desc: A shell was used as the entrypoint/exec
  condition: spawned_process and container and shell_procs and proc.tty != 0
  output: Shell spawned in %container.name (user=%user.name shell=%proc.name)
  priority: WARNING

A complete Falco rule. Detect signature, message format, and severity.

Production note: Output template uses Sysdig field names (proc.name, container.image.repository, fd.name). The full list lives in `falco --list`.

High-signal Falco rules (start here)

5 commands

rule: Shell in container

Detects bash/sh spawn inside a container. Aggressive but very high signal — most apps don't spawn shells in production.

Production note: Add an exception list for sidecars/init-containers that legitimately spawn shells (e.g. `kubectl exec` in your toolkit pods).

rule: Sensitive file read by privileged container

Reads of /etc/shadow, /etc/passwd, ~/.ssh/* — classic credential-stealing pattern.

Warning: Whitelist legit security agents that read these files. False positives here erode trust in the signal.

rule: Outbound connection to disallowed CIDR

Network egress to unexpected destinations — crypto-mining pools, C2 infrastructure.

Production note: Maintain an allowlist of egress destinations (npm, pypi, your registry, your APIs). Anything else triggers.

rule: Write below /etc by container

Most container filesystems are immutable. Writes to /etc are an indicator of in-container persistence attempts.

Production note: Container images that need writable /etc usually have a config-rendering step. Audit and except the legitimate ones.

rule: Mount /var/run/docker.sock or kubelet socket

Mounting the container runtime socket grants the container effective control over the node.

Warning: Almost always a misconfiguration outside of cluster-management agents. High-priority alert.

Falco operations

5 commands

falco -V

Show version + driver. Falco supports kernel-module, eBPF, and modern eBPF (CO-RE) drivers.

Production note: On modern kernels (5.8+), use the modern eBPF driver — no kernel module compilation, works across distros.

falco --list

List all available filtering fields (proc.name, container.image, fd.name, etc).

Production note: Use this to discover what fields your rule outputs can reference.

falco -c /etc/falco/falco.yaml -r /etc/falco/falco_rules.yaml

Run with explicit config and rules. Default behavior loads all *.yaml under /etc/falco/.

Production note: In Kubernetes, deploy as DaemonSet via the Falco Helm chart; mount custom rules as a ConfigMap.

falco --validate /etc/falco/falco_rules.local.yaml

Syntax-check a rules file without loading. Run in CI on every change.

Production note: Most "rules silently not firing" issues are syntax errors that didn't surface in CI.

falcoctl artifact install ghcr.io/falcosecurity/rules/falco-rules:latest

Manage rule artifacts via the falcoctl tool. OCI-distributed rule sets.

Production note: Pin rule artifacts to specific versions; rule changes can shift signal volume dramatically.

Falco output destinations

3 commands

json_output: true
log_stderr: true

Structured JSON to stderr — for ingestion by fluentd/vector/Loki.

Production note: Also enable `time_format_iso_8601: true` for searchable timestamps.

http_output:
  enabled: true
  url: https://falcosidekick.example.com/

Send alerts to Falcosidekick, which fans out to Slack, PagerDuty, Elasticsearch, S3, etc.

Production note: Falcosidekick is the standard. Don't build your own webhook fan-out.

priority: emergency, alert, critical, error, warning, notice, informational, debug

Falco priority levels. Set per-rule.

Production note: Rule of thumb: critical = page, warning = ticket, notice = audit log only.

Tetragon (Cilium's eBPF-based policy engine)

3 commands

kubectl apply -f tetragon-policy.yaml

Apply a TracingPolicy CRD. Tetragon enforces it at the kernel level via eBPF.

Production note: Unlike Falco (detection), Tetragon can also block/kill processes in real-time.

kind: TracingPolicy
spec:
  kprobes:
  - call: "sys_write"
    selectors:
      matchPIDs: [...]
      matchActions: [{action: Sigkill}]

Kill any process matching selectors that calls sys_write. The "policy as code" model for runtime enforcement.

Warning: Action: Sigkill terminates processes — test thoroughly in dev/staging before production.

tetra getevents -o compact

CLI to stream Tetragon events from the agent. Like falco output but at kernel level.

Production note: Combine with grep / jq for ad-hoc investigation. For production, ship to a log pipeline.

Linux syscalls (the detection alphabet)

6 commands

execve

Process execution. Foundation of "what ran in my container".

Production note: execve + procfs lookups give you the full process tree, working dir, and environment.

openat / open

File opens. "What was read/written" detection lives here.

Production note: Watch for opens of /etc/shadow, /proc/*/maps, ~/.ssh, /var/run/secrets/kubernetes.io.

connect

Outbound network connections. Egress detection lives here.

Production note: Combined with DNS resolution events, you can map "what process connected where" without sniffing packets.

ptrace

Process introspection. Used by debuggers — and by attackers to inject code or read memory.

Warning: Legit ptrace users are few (gdb, strace). Almost any other context is suspicious.

mount

Filesystem mount syscalls. Watch for mounting /proc, /sys/fs/cgroup with rw, or the host root.

Warning: Container escapes often involve manipulating mount namespaces. High-signal syscall.

setns / unshare / clone(NEWNS)

Namespace manipulation. Legitimate uses are rare outside of container runtimes.

Warning: Inside a workload container, these are red flags — possible namespace breakout attempt.

Tuning out noise

3 commands

exception lists per rule

Falco rules support `exceptions` — dimensional allowlists (process, image, command, parent).

Production note: Always tune via exceptions on the existing rule, not by disabling the rule entirely. Document why each exception exists.

aggregate alerts at the alerting layer, not in Falco

Suppression in Falco hides events. Send everything; aggregate/dedupe at the SIEM.

Production note: A weekly review of "top noisy rules" is a healthy ritual.

rate-limit identical events at egress

falcosidekick / fluentd can dedupe identical alerts in a window.

Production note: Without rate limiting, a single bad pod can flood the channel and bury real signals.

Hardened patterns

Common misconfigurations

The unsafe pattern, the replacement, and the reason the two are not equivalent in production.

FIXReview

Risky

# Disable a noisy rule
- rule: Terminal shell in container
  enabled: false

Hardened

# Tune the rule with explicit exceptions
- rule: Terminal shell in container
  exceptions:
  - name: cluster-debug-tools
    fields: [container.image.repository]
    values:
    - my-registry/cluster-debug
  - name: certbot-init
    fields: [proc.name, container.image.repository]
    values:
    - [certbot, my-registry/certbot]

Why it matters: Disabling a rule deletes detection. Exceptions surgically allow known-good cases while keeping the rule firing for everything else. Each exception is documented and reviewable, and reverting a single exception is cheap.

FIXReview

Risky

# Forward Falco JSON straight to Slack channel
http_output:
  enabled: true
  url: https://hooks.slack.com/services/...

Hardened

# Send to falcosidekick, which routes:
http_output:
  enabled: true
  url: http://falcosidekick:2801

# falcosidekick config:
#   slack: critical+warning -> #security-alerts
#   loki: all priorities    -> long-term storage
#   pagerduty: critical only -> on-call

Why it matters: Direct webhook to Slack means every notice/warning becomes Slack noise; on-call gets paged for non-events. Falcosidekick lets you route by priority and destination — critical to PagerDuty, warning to Slack, everything to Loki for forensic search.

FIXReview

Risky

# Run Falco with kernel module driver
falco -k

Hardened

# Use modern eBPF driver (CO-RE)
falco --modern-bpf

Why it matters: Kernel module drivers require compilation against the running kernel — every kernel upgrade can break ingestion. Modern eBPF (CO-RE) compiles once and runs on any kernel ≥5.8, eliminating an entire class of "Falco isn't collecting events after node upgrade" outages.

Go deeper

LINKInternal