Before
- Chase a suspicious label
- Trust one evidence source
- Write alerts without operations
- Patch the visible symptom
Module 17 of 17
Capstone: The Northstar Developer Tool Incident
Investigate a fully synthetic release-integrity incident and deliver detections, response, and prevention.
Start here
Learning objectives
- Plan and complete a safe investigation using only instructor-authored benign artifacts and synthetic telemetry.
- Create an evidence-linked timeline that separates facts, hypotheses, confidence, and missing data.
- Use static observations, endpoint logs, offline network records, release metadata, and component inventory together.
- Produce tested YARA and Sigma detections with false-positive analysis and operational guidance.
- Recommend authorized containment, recovery, and secure-development improvements to prevent recurrence.
- Communicate the incident to engineering and leadership without unsupported attribution or unsafe sharing.
After
- Verify the integrity boundary
- Correlate and expose gaps
- Test detections and response
- Harden the release system
Scenario and Safety Boundary
Northstar is a fictional developer-tool company. Its harmless Build Helper application reads a course project manifest, writes a marker into an allowlisted temporary course directory, and emits synthetic JSON events. The capstone begins when a release-integrity monitor reports that the manifest for version 4.2.1 does not match the approved build record. Every company, person, host, repository, package, signer, domain, address, and hash in the scenario is fictional or reserved for documentation.
The evidence bundle contains plain-text manifests, source-visible toy metadata, synthetic endpoint events, sanitized offline network records, inert YARA fixtures, synthetic Sigma logs, an SBOM excerpt, SLSA-style provenance statements for learning, release approvals, and stakeholder messages. There is no executable download, live sample, credential, persistence, propagation, command-and-control service, destructive behavior, or security-control bypass.
Learners must not add external artifacts, search for a matching real incident, query public reputation services, upload course content to third-party analysis platforms, or test a real environment. If a fixture hash differs or any unexpected active behavior appears, stop and use the provided prerecorded evidence path.
Your Investigation Brief
You are the incident lead working with application, build-platform, identity, and product owners. Leadership needs to know whether the release channel should pause, which fictional versions require review, what evidence supports the decision, how customers can recover safely, and which engineering controls prevent the same failure.
Begin with an investigation plan. Define scope, evidence sources, preservation method, timeline standard, roles, decision authority, communication cadence, and explicit unknowns. Verify the course manifest before reading the evidence. Give each conclusion an evidence reference and confidence statement.
Do not guess a malware family or threat actor. The scenario assesses whether you can reason from system evidence and make proportionate defensive decisions, not whether you can attach a dramatic label.
Evidence Streams
The static stream includes approved and candidate release manifests, signer metadata, an inert toy-marker corpus, component inventory, and provenance statements. The endpoint stream includes synthetic process, file, and identity events. The network stream is sanitized offline JSONL metadata containing only reserved addresses and .invalid names; no replayable packet capture is supplied. The delivery stream includes workflow changes, approval records, staged-release results, and update-channel metadata.
No single stream is complete. Compare clocks, identities, artifact digests, versions, and stated build inputs across sources. Record conflicts rather than selecting the evidence that fits the first hypothesis. Use the provided data-health sheet to distinguish “not observed” from “observed not to occur.”
The final timeline should identify the earliest verified integrity difference, affected trust boundaries, evidence gaps, authorized containment decision, known-good recovery point, and the checks that support reopening the release channel.
Detection Portfolio
Create one YARA rule for the inert course corpus and one Sigma rule for the synthetic event sequence. Each needs a written hypothesis, metadata, fixture or schema provenance, positive and negative tests, false-positive analysis, owner, review date, expected operational use, and rollback. A match must not be described as proof of compromise.
Map the behavior to the current course-pinned MITRE ATT&CK version only where the observable supports the mapping. Pair each mapping with the relevant data source, tested scenario, confidence, and known gap. Use MITRE D3FEND to explore defensive countermeasure concepts, then translate the selected ideas into concrete Northstar controls.
Explain why the two detections are complementary. The file-oriented rule can classify supplied inert content before runtime. The telemetry rule can identify a tested relationship after events exist. Neither replaces provenance verification or authorized response.
Response and Recovery Plan
Compare at least three response options: continue the staged release while observing, pause only the affected channel and version, or pause the broader release capability. Assess risk reduction, business impact, evidence effect, authorization, reversibility, customer consequence, and verification. Choose one and record the decision.
Design eradication around the failed trust boundary shown by the evidence. Recovery must use an approved source revision, expected builder, verified artifact digest, authorized signer, reviewed provenance, bound SBOM, staged health checks, functioning telemetry, and a rollback threshold. Include customer-safe communication that does not expose internal data or overstate attribution.
Finish with a prioritized engineering backlog mapped to SSDF outcomes. Include source and workflow protection, builder trust, release approval, provenance policy, signer authorization, update verification, least privilege, plugin boundary, observability, revocation, and tested rollback where justified by the evidence.
Required Deliverables and Review
The submission contains an evidence manifest verification record, investigation plan, evidence-linked timeline, scope and confidence statement, YARA and Sigma portfolio, ATT&CK and D3FEND mapping, containment decision matrix, eradication and recovery plan, executive brief, technical report, sanitized STIX 2.1 toy bundle with TLP 2.0 rationale, and secure-development remediation backlog.
Peer review should challenge the reasoning, not attempt to discover the “correct villain.” Reviewers trace conclusions to evidence, test whether alternatives were considered, run the inert detection corpora, examine false-positive decisions, and verify that every response action is authorized, reversible where possible, and measurable.
Safety is a release gate. Introducing a real artifact, real target, external callback, secret, exploit, persistence action, credential behavior, propagation mechanism, evasion technique, destructive operation, or unapproved third-party upload results in an immediate stop and an incomplete capstone.
Suggested Study Plan (14 hours)
The capstone duration is assessed work, not playback time. Preserve each deliverable so a reviewer can reproduce the evidence trail and challenge the decisions safely.
| Study block | Time | Required evidence |
|---|---|---|
| Independent capstone investigation | 10 hours | A reproducible defensive case file containing the verified evidence record, timeline, detections, mapping, decision log, recovery plan, reports, sanitized intelligence bundle, and prioritized remediation backlog. |
| Structured review and corrections | 4 hours | Re-run fixture and detection checks, trace challenged claims to evidence, resolve or record disagreements, and complete teardown. |
Evidence Check
Question: The candidate release has a different digest from the approved manifest, but the synthetic endpoint analytic has not alerted. Can the release continue?
Reveal the defensive reasoning
Not on that evidence alone. A digest mismatch is a direct integrity failure that requires the approved release policy to stop or pause promotion while owners verify provenance, signer authorization, build inputs, and artifact identity. The quiet analytic does not override the mismatch, especially until collection health and analytic coverage are verified.
Primary References
Use these primary sources for the current standard or tool behavior. The course records framework versions so mappings can be reviewed when upstream guidance changes.
Real world
Where this shows up
- Leading a cross-functional response to a software release-integrity alert
- Building a detection portfolio with reproducible file and telemetry tests
- Designing a safe recovery path for a developer tool or update channel
- Turning incident evidence into an owned secure-development backlog
- Briefing leadership and engineers from one consistent fact base
Production notes
Keep these close
- Keep the capstone fixture manifest immutable and review every future asset change for safety and licensing.
- Pin framework, schema, rule-engine, and translator versions so learner results remain reproducible.
- Provide prerecorded evidence paths; never require learners to disable protection or connect a lab to public networks.
- Grade reasoning and verification rather than rewarding a predetermined attribution story.
Common mistakes
What usually breaks
- Skipping manifest verification because the assets came from the course
- Choosing a malware family or actor before the evidence supports even an incident scope
- Treating a detection hit or quiet dashboard as conclusive proof
- Restoring a release without correcting signer, provenance, update, or rollback policy
- Putting internal-style detail into a broadly shared executive report
Security risks
Threats to watch
- Learners may try to substitute real suspicious artifacts; the lab must stop and redirect them to authorized responders.
- An external reputation or analysis upload could expose course or employer data and violates the exercise boundary.
- Over-broad response recommendations can cause avoidable product or customer harm.
- Unsupported attribution can mislead leadership and distract from the failed control.
Tradeoffs
Design choices you should be able to defend
Pause only the candidate version
Pros
- Limits immediate customer impact
- Preserves unrelated release work
Cons
- Unsafe if the failed control affects multiple versions
- Needs trustworthy scope
Pause the complete release channel
Pros
- Reduces uncertainty while the trust path is verified
- Simple external posture
Cons
- Higher business impact
- Can delay unrelated security fixes
Continue staged release with observation
Pros
- Provides controlled evidence
- Avoids an immediate broad outage
Cons
- Accepts continued exposure
- Requires explicit authority, strong isolation, and reliable telemetry
Think like an engineer
Questions to answer before shipping
- Which evidence would most strongly disprove your current leading hypothesis?
- What control would have prevented this integrity failure before runtime?
- How can you prove the release channel is healthy instead of merely quiet?
- Which remediation reduces systemic risk across products rather than only this fictional version?
- Can another learner reproduce your detections and decision trail without asking what you meant?
Key terms
Vocabulary used in this module
Integrity incident
An event in which software, metadata, or a delivery process differs from the authorized state.
Known-good
A source, artifact, configuration, or state verified against approved identity, provenance, and policy.
Evidence gap
A known absence or limitation in the information needed to evaluate an incident hypothesis.
Compensating control
A temporary or alternate safeguard used when the preferred control is unavailable, with owned risk and review.
Peer review
Independent examination of evidence, reasoning, tests, safety, and decisions by another learner or reviewer.
Exercises
Practice inside the lesson
Investigate the Synthetic Northstar Release Incident
Complete an end-to-end defensive investigation, detection portfolio, response plan, and engineering remediation using benign course artifacts only.
- Read and sign the capstone safety contract, verify every supplied SHA-256 manifest entry, and stop if any file, hash, or behavior differs.
- Write the investigation plan with authorized scope, roles, evidence handling, clock standard, decision owners, communication cadence, and known unknowns.
- Build an evidence-linked timeline across static, endpoint, offline network, identity, build, and release records; separate facts from hypotheses.
- Define the affected versions and trust boundaries with confidence and alternative explanations; document collection-health limits.
- Create and corpus-test one YARA rule using inert fixtures and one Sigma rule using synthetic logs; document false positives and operational use.
- Map only evidence-supported behavior to the pinned ATT&CK version and select relevant D3FEND countermeasure concepts.
- Compare containment choices, record the authorized decision, and design root-cause eradication plus staged recovery acceptance criteria.
- Create executive and technical reports plus a sanitized STIX 2.1 toy bundle and justified TLP 2.0 marking.
- Prioritize a developer-security backlog using SSDF, SLSA 1.2, signing, provenance, SBOM, least privilege, telemetry, revocation, and rollback controls.
- Conduct peer review, reproduce the detection tests, resolve or record disagreements, delete temporary copies, and retain only the approved defensive submission.
Expected evidence
A reproducible defensive case file containing the verified evidence record, timeline, detections, mapping, decision log, recovery plan, reports, sanitized intelligence bundle, and prioritized remediation backlog.
Assessment criteria
- 25% - Evidence and timeline accuracy: claims trace to verified fixtures, sources and time handling are clear, and gaps are visible.
- 20% - Reasoning and uncertainty: conclusions distinguish facts from hypotheses, evaluate alternatives, and use calibrated confidence.
- 20% - Detection and false-positive quality: YARA and Sigma hypotheses, tests, tuning, metadata, and operational guidance are reproducible.
- 20% - Containment, recovery, and prevention: decisions are authorized and evidence-based, root cause is addressed, and controls are verifiable.
- 15% - Safety, ethics, and communication: all boundaries are followed, information is sanitized, and reports support their intended audiences.
Course-owned resources
Teardown
- Delete all temporary capstone copies, validator output, and scratch notes after review.
- Retain only the approved defensive submission and its manifest verification record.
- Restore the clean course workspace before using the environment for any other task.
Recap
Key takeaways
- A trustworthy investigation traces each conclusion to verified evidence and keeps uncertainty visible.
- File, telemetry, provenance, identity, and release evidence answer different parts of the same incident.
- Detection quality includes negative tests, false-positive decisions, ownership, data health, and rollback.
- Recovery addresses the failed trust path and proves that the known-good release can be restored safely.
- Defensive mastery does not require live malware or usable malicious behavior.
Related resources