Module 7 of 16

Testing and Data Quality

Use tests to catch broken assumptions before users lose trust.

Updated

110 minutes1 exercisesFree

Start here

Learning objectives

  • Use not_null, unique, relationships, and accepted_values tests
  • Write testable assumptions in model YAML
  • Connect data quality to user trust
Testing and Data Quality Follow the arrows. Each box is one idea you will practice in this module. Assumption step 1 Test step 2 Fail step 3 Fix step 4 Trust step 5 Production analytics engineering turns raw records into governed, trusted business meaning.

The Mental Model

Data tests are executable assumptions. They do not prove data is perfect, but they catch the breakages you know would make the model unsafe.

A test is a smoke alarm. It does not stop every fire, but it tells you when a known danger is happening.

Dataset reference: ecommerce tables and grain assumptions.

Turn a key assumption into a failing test

A dimension containing customer IDs 7, 7, and NULL should fail both uniqueness and non-null checks. This dbt model properties file expresses those assumptions:

version: 2
models:
  - name: dim_customers
    columns:
      - name: customer_id
        data_tests:
          - unique
          - not_null

Run dbt test --select dim_customers after building the model. A test failure means the assumed key contract is broken; it does not prove which upstream source caused it. Add a relationship test from orders to customers separately if referential integrity is required. See dbt data tests.

Acceptance check: deliberately duplicate a key in a development fixture and confirm the test fails before relying on it as a release gate.

Interactive Check

Question: Which test should protect customer_id in dim_customers?

Reveal the answer

Use unique and not_null. A customer dimension needs exactly one non-null row per customer_id.

Practice: Add the First Tests

Add basic dbt-style tests to fct_orders and dim_customers.

Use the guided lab below to record your result, assumptions, and the check that would catch an incorrect result.

Production notes

Keep these close

  • Every model should have at least one grain-protecting test. For facts, test the event key. For dimensions, test the entity key.

Common mistakes

What usually breaks

  • Testing only columns that already look clean
  • Adding hundreds of noisy tests nobody investigates
  • Treating warnings and failures without a clear policy

Key terms

Vocabulary used in this module

Data test

A check that validates an expected property of a dataset.

Relationship test

A test that checks whether foreign key values exist in a referenced table.

Exercises

Practice inside the lesson

30-45 minutesBeginner to Intermediate

Add the First Tests

Add basic dbt-style tests to fct_orders and dim_customers.

  1. Mark customer_id in dim_customers as unique and not_null
  2. Mark order_id in fct_orders as unique and not_null
  3. Add accepted_values for order_status
  4. Add a relationship test from fct_orders.customer_id to dim_customers.customer_id

Expected evidence

A short answer, SQL/YAML snippet, or lineage map that can live directly in the course page notes.

Recap

Key takeaways

  • Tests turn assumptions into automated checks
  • Basic tests catch many expensive dashboard failures
  • Quality is an engineering workflow, not a cleanup sprint

Related resources

Keep learning across CodersSecret