Advanced

Understanding the distinction between statuses and events is fundamental to building correct automation conditions.

Statuses

Statuses are persistent conditions that remain true for multiple evaluation ticks.

Examples:

• AutomationCondition.missing() - Stays true until the partition is materialized
• AutomationCondition.in_progress() - True while a run is executing
• AutomationCondition.in_latest_time_window() - True for the latest time partition(s)

Characteristic: If the underlying state doesn’t change, the status will be true for consecutive evaluations.

Events

Events are transient conditions that are true only on a single evaluation tick.

Examples:

• AutomationCondition.newly_updated() - True only on the tick when materialization occurs
• AutomationCondition.code_version_changed() - True only on the first tick after code changes
• AutomationCondition.cron_tick_passed() - True only on the first tick after the cron tick

Characteristic: Even if evaluated immediately again, the event would be false (assuming no new change).

Converting Between Status and Event

Status → Event with newly_true():

Use case: Prevent repeated requests during persistent states. A partition stays missing while a run is in progress. Using newly_true() ensures you only request it once.

Two Events → Status with since():

Use case: Create persistent states from transient events. This condition becomes true when an update occurs and stays true until a request is made.

Example: Preventing Duplicate Requests

The default eager() condition uses this pattern:

• newly_missing() and any_deps_updated() are events
• since_last_handled() converts them to a status that persists until the asset is requested or updated
• Without this conversion, the condition would only be true for a single tick, potentially missing the opportunity to launch a run

Run Grouping

Run grouping allows multiple assets to execute in a single run even though downstream assets’ dependencies haven’t been materialized yet.

The Problem

Consider assets A → B → C, all with eager() conditions:

1. A’s upstream updates, triggering A
2. A is requested and begins executing
3. On the next tick, B sees that A hasn’t finished materializing
4. Without run grouping, B would wait for A to complete
5. This results in three separate runs instead of one

The Solution: will_be_requested()

The will_be_requested() operand is true for assets that will be requested in the current tick. Dependency conditions use this to group assets:

When evaluating B:

1. B checks if any dependencies are updated OR will be requested this tick
2. A is marked as “will be requested” this tick
3. B treats A as if it were already updated
4. B is also marked for execution in the same run as A

Requirements for Same-Run Execution

Two assets can execute in the same run if:

1. Same repository: They must be in the same code location
2. Compatible partitions: They must have matching PartitionsDefinition objects
3. Compatible partition mapping: Must use TimeWindowPartitionMapping or IdentityPartitionMapping

If these requirements aren’t met, assets execute in separate runs even with run grouping logic.

Dependency Filtering with allow() and ignore()

Dependency operators (any_deps_match(), all_deps_match()) check conditions on upstream assets. Filtering controls which upstreams are checked.

allow() Creates Intersection

Only dependencies in the selection are checked:

If the asset has 10 upstreams but only 2 are in the “critical” group, only those 2 are checked.

ignore() Creates Subtraction

Dependencies in the selection are excluded:

Updates to “test_data” and “staging_data” won’t trigger the condition.

Propagation Through Operators

When applied to composite conditions (AND/OR), filtering propagates to all sub-conditions:

What gets filtered: All any_deps_match() and all_deps_match() calls

What doesn’t get filtered: Direct operands like missing() on the asset itself

Understanding since_last_handled()

since_last_handled() is a convenience method that converts events to a status:

Behavior:

• Becomes true when condition becomes true
• Stays true until the asset is requested, updated, or the condition is first applied
• Resets on initial evaluation to handle condition changes

Use case: Persist an event until it’s “handled” by either requesting or materializing the asset. This prevents duplicate requests while ensuring the event isn’t lost.

Composite Conditions Deep Dive

any_deps_updated()

Checks if any dependency has newly updated (excluding same-run updates) OR will be requested this tick.

any_deps_missing()

Checks if any dependency is missing AND will NOT be requested this tick. Dependencies that will be requested aren’t considered blocking.

all_deps_updated_since_cron()

For each dependency, checks if it has been updated since the last cron tick. All dependencies must have at least one partition updated since the tick.