Upgrade Engine · Upgrading Mwaa Environments · aws/agent-toolkit-for-aws · Skills Docs

Upgrade Engine · Upgrading Mwaa Environments · aws/agent-toolkit-for-aws · Skills Docs

Deploy the current DAG set to the environment; wait for parsing. 1a. Unpause all deployed DAGs. Newly synced DAGs register as paused by default on MWAA. For each DAG in the deployed set:
```
aws mwaa invoke-rest-api --name <env> --method PATCH \
  --path /dags/<dag_id> --body '{"is_paused":false}'
sleep 0.2
```
(invoke-rest-api has a 10-second timeout and a 6 MB response cap (docs (opens in a new tab)); it is also throttled per environment, so the sleep spaces calls out — prefer exponential backoff on a throttling error over assuming a fixed rate.)

64-char --path caveat (applies to every per-DAG pause/unpause in this skill — step 1a here, the AF2 pause loops below, and the batched switchover in SKILL.md): the per-DAG path /dags/<dag_id> can exceed invoke-rest-api‘s 64-char --path limit for long dag_ids, and the PATCH is then rejected. For an over-length dag_id, pause/unpause via the Airflow CLI (aws mwaa create-cli-token -> airflow dags pause|unpause <dag_id>), which has no path-length limit; on AF3 the bulk collection form --path /dags --query-parameters '{"dag_id_pattern":"<dag_id>"}' also avoids the per-DAG path.
Check has_import_errors is false for all DAGs (invoke-rest-api --path /dags).
Scan logs when the NEXT version jump crosses into a new major version (this version is the required stepping-stone before the crossing): search CloudWatch scheduler and task logs for RemovedInAirflow3Warning / RemovedInAirflow{N}Warning and DeprecationWarning. Combine with the Ruff AIR scan (ruff check --preview --select AIR .) and fix using airflow2-to-3-quick-reference.md and airflow2-to-3-checklist.md. Re-deploy until no RemovedInAirflow{N}Warning remains and imports are clean.
Python-line re-validation: if the just-reached version jump was flagged python_change (it crossed a Python-line boundary, e.g. 3.11 to 3.12), run a dedicated requirements re-validation against the new Python-line constraints file and flag compiled-dependency conflicts before continuing. This is the generalized reason the intermediate step exists (to move Python safely and isolate compiled-dependency risk); it applies whether or not the version jump also crosses a major version.
Same-major version jump with no crossing ahead: scan logs for generic DeprecationWarning best-effort (non-blocking) to surface APIs removed in the target.

Validate at the starting version (version jump 1’s to): set current_jump: 1, jump_status: validating, then run the validate / scan-logs / fix loop (so a resume during this window is unambiguous). If there is no version jump 2, version jump 1’s to is the target — validate and go to Finish.

For each remaining version jump j = 2..N: set requirements to jump_j.to‘s constraint URL (apply provider changes from the quick reference if crosses_major); pause all DAGs before upgrading to prevent the scheduler from firing runs during the transition:

AF3 (bulk):

aws mwaa invoke-rest-api --name <env> --method PATCH \
  --path /dags --query-parameters '{"dag_id_pattern":"~"}' \
  --body '{"is_paused":true}'

"~" is Airflow’s shorthand regex for “match all DAGs”.

AF2 (per-DAG loop, paginated):

# /dags returns up to 100 DAGs per call (default limit) — page through ALL of
# them so every DAG is paused, not just the first 100 (see discovery-preflight.md)
offset=0
while :; do
  dag_ids=$(aws mwaa invoke-rest-api --name <env> --method GET \
    --path /dags --query-parameters "{\"limit\":\"100\",\"offset\":\"$offset\"}" \
    --query 'RestApiResponse.dags[].dag_id' --output text)
  [ -z "$dag_ids" ] && break
  n=0
  for dag_id in $dag_ids; do   # dag_ids are space-free identifiers
    aws mwaa invoke-rest-api --name <env> --method PATCH \
      --path "/dags/$dag_id" --body '{"is_paused":true}'
    sleep 0.2  # invoke-rest-api is throttled per environment
    n=$((n + 1))
  done
  [ "$n" -lt 100 ] && break
  offset=$((offset + 100))
done

Set plan jump_status: upgrading; present update-environment --airflow-version <jump_j.to> for confirmation; execute; poll to AVAILABLE (report every 5 minutes; stop on UPDATE_FAILED/UNAVAILABLE); set jump_status: validating; run the validate / scan-logs / fix loop (which unpauses DAGs in step 1a after validation confirms the new version is healthy); mark the version jump complete and save progress (version jump Status, ## Where we are, Last updated) BEFORE presenting the next version jump; advance current_jump.

For each version jump j = 1..N: set requirements to jump_j.to‘s constraint URL (provider changes if crosses_major); pause all DAGs before upgrading to prevent the scheduler from firing runs during the transition:

AF3 (bulk):

aws mwaa invoke-rest-api --name <env> --method PATCH \
  --path /dags --query-parameters '{"dag_id_pattern":"~"}' \
  --body '{"is_paused":true}'

"~" is Airflow’s shorthand regex for “match all DAGs”.

AF2 (per-DAG loop, paginated):

# /dags returns up to 100 DAGs per call (default limit) — page through ALL of
# them so every DAG is paused, not just the first 100 (see discovery-preflight.md)
offset=0
while :; do
  dag_ids=$(aws mwaa invoke-rest-api --name <env> --method GET \
    --path /dags --query-parameters "{\"limit\":\"100\",\"offset\":\"$offset\"}" \
    --query 'RestApiResponse.dags[].dag_id' --output text)
  [ -z "$dag_ids" ] && break
  n=0
  for dag_id in $dag_ids; do   # dag_ids are space-free identifiers
    aws mwaa invoke-rest-api --name <env> --method PATCH \
      --path "/dags/$dag_id" --body '{"is_paused":true}'
    sleep 0.2  # invoke-rest-api is throttled per environment
    n=$((n + 1))
  done
  [ "$n" -lt 100 ] && break
  offset=$((offset + 100))
done

Set jump_status: upgrading; present update-environment --airflow-version <jump_j.to> for confirmation; execute; poll to AVAILABLE; set jump_status: validating; run the validate / scan-logs / fix loop (which unpauses DAGs in step 1a after validation; scan logs when version jump j+1 is crosses_major); mark the version jump complete and save progress (version jump Status, ## Where we are, Last updated) BEFORE presenting the next version jump; advance current_jump.

Verify health:
```
aws mwaa invoke-rest-api --name <env> --method GET --path /health
```
Expected: metadatabase=healthy, scheduler=healthy.
Confirm has_import_errors is false for every DAG.
On failure, hand off to debugging-mwaa-workflow with context (“upgrade to <target> on <env>; DAGs <list> have import errors”), then re-verify.
Return control to the calling approach. The engine’s scope is parse validation (health + no import errors). Approaches that use the new environment or the test copy for validation (New-environment upgrade, Rehearsed in-place upgrade) are responsible for invoking execution validation (testing-mwaa-workflow) as a required checkpoint before proceeding to the live upgrade or switchover. Parse success alone does not prove artifacts will execute correctly at the target version.