Strategy Blue Green Fresh

Plan durability: plan maintenance, self-consistency invariants, and resume reconciliation follow plan-materialization.md. Save progress after every step below - status token plus durable facts (run_ids, ARNs, S3 versions, reasons) plus Last updated and ## Where we are - before proceeding to the next action.

Prerequisites

• Approach recorded as new-environment in the plan.
• New environment name and S3 paths determined in the plan.
• Upgrade path determined in the plan.

Execution Steps

1. Create the new environment

Create the new environment at the first version jump’s target (jump_1.to): the target directly when the upgrade path has a single version jump, or the latest 2.11.x stepping-stone version when the path requires one. Match ALL configuration from your current environment: environment class, VPC, subnets, security groups, execution role, Airflow configuration overrides, logging configuration, weekly maintenance window, webserver access mode, and capacity settings (workers, schedulers, webservers). The new environment owns independent S3 paths for every mutable artifact — DAGs, requirements, plugins, and startup script — each seeded from your current environment’s version (Step 1a) so the new environment can diverge without touching your current environment; the engine advances requirements per version jump toward the target (at create, the new environment’s requirements matches the starting version jump_1.to, not the target; see upgrade-engine.md), and DAGs receive the fixed code. Explicit user confirmation required.

1a. Seed the new environment’s artifacts from your current environment. Copy your current environment’s in-use plugins and startup script to the new environment’s paths verbatim — pin the exact PluginsS3ObjectVersion / StartupScriptS3ObjectVersion from Phase 1 discovery with s3api copy-object; on an unversioned bucket omit ?versionId=, and skip an artifact your current environment does not use:

Requirements is not copied verbatim — your current environment’s is pinned to the source version and would conflict on the starting-version environment. Build <new-env-requirements-path> from your current environment’s requirements so create-environment does not fail late on a pip resolution:

1. Constraint URL: if your current environment’s file has a --constraint line, re-point it to jump_1.to‘s Airflow+Python constraint file (URL template in mwaa-version-matrix.md); if it has none and jump_1.to is >= 2.7.2, add it.
2. Pin alignment: keep explicit == pins. For each constraint-governed package (airflow, providers, core deps), update the pin to the version in jump_1.to‘s constraint file; leave third-party packages (not in the constraint file) at your current environment’s pin. If a package genuinely needs a version newer than the constraint, modify the constraint file for that package (download it, edit that line, save it to the new environment’s DAGs folder, reference it) rather than leaving a pin that fights the constraint.
3. Verify (Phase 6 Docker, if available): build the jump_1.to image with amazon-mwaa-docker-images and confirm the requirements, startup script, and plugins install cleanly; fix conflicts and retry until clean. If Docker is unavailable, rely on the constraint cross-reference above.

The engine advances the constraint per version jump toward the target after create (see upgrade-engine.md). DAGs are written to <new-env-dag-path> by the engine (fixed code). Each of the new environment’s S3 keys is dedicated, so create-environment references the new environment’s paths without pinning an object version. Then create the new environment (only after the requirements verify clean):

Omit any path/version flags for artifacts your current environment does not use. Poll until AVAILABLE (may take up to a couple of hours):

2. Migrate Operational Metadata via REST API

No historical run data is migrated. Migrate only operational config:

Secrets caveat — read before migrating. Raw GET->POST is incomplete and sensitive for secret-bearing config:

• Connection passwords do NOT round-trip. GET /connections omits the password in Airflow 2.x (schema load_only) and masks it in 3.x (redacted validator), so the POST below recreates a passwordless connection. Treat a migrated connection as incomplete: point it at a Secrets Manager / SSM secrets backend (preferred — the secret never touches the API) or re-enter the password out of band, and verify each connection on the new environment before switchover.
• The values that do transit are sensitive. Variable values (returned cleartext by GET /variables) and connection extra land in the --body argument, visible in process args, shell history, and the MCP-server / CloudTrail audit log. Prefer a secrets backend so they never traverse the API/CLI; if inline migration is unavoidable, treat them as secrets and do not echo them.

Paginate every read. GET /variables, /connections, and /pools each return at most 100 items per call (the Airflow REST API’s default limit), so reading once silently migrates only the first 100 and drops the rest on an environment that has more than 100 of any of them. Pass --query-parameters '{"limit":"100","offset":"0"}' and re-issue with offset incremented by 100 until you have read total_entries items — the same limit/offset loop discovery-preflight.md Step 1.4 uses for /dags — POSTing every page to the new environment.

Variables (skip if secrets backend handles these):

Connections (skip if secrets backend handles these):

Pools (skip default_pool):

3. Run the step-by-step upgrade on the new environment

Run the step-by-step upgrade on the new environment; starting condition: newly created. See upgrade-engine.md. The engine validates at the starting version and upgrades through the remaining version jumps to the target.

4. Deploy Fixed DAGs in Batches

Two-phase deployment (multi-jump paths only): When the upgrade path has more than one version jump (e.g., 2.7.2 -> 2.11.2 -> 3.2.1), DAG deployment is split into two phases because AF3-only imports (from airflow.sdk import ...) fail to parse on AF2.x environments:

• Phase A (first version jump validation at starting version): Deploy the ORIGINAL AF2-compatible DAGs. These are the unmodified source DAGs (or minimally patched for the intermediate version). Validate that all DAGs parse and run on the starting version before proceeding to the second version jump.
• Phase B (after the second version jump completes at target version): Deploy the FIXED DAGs (Ruff-applied, airflow.sdk imports, Asset renames, etc.). These contain AF3-only code and can only be deployed once the environment is running the target 3.x version.

For single-jump paths (source is already 2.11.x, target is 3.x), only Phase B applies — deploy fixed DAGs directly after the single version jump completes.

For each batch (0..N):

• Upload batch DAGs to the new environment:

  Copy

  aws s3 cp <dag-file>.py s3://<bucket>/<new-env-dag-path>/

• Wait for parsing (poll /dags up to 5 min).

• Verify has_import_errors is false:

  Copy

  aws mwaa invoke-rest-api --name <new-env> --method GET --path /dags/<dag_id>

• On import errors, hand off to debugging-mwaa-workflow; fix, redeploy, re-verify. Loop within the batch until parsing is clean.

• Update plan: mark batch DAGs deployed.

This step establishes clean parsing only — test-run validation is owned by the required checkpoint in Step 4-checkpoint, not here.

4-checkpoint. Test-run the new environment (REQUIRED)

Parse validation (no import errors) is necessary but NOT sufficient. Before switchover (Phase 8), invoke skill testing-mwaa-workflow on the new environment to confirm DAG execution succeeds at the target version. At switchover the new environment becomes the live environment; a DAG that only parses but fails at runtime will fail on the new environment once your current environment is paused and no longer serving it.

• Invoke skill testing-mwaa-workflow with delegated mode — do NOT trigger DAG runs manually via the REST API (POST /dags/{id}/dagRuns) or poll run states inline. The testing skill owns triggering, polling, classification, and the fix loop.
• Every DAG must reach terminal SUCCESS or be classified as blocked on an infrastructure/data dependency unrelated to the Airflow version (NEEDS-DECISION items from the testing skill’s fix loop).
• On artifact failures (wrong operator params, missing imports, incompatible API usage): invoke skill debugging-mwaa-workflow + authoring-mwaa-workflow, redeploy to the new environment, retest until SUCCESS.
• Do NOT begin the batch-by-batch switchover until test-run validation passes. Parse success alone does not prove artifacts execute correctly at the target version.

Record execution results in the plan under a ## Test-run results section (DAG ID, run_id, state, duration, and any NEEDS-DECISION items).

5. Proceed to Phase 8

All batches test-run verified (Step 4-checkpoint), operational metadata migrated. Ready for switchover.