Subchapter 76.2
references/multi-transformation.mdMarkdown10 KBView on GitHub
Apply TDs to multiple repositories in parallel. TD-to-repo assignments and config are already confirmed from the match report. Do NOT re-discover TDs or re-prompt.
From the match report: repo list, TD per repo, config per TD, execution mode.
Verify AWS credentials ONCE. Do NOT repeat per repo.
aws sts get-caller-identityLocal mode also: atx --version
If any repos were provided as git URLs (HTTPS or SSH), clone them locally first. The user’s local git config handles authentication — no Secrets Manager needed.
CLONE_DIR=~/.aws/atx/custom/atx-agent-session/repos/<repo-name>-$SESSION_TS
git clone <git-url> "$CLONE_DIR"If repos were provided as an S3 bucket path with zips, download and extract locally:
mkdir -p ~/.aws/atx/custom/atx-agent-session/repos
aws s3 sync s3://user-bucket/repos/ ~/.aws/atx/custom/atx-agent-session/repos/ --exclude "*" --include "*.zip"
for zip in ~/.aws/atx/custom/atx-agent-session/repos/*.zip; do
name=$(basename "$zip" .zip)
unzip -qo "$zip" -d "$HOME/.aws/atx/custom/atx-agent-session/repos/${name}-$SESSION_TS/"
doneUse the cloned/extracted paths as <repo-path> for each repo.
For each repo, verify it’s a git repo:
ls -la <repo-path>
git -C <repo-path> statusIf not a git repo: cd <repo-path> && git init && git add . && git commit -m "Initial commit"
The active language runtime must match the transformation’s target version so that
builds and tests run correctly. Check the current version, and if there is a
mismatch, first check whether the target version is already installed (e.g.,
/usr/libexec/java_home -V 2>&1 (macOS) or ls /usr/lib/jvm/ (Linux), pyenv versions, nvm ls). If found, switch
to it (e.g., export JAVA_HOME=<path to JDK> && export PATH="$JAVA_HOME/bin:$PATH", pyenv shell 3.12, nvm use 22). Only if
the target version is not installed at all, ask the user for permission before installing. Suggest:
brew install --cask corretto23 (macOS), sudo yum install java-23-amazon-corretto-devel (RHEL/AL2), or sudo apt install java-23-amazon-corretto-jdk (Debian/Ubuntu)pyenv install 3.15.0 && pyenv shell 3.15.0nvm install 23 && nvm use 23Do NOT proceed until the correct version is active. Verify the switch succeeded before proceeding.
When running atx custom def exec, always include the --telemetry flag (see the Telemetry section in SKILL.md). Format:
--telemetry "client=<client>,agent=<agent>,executionMode=<local|remote>"
client is the MCP client or tool hosting this session (lowercase, no spaces) — e.g., kiro, vscode, cursor, windsurf, claudecode. Use the real tool name, not a default.agent is the AI assistant driving this session (lowercase, no spaces) — e.g., kiro, amazonq, claude, copilot, cline, codex. Use the real assistant name, not a default.executionMode is local for direct CLI invocation, remote when submitting via LambdaRun transformations in parallel — maximum 3 concurrent repos at a time (the user can override this, but 3 is recommended to avoid overloading the machine). If there are more than 3 repos, process them in batches of 3 (wait for a batch to finish before starting the next). Maximum 9 repos total for local mode (user can override, but recommend remote mode for more). If the total repo count exceeds 9, suggest remote mode instead.
For each repo, use bash to create a runner script that captures the exit code, following this exact format:
mkdir -p ~/.aws/atx/custom/atx-agent-session
cat > ~/.aws/atx/custom/atx-agent-session/run-<repo-name>.sh << 'RUNNER'
#!/bin/bash
atx custom def exec -n <td-name> -p <repo-path> -x -t \
--configuration 'additionalPlanContext=<config>' \
--telemetry "client=<client>,agent=<agent>,executionMode=local"
echo $? > ~/.aws/atx/custom/atx-agent-session/<repo-name>.exit
RUNNER
chmod +x ~/.aws/atx/custom/atx-agent-session/run-<repo-name>.sh
nohup ~/.aws/atx/custom/atx-agent-session/run-<repo-name>.sh > ~/.aws/atx/custom/atx-agent-session/<repo-name>.log 2>&1 &
echo $! > ~/.aws/atx/custom/atx-agent-session/<repo-name>.pidOmit --configuration if no config needed. The --telemetry flag is always included — see the Telemetry section above for field values. Launch each repo’s script in rapid
succession — do NOT wait between launches. Each runner script is backgrounded
via nohup; the exit code is captured to ~/.aws/atx/custom/atx-agent-session/<repo-name>.exit when ATX finishes.
After launching all repos, find each repo’s conversation log by grepping its process log (ATX outputs the path within 30-60 seconds of starting):
grep "Conversation log:" ~/.aws/atx/custom/atx-agent-session/<repo-name>.log 2>/dev/nullIf it hasn’t appeared yet, wait 15 seconds and retry. Extract the full path from
each — do NOT use ls -t across all conversations, as that may match a different run.
Then start monitoring. On each 60-second cycle:
kill -0 $(cat ~/.aws/atx/custom/atx-agent-session/<repo-name>.pid) 2>/dev/null && echo "RUNNING" || echo "DONE"You MUST continue polling without waiting for user input. The user should see continuous progress updates across all repos.
A repo’s transformation is done ONLY when its background process exits (i.e.,
kill -0 returns non-zero). Do NOT treat exit code 0 from any other command
(grep, cat, test, ls, etc.) as transformation completion. Do NOT treat log
messages like “TRANSFORMATION COMPLETE” as completion — ATX performs additional
steps after that (validation summary generation).
Prepare each repo’s source before submitting the batch. Follow the source prep
rules from single-transformation.md: HTTPS and SSH git URLs (with credentials
configured) are passed directly; S3 zips from the user’s bucket must be copied
to the managed source bucket (atx-source-code-{account}) first; local repos
must be zipped and uploaded to the same managed bucket.
Submit jobs via the batch Lambda in chunks of up to 128. If there are more than
128 jobs, split them into multiple atx-trigger-batch-jobs calls (e.g., 500 repos
= 4 calls of 128 + 128 + 128 + 116). Each call returns its own batchId. Track
all batch IDs for monitoring.
Include the environment field on each job to set the language version matching the transformation’s target (e.g., "JAVA_VERSION":"21" for a Java upgrade targeting 21). Include --telemetry in each job’s command string always (see Telemetry section):
aws lambda invoke --function-name atx-trigger-batch-jobs \
--payload '{"batchName":"<name>-chunk-1","jobs":[{"source":"<url>","command":"atx custom def exec -n <td> -p /source/<project> -x -t --telemetry \"client=<client>,agent=<agent>,executionMode=remote\"","jobName":"<name>","environment":{"JAVA_VERSION":"<target>"}}]}' \
--cli-binary-format raw-in-base64-out /dev/stdoutIf the total exceeds 128, repeat with the next chunk:
aws lambda invoke --function-name atx-trigger-batch-jobs \
--payload '{"batchName":"<name>-chunk-2","jobs":[...next 128 jobs...]}' \
--cli-binary-format raw-in-base64-out /dev/stdoutMonitor each batch by its batchId:
aws lambda invoke --function-name atx-get-batch-status \
--payload '{"batchId":"<batch-id>"}' \
--cli-binary-format raw-in-base64-out /dev/stdoutPolling: every 60 seconds for the first 10 polls, then every 5 minutes after. Report only on status change.
[1/N] repo-name TD-name Status
[2/N] repo-name TD-name StatusCollect per repo: success/failure, transformed code path, error details.
Succeeded:
- repo-name: TD-name (config)
Failed:
- repo-name: TD-name (error)For remote executions, include the CloudWatch dashboard link in the final output:
REGION=${AWS_REGION:-${AWS_DEFAULT_REGION:-$(aws configure get region 2>/dev/null)}}
REGION=${REGION:-us-east-1}
echo "https://${REGION}.console.aws.amazon.com/cloudwatch/home#dashboards/dashboard/ATX-Transform-CLI-Dashboard"Hand off to results-synthesis.md for consolidated reporting.
| Scenario | Action |
|---|---|
| Git clone fails | Log error, continue with remaining repos |
| Transformation fails | Log repo and error, do not auto-retry |
| Partial results | Generate summary from successes, report failures |
Clean up session files before starting and after completing each batch:
[ -d ~/.aws/atx/custom/atx-agent-session ] && find ~/.aws/atx/custom/atx-agent-session -maxdepth 1 -type f \( -name "*.sh" -o -name "*.log" -o -name "*.pid" -o -name "*.exit" -o -name "*.zip" \) -delete 2>/dev/null || trueFor remote mode: after presenting results, also prompt the user about infrastructure teardown. See the Cleanup section in remote-execution.md for the exact prompt and flow.
atx-trigger-batch-jobs call; split larger sets into multiple calls (max 512 repos per session)