Preflight

• 0. Plan approval gate
• 0.5 Resource discovery
• 1. Credentials and region
• 2. Source is Glue Hive Parquet
• 3. Partition locations
• 4. Source prefix
• 5. Destination bucket and namespace
• 6. Lake Formation (stop if registered)
• 7. IAM role and KMS
• Checklist

0. Plan approval gate

A conversation gate, not a CLI check. Follow the approval rule in “Before you migrate” in SKILL.md; it covers every change, including IAM, KMS, and bucket changes. Read-only checks below may run first to build the plan.

0.5 Resource discovery (discover first, then confirm before proceeding)

The only inputs you need from the customer are the source table, the destination table bucket, and the destination table name. Discover everything else, and ask only if discovery fails or is ambiguous:

• Account id and caller: from aws sts get-caller-identity (§1). Do not ask.

• Migration service role: search for an existing one before proposing to create it:

  bashCopy

  aws iam list-roles --query "Roles[?contains(RoleName,'igration') || contains(RoleName,'S3Tables')].RoleName"

  This name match is a coarse first pass (it also matches DMS, MigrationHub, and partner roles). A role is a real candidate only if its trust policy names migration.s3tables.amazonaws.com with aws:SourceAccount/aws:SourceArn; inspect trust and inline policies (see references/iam-roles.md). Present a matching role (ARN, trust principal, how it matches) and confirm before using it. If none exists, offer to create one. If a role is missing a required permission, report what is missing and let the user fix it rather than modifying their role.

• Report bucket: propose an existing general-purpose bucket in the migration region. Do not reuse an Athena-results or other broadly readable bucket; the reports contain schema and data-layout detail. Verify before using it (offer to create one if none passes):

  • Ownership: aws s3api head-bucket --bucket <B> --expected-bucket-owner <ACCT> succeeds.
  • Same region: aws s3api get-bucket-location --bucket <B> returns the migration region. An empty or null LocationConstraint means us-east-1. A cross-region report bucket fails with reportBucketNotSupported.
  • Not public: aws s3api get-public-access-block --bucket <B> shows all four settings true (a NoSuchPublicAccessBlockConfiguration error is a FAIL), and aws s3api get-bucket-policy-status --bucket <B> reports IsPublic: false.
  • Encrypted: aws s3api get-bucket-encryption --bucket <B> shows SSE-S3 or SSE-KMS.
  • Owner enforced: aws s3api get-bucket-ownership-controls --bucket <B> shows BucketOwnerEnforced.
  • TLS only: the bucket policy denies requests where aws:SecureTransport is false. If it does not, read the current policy with get-bucket-policy, add one deny statement, show the user the full result, and apply it only after approval. Never replace the whole policy.
  • Retention: reports contain schema details and object keys. Ask how long to keep them. When your coding agent creates a new report bucket (after approval), it sets all four public-access-block settings, default encryption, BucketOwnerEnforced, a TLS-only bucket policy, and a lifecycle rule that expires report objects after the retention period the user chose.

• Namespace: default to the source database name, normalized: S3 Tables namespaces allow only lowercase letters, digits, and underscores. Lowercase it, replace hyphens and periods with underscores, show the mapping, and confirm (reuse it if it already exists, §5).

Take facts from get-caller-identity without asking. For resources you select or default (role, report bucket, namespace), state what you found and confirm before proceeding.

1. Credentials and region

• Must succeed. RC 255 or empty output usually means expired credentials; refresh and retry.
• The source Glue region, the destination table bucket region, and --region MUST match. Same account and same region is a hard constraint.

2. Source is Glue Hive Parquet

How to know it is a Hive source

All three together:

• Table.TableType = EXTERNAL_TABLE.
• InputFormat / OutputFormat = org.apache.hadoop.hive.ql.io.parquet.MapredParquet{Input,Output}Format.
• SerdeInfo.SerializationLibrary = org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe.

Not a valid source (stop and tell the user):

• An Iceberg table (Parameters.table_type = ICEBERG, or Iceberg format classes). It is already Iceberg.
• A managed table (TableType = MANAGED_TABLE); planning fails with sourceNotSupported.
• Any non-Parquet format (ORC, Avro, JSON, CSV, text).

Unsupported table properties (stop if present)

Check Table.Parameters and SerdeInfo.Parameters from the call above:

Caveat: these fields prove Hive Parquet metadata, not that every data file is valid Parquet. The object compatibility report from planning is the follow-on gate.

• Record the account id as catalogId; it is required in --source.

Lowercase names (capitals break Athena/Glue visibility)

S3 Tables, the Glue Data Catalog, and Lake Formation do not support capital letters in table or column names. A table with uppercase names migrates and reports succeeded, but Athena then fails with GENERIC_INTERNAL_ERROR: ... Invalid table or column names. Confirm the source column names and destination table name are lowercase before Start. Ref: Querying S3 tables with Athena (opens in a new tab).

3. Partition locations

Every partition location must be under the table location. Partitions stored elsewhere (a different prefix or bucket, common after ALTER TABLE ADD PARTITION ... LOCATION) fail planning with UNSUPPORTED_PARTITIONS. Skip this section for an unpartitioned table.

• No output: all partitions are under the table location. Continue.
• Any output: those partitions are outside it. Stop, show the user the locations, and explain that the table cannot be migrated as is. Do not move data or edit partitions for them.
• Record the partition count (get-partitions ... --query 'length(Partitions)'); compare it with planningStats.partitionCount at plan review.
• get-partitions paginates; the CLI follows pages automatically, but very large tables can take a while. Use --segment for parallel listing if needed.

4. Source prefix (let planning classify it)

Start has no exclusion parameter. Do not remove objects from the source prefix; the source is not modified, and markers such as _SUCCESS are read by other jobs.

Planning’s object compatibility report names every object it could not process. Non-data objects (keys with _/. path components, $folder$ markers, zero-byte objects) are recorded as non-data rather than failing the run. If a real data object is flagged, resolve it with the user.

Check one thing up front: a table Location under a _- or .-prefixed path component hides everything beneath it. A non-recursive listing (aws s3 ls s3://<SOURCE_BUCKET>/<PREFIX>/) is enough; do not run aws s3 ls --recursive over a large prefix.

5. Destination bucket and namespace exist

• If the namespace is missing, propose creating it (a missing namespace fails Start with NotFoundException (404)). Creating it is a change, so it is part of the approved plan:

• To validate in Athena, the destination table bucket must be integrated with the Glue Data Catalog; see references/validation.md §2.

6. Lake Formation (stop if registered)

Glue tables registered with Lake Formation are not supported (confirm current support in the migration guide (opens in a new tab)). Planning fails with metadata.glueLakeFormation. Check both the table and its S3 location:

Compare the registered ARNs with the table location (arn:aws:s3:::<bucket>/<prefix>); a registered parent prefix or bucket also counts.

• If either check shows the table is governed by Lake Formation: stop and tell the user the table is not supported.
• Do NOT change Lake Formation settings, deregister locations, or grant permissions to work around this. Those are governance decisions that belong to the customer’s data lake administrators.
• If list-resources is denied, say so and ask the user to confirm the registration status rather than assuming it is not registered.

7. IAM role trust, permissions, and KMS

See references/iam-roles.md: single migration.s3tables.amazonaws.com principal with aws:SourceAccount and aws:SourceArn, scoped permissions, report bucket write, and KMS.

• If the source data or the report bucket uses a customer managed KMS key, the role needs kms:Decrypt (source) and kms:GenerateDataKey (report and destination writes) on that key.
• If the destination table bucket uses SSE-KMS, the key policy must grant maintenance.s3tables.amazonaws.com kms:GenerateDataKey and kms:Decrypt. Without it, creating the encrypted table fails and planning never completes.

Pre-flight checklist (all must pass)

• Not done: plan presented and approved (§0)
• Not done: sts get-caller-identity succeeds; source and destination in the same account and region
• Not done: source is Glue Hive Parquet (EXTERNAL_TABLE, Hive Parquet formats and SerDe); not Iceberg, not managed
• Not done: no transactional, storage_handler, or projection.enabled properties; skip.header.line.count explained
• Not done: catalogId recorded
• Not done: source table and column names are lowercase
• Not done: every partition location is under the table location; partition count recorded
• Not done: table Location not under a _/. path component
• Not done: destination table bucket and namespace exist (or namespace creation approved)
• Not done: destination bucket integrated with the Glue Data Catalog if validating in Athena
• Not done: table not registered with Lake Formation (table flag and data location both checked)
• Not done: migration role trust and permissions verified; report bucket write covered
• Not done: KMS key access and key policy covered if encrypted

• references/preflight.md(opens in a new tab)
• Raw file(opens in a new tab)
• aws/agent-toolkit-for-aws(opens in a new tab)