Skill 75 · Migrating Hive Tables To S3 Tables
Subchapter 75.6
references/validation.mdMarkdown5 KBView on GitHub
Confirm the migration actually succeeded. Do not assume from state alone. Run §1 to §6 on the
succeeded table before Complete: the table is queryable then, and a shortfall can still be
reconciled with . After Complete, Sync is refused, so the only post-Complete
check is that shows cleared to null.
sync-table-migrationget-tablemanagedByServiceaws s3tables get-table-migration \
--region <R> --migration-id <ID> \
--query '{state:state,migrated:migrationStats.objectsMigrated,pending:migrationStats.objectsPendingMigration,failed:migrationStats.objectsFailedMigration,failureReason:failureReason}'state + failureReason: expect succeeded with failureReason: null. Complete returns an empty body and there is no active state; after Complete, confirm via get-table that managedByService is null. A non-null failureReason, or a stall in migrating, is the real failure signal.objectsFailedMigration is 0 and objectsMigrated equals planningStats.objectsPlanned. If they differ, check the source object count first (an extra file from an earlier sync is a legitimate difference). Always confirm with the row-count parity in §3, which reads the data itself.Prerequisite: Athena can only query the migrated table once the destination table bucket is integrated with the AWS Glue Data Catalog (the integration uses AWS Glue and Lake Formation). Without it, a migration that succeeded is still invisible to Athena. See Integrating S3 Tables with AWS analytics services (opens in a new tab) and Querying S3 tables with Athena (opens in a new tab).
The migrated table is queried in Athena with the table bucket as part of the catalog
identifier. The catalog identifier is a single level, s3tablescatalog/<table-bucket>
(slash-joined), followed by namespace and table:
Catalog identifier: s3tablescatalog/<table-bucket> (one quoted level, slash-joined)
Fully qualified: "s3tablescatalog/<table-bucket>"."<namespace>"."<table>"In Athena SQL (quote each level):
SELECT * FROM "s3tablescatalog/<table-bucket>"."<namespace>"."<table>";The #1 mistake is writing it dotted (s3tablescatalog.<table-bucket>. ..., which is not valid
Athena) or defaulting to awsdatacatalog (the 3-level catalog.database.table form used for
regular Glue tables). Neither works for S3 Tables.
-- source (regular Glue table, 3-level)
SELECT count(*) FROM "awsdatacatalog"."<db>"."<tbl>";
-- destination (S3 Tables, slash-joined catalog)
SELECT count(*) FROM "s3tablescatalog/<table-bucket>"."<namespace>"."<table>";Counts must match.
Row-count parity is the authoritative success check.
If the migration or a sync was scoped to a subset of the source, compare against that subset, not the full source.
Skip for an unpartitioned table. Compare per-partition row counts on both sides:
SELECT <part_cols>, count(*) FROM "awsdatacatalog"."<db>"."<tbl>" GROUP BY <part_cols> ORDER BY <part_cols>;
SELECT <part_cols>, count(*) FROM "s3tablescatalog/<table-bucket>"."<namespace>"."<table>" GROUP BY <part_cols> ORDER BY <part_cols>;planningStats.partitionCount and the partition count recorded in preflight.Sample a handful of rows from both and compare values, especially timestamp columns (INT96
timestamps are Converted to timestamptz) and any widened numeric types.
Re-confirm every column’s type change in the table compatibility report was a class you accept (Lossless, Widened, or a Converted class you consciously accept); none is Unsupported or an Incompatible change you did not intend. The report spells these lowercase (lossless/widened/converted/incompatible/unsupported), so match case-insensitively.
state succeeded, no failureReasonobjectsFailedMigration; objectsMigrated matches the planned count (or a difference is explained by the source object count and row-count parity passes)unsupported, no unintended incompatible)get-table shows managedByService cleared to null