Skill 61 · AWS Marketplace Metering
Subchapter 61.13
references/setup-path.mdMarkdown29 KBView on GitHub
IMPORTANT: Before proceeding, verify the product is SaaS with usage-based pricing (ExternallyMetered dimensions). Query the Catalog API after getting credentials. If the product is AMI, Container, or SaaS Contract-only, respond that the product type is out of scope and redirect to the public AWS documentation (opens in a new tab).
Ask ALL five questions before generating any code:
Q1: Product code and dimensions?
4ml54db8vrmjuykaw1psroool) and at least one dimension key.Q2: Are you the ISV/seller or building for one?
Q3: Multi-region or single region?
BatchMeterUsage is a regional API — call it in-region from where the usage occurred (for SaaS on another cloud or on-premise, use the nearest AWS region). There is no benefit to routing all usage through one region, and doing so works against data-residency (e.g. EU usage should stay in eu-west-1).Q4: API stage name (mandatory)?
v1, live, or prod? This is your choice and there is no default.”StageName template parameter has no default and an AllowedPattern that rejects an empty value. The seller MUST supply a non-empty value; there is no silent default and no fallback to prod. A direct sam deploy fails fast at the template level if it is missing/empty, and deploy.sh performs the same check (pass the seller’s choice via STAGE_NAME; the fulfillment URL tracks it).Q5: Does a metering stack already exist for this product?
awsmp-* naming — see references/existing-sellers.md.CCP is out of scope (handled at the Product Type Gate above). If the product is Contract with Consumption Pricing, contract-only, AMI, or Container, respond that it is out of scope and redirect to the public AWS documentation (opens in a new tab); do NOT provide CCP implementation steps, code, or API sequences.
Concurrent Agreements is the default. All new products use
CustomerAWSAccountId+LicenseArn. Do not ask the seller — this is the default for all new products.
| Component | Pattern | Example |
|---|---|---|
| Events stack | awsmp-events-stack | Shared across all products, us-east-1 |
| Main stack | <prefix>-metering | awsmp-prod-abc123-metering |
| Prefix | awsmp-<env>-<product-shortcode> | awsmp-prod-u32tk6e2xza22 |
| Feature | Status |
|---|---|
| SaaS Usage (PAYG) | ✅ Supported |
| Concurrent Agreements | ✅ Supported |
| Contract with Consumption Pricing (CCP) | ❌ Out of scope — redirect to public AWS docs (opens in a new tab) |
| Vendor Metered Tagging (VMT) (opens in a new tab) | ✅ Supported |
| Multi-region SaaS runtime | ✅ Supported — BatchMeterUsage is regional; deploy a main stack per metering region and meter in-region (nearest AWS region for on-prem/other-cloud). Metering records are per-region; the same customer/dimension/hour can be metered in each region where that customer’s usage occurred. Do NOT force all of a customer’s usage into one region. |
| Multi-product portfolio | ✅ Supported |
| AMI/Container metering | ❌ Out of scope — redirect to public AWS docs (opens in a new tab) |
| SaaS Contract-only (GetEntitlements) | ❌ Out of scope — redirect to public AWS docs (opens in a new tab) |
Have the seller configure ephemeral credentials in their own environment before you deploy or run tests — do NOT ask them to hand over long-lived IAM user access keys, and do NOT ask for credentials outright as a first step. Preferred order:
aws configure sso && aws sso loginAWS_SESSION_TOKEN)AWS_SESSION_TOKEN) — never long-lived user keysThen verify with aws sts get-caller-identity and confirm the correct seller account before
proceeding. The same ephemeral credentials are used for Catalog API validation, deployment,
and integration testing.
For the ephemeral-first setup commands and the full least-privilege IAM deployer policy (conditioned trust policy + permissions-boundary’d
iam:CreateRole), loadreferences/iam-credentials.md.
After credentials are verified, run this command. Do NOT skip — incorrect values cause runtime failures.
aws marketplace-catalog describe-entity \
--catalog AWSMarketplace \
--entity-id <PRODUCT_ID> \
--region us-east-1 \
--query 'Details' --output text | python3 -c "
import sys, json
details = json.loads(sys.stdin.read())
print('=== Dimensions (use these Key values for metering) ===')
for d in details.get('Dimensions', []):
print(f\" {d['Key']} - {d['Name']} ({d['Unit']}) - Types: {d.get('Types', [])}\")
"Important: The product ID (
prod-xxx) is NOT the product code. Always verify via the Catalog API or AMMP.
sam --version) and AWS CLI v2 (aws --version). Verify the installed versions work; consult the SAM CLI / AWS CLI release notes if a command is unsupported.For the full 6-step deployment walkthrough, load
references/deployment-steps.md.
Summary:
Aggregated + Success)The deployed stacks provide the metering pipeline (discoverer → aggregator → cleanup → submitter; a shared subscription path) that reads usage rows from the usage DynamoDB table and submits them to BatchMeterUsage. However, writing usage rows into the table is entirely the seller’s responsibility.
The seller must build:
The pipeline meters whatever conforms to the schema; it does not distinguish billable from non-billable usage and applies no business filtering. The seller must ensure only metered-eligible usage lands in the table.
Recommended pattern: Durable buffer (SQS/Kinesis) → per-second/hour idempotent aggregator (Lambda/Step Functions) → usage DynamoDB table.
The pipeline reads rows on trust for what it cannot re-derive, and reason-code-rejects (raw-row meteringStatus = RejectedClientSide, UsageRecordRejected metric) what it can check. A row MUST have:
| Attribute | Requirement | If wrong |
|---|---|---|
licenseArn (PK) | The buyer’s LicenseArn from the License Updated event (or, legacy pre-CA, the ProductCode fallback). | Group keyed wrong / not billed for the right buyer. |
customerAWSAccountId_dimension_timestamp (SK) | EXACTLY {customerAWSAccountId}#{dimension}#{YYYY-MM-DDTHH:MM:SS} — whole-second precision in UTC, NOT millisecond/.000Z and NOT local time. The account + dimension segments MUST equal the row’s own customerAWSAccountId / dimension attributes. | Rejected client-side with one code per condition (in order): MalformedSortKey (empty / fewer than 3 #-segments) → SortKeyMismatch (account or dimension segment ≠ the row’s own attribute) → MalformedTimestamp (timestamp segment not exact whole-second, incl. millisecond/fractional or hour-truncated). A local-time timestamp instead buckets in the wrong UTC hour (silently mis-metered). |
customerAWSAccountId, dimension, timestamp | Stored ALSO as separate top-level attributes (the pipeline reads these). dimension present and non-empty; timestamp in UTC. | MissingDimension reject; or discovered under the wrong group. |
meteringPending | Set to the row’s hour bucket YYYY-MM-DDTHH in UTC — this is the GSI HASH that makes the row discoverable, and the pipeline queries it by UTC bucket (now is UTC). It MUST equal the UTC hour of the row’s own timestamp. The pipeline REMOVES it when the row is finalized; the writer sets it only on a new pending row. | Wrong value (incl. a LOCAL-time bucket) ⇒ row is discovered under the wrong hour and silently never metered, or malformed ⇒ never discovered at all. The pipeline cannot catch this — it is verified only by the hands-on test. |
quantity | Non-negative INTEGER (no negative, fractional, or non-numeric). | NegativeQuantity / NonIntegerQuantity / NonNumericQuantity reject. |
usageAllocations (optional, VMT) | If present, exact BatchMeterUsage UsageAllocation shape; all-or-nothing across a group’s non-zero rows; ≤2500 tag-sets; ≤5 tags each; allocations sum to the row quantity. | MalformedAllocations / TooManyTags / TooManyAllocations / AllocationSumMismatch / MixedAllocatedAndUnallocated reject. |
ttl (optional) | Epoch-seconds; only honored if the raw-table TTL is enabled. MUST be far enough out that a row is never TTL-deleted before it is metered (the template validates the floor). | Premature deletion of un-metered usage. |
createdAt, updatedAt (recommended) | ISO-8601 UTC audit timestamps — createdAt set once when the writer first inserts the row, updatedAt refreshed on any rewrite. Audit/reference metadata only (not keyed, not billing-relevant). The pipeline stamps updatedAt when it finalizes a row (and createdAt if absent); the seller SHOULD stamp them at write time for its own rows. | No metering impact; only lost audit trail if omitted. |
The
meteringPendingaccuracy point is the one the pipeline cannot self-correct. If the writer stamps a bucket that disagrees with the row’s timestamp hour, the row is discovered under a bucket whose read-prefix its sort key does not match, so its quantity is silently omitted and its marker is never cleared. There is no client-side reject for it (the row is effectively invisible to the group that would validate it). GettingmeteringPending == floor(timestamp, hour)right is a hard writer obligation — the hands-on test below is how the seller confirms it.
After the two stacks deploy successfully, the skill SHALL make these ongoing seller responsibilities explicit:
AlertsTopicArn); the stack raises them but acting on them is the seller’s job (deployment-steps.md).UsageRecordRejected (writer-contract violations, sliced by Reason), BatchMeterUsageException / CustomerNotSubscribed (terminal, server-side), and UsageAggregationExpired (raw usage aged out before aggregation) / UsageSubmissionExpired (aggregated but not submitted in time).RegistrationUrl); the raw RegistrationUrl is only a bare-bones smoke-test fallback. See references/registration-page.md. (A one-time manual seller step.)TODO: sections.The skill SHALL then walk the seller through the hands-on end-to-end test in references/deployment-steps.md Step 5 — subscribe → verify registration/EventBridge → write a conforming test usage row (correct second-precision sort key + matching meteringPending) → trigger discoverer/aggregator/submitter → verify the raw row finalizes as Aggregated and the aggregated record reaches Success. This exercise is how each writer-contract expectation above is demonstrated and confirmed to be met, and how a mis-stamped meteringPending (otherwise silent) is surfaced.
references/concurrent-agreements.mdreferences/architecture.mdreferences/troubleshooting.mdreferences/existing-sellers.mdCustomerIdentifier instead of CustomerAWSAccountId in UsageRecordsProductCode and LicenseArn for the same customer/hourCustomerAWSAccountId)BatchMeterUsage fails the call with a request-level InvalidUsageDimensionException (server-authoritative; the submitter isolates the record → RejectedClientSide reason InvalidUsageDimensionException; surfaced on the BatchMeterUsageException metric/alarm, not billed). Fix the key or add the dimension in AMMP; no redeploy needed..000Z) instead of whole-second YYYY-MM-DDTHH:MM:SS — rejected as MalformedTimestamp.meteringPending with an hour that doesn’t match the row’s timestamp hour — the row is discovered under the wrong bucket and silently never metered (the one failure the pipeline can’t catch; verify via the Step 5 test).Purchase Agreement Created — that event carries no licenseArn and is not consumed by the metering path. Metering follows License Updated (access granted) and the usage table’s metering_pending GSI.aws-marketplace:BatchMeterUsage + DynamoDB read/writesource: ["aws.agreement-marketplace"]Use ONE unified DynamoDB table for SUBSCRIPTION/AGREEMENT STATE (do NOT split subscription state across separate “customer-profiles” + “subscriptions” tables). Buyer PII / registration-form data is a SEPARATE concern and lives in its own per-Region customer-profile table (below) — that separation is intentional (keeps the subscribers table PII-free), not the old split-subscription-state anti-pattern.
licenseArn = PARTITION key (HASH), customerAWSAccountId = SORT key (RANGE). LicenseArn MUST be the partition key. One buyer with N concurrent agreements = N rows (same customerAWSAccountId, different licenseArn), tracked independently.customerAWSAccountId = partition key (understand the seller’s actual schema first; do not impose the CA layout).productCode, agreementId, agreementStatus (active/inactive — agreement lifecycle), subscriptionStatus (active/deprovisioning/inactive — license lifecycle; no deprovisioned value), registeredRegions (DynamoDB String Set of AWS Region names, added idempotently via ADD, reference-only). It does NOT carry customerIdentifier (deprecated, legacy-only) and does NOT carry buyer PII / registration-form data. Buyer PII lives in the per-Region customer-profile table (awsmp-<productCode>-customer-profile, PK licenseArn + SK customerAWSAccountId, seller-requested GSIs), written in-region by the register Lambda — so the subscribers table stays PII-free and safe in us-east-1 for opt-in-Region products.Scan: customerAWSAccountId-index (register lookup / all agreements for one buyer) and agreementId-index (subscription lookup when an event omits license.arn).Usage table: keyed by licenseArn (PARTITION key) + customerAWSAccountId#dimension#timestamp (SORT key) — the sort-key value is the #-delimited composite {customerAWSAccountId}#{dimension}#{timestamp} at second precision (YYYY-MM-DDTHH:MM:SS), bounding a group at ≤3600 rows/hour. Each item ALSO stores customerAWSAccountId, dimension, and timestamp as top-level attributes so the pipeline reads them without parsing the sort key. It has a sparse metering_pending GSI (HASH = meteringPending hour-bucket, RANGE = licenseArn): the usage writer sets meteringPending when it writes a row, and the pipeline clears it after the row’s aggregation group reaches a terminal result.
Metering is a decoupled, SQS-connected pipeline driven by the usage table’s metering_pending GSI, NOT by registration state — not every seller uses ResolveCustomer (some obtain the LicenseArn via EventBridge or SDDS and never register), so metering must not depend on a registered subscriber. The stages:
metering_pending GSI): queries the GSI for a completed hour bucket and enqueues ONE work message per (licenseArn, customerAWSAccountId, dimension, hourBucket) group onto the work SQS queue. It only meters completed hours in the now-23h … now-1h window (never the in-progress hour) and only once an hour has been closed for MeteringLockHours (default 1). It does NOT read group rows, aggregate, submit, or write back. The same discoverer, on its age-out targets, terminally expires raw rows older than 24h that were never aggregated (REMOVE meteringPending, meteringStatus=AggregationExpired, UsageAggregationExpired metric).begins_with prefix (streamed fold), takes customerAWSAccountId/dimension from the rows (no subscriber lookup), aggregates all second-precision rows into ONE quantity, and MERGES seller-provided usageAllocations. It runs the structural/format client-side validations (dimension present; second-precision sort key with matching segments; non-negative integer quantity; timestamp window; identifier shape; allocation invariants) — dimension NAME validity is left to BatchMeterUsage. A valid group is written as ONE record via conditional PutItem into the awsmp-aggregated-usage table (idempotency commit point); a rejected group is finalized RejectedClientSide with a reason code + UsageRecordRejected metric.meteringPending on the raw rows and stamps meteringStatus=Aggregated. The raw usage table NEVER carries the submission outcome — its meteringStatus domain is exactly Aggregated / RejectedClientSide / AggregationExpired.rate(5 minutes), reserved=1): reads pending records from awsmp-aggregated-usage over the now-23h … now-1h window OLDEST-first, coalesces ≤25 UsageRecords per BatchMeterUsage call, retries UnprocessedRecords once, and writes the returned MeteringRecordId + per-record Status (Success/DuplicateRecord/CustomerNotSubscribed/…) back onto the awsmp-aggregated-usage record only (REMOVE its meteringPending). Records still unprocessed after the retry keep their marker and the invocation FAILS so the Errors alarm fires. It finalizes deprovisioning subscribers after a successful final flush.rate(5 minutes) target, reserved=1): terminally expires a pending awsmp-aggregated-usage record that can no longer be metered — its hour is >24h in the past, OR it is a previous-month record and the month-end grace has closed (on/after 06:00 UTC on the 1st) — REMOVE meteringPending, meteringStatus=SubmissionExpired, UsageSubmissionExpired metric.Zero-quantity groups: while its hour is still inside the window, a group summing to 0 is left pending (not submitted) so the seller can still write usage for that hour — submitting a 0 early would lock the hour at 0 via first-write-wins dedup. At the oldest edge of the window it IS submitted as Quantity: 0 to close the hour out.
Throughput / concurrency: the pipeline uses FIXED concurrency (there is no single meter Lambda and no MeterReservedConcurrency parameter): discoverer ReservedConcurrentExecutions=30, aggregator SQS-ESM MaximumConcurrency: 50 with ReportBatchItemFailures, cleanup SQS-ESM MaximumConcurrency (bounded per-partition writer), submitter and expiry reserved=1. SQS absorbs bursts, so scale comes from queue depth draining rather than raising Lambda concurrency. If a seller needs higher sustained metering TPS than the reserved=1 serial submitter + BatchMeterUsage rate limit allow, advise them to contact AWS Marketplace Seller Operations rather than raising these values unsafely.
The full pipeline above (discoverer → aggregator → cleanup → submitter → submission-expiry) is the default and is for sellers who write RAW per-second usage rows. A seller who already produces finalized hourly aggregated records upstream can instead choose direct-submit mode (DEPLOYMENT_MODE=direct-submit): the skill deploys ONLY the aggregated_usage table + submitter + submission-expiry + the register/subscription path — no raw usage table, no work/cleanup queues, and no discoverer/aggregator/cleanup. The seller writes finalized records straight into awsmp-aggregated-usage:
| Attribute | Requirement |
|---|---|
licenseArn (PK) | buyer LicenseArn (or legacy ProductCode fallback) |
account_dimension_hour (SK) | {customerAWSAccountId}#{dimension}#{YYYY-MM-DDTHH} (hour precision, UTC) |
customerAWSAccountId, dimension, hourBucket, quantity | top-level attributes (hourBucket in UTC) |
meteringPending | the hour bucket YYYY-MM-DDTHH in UTC (makes the record discoverable by the submitter, which queries by UTC bucket) |
Finalize before insert (critical). ANY record present in
aggregated_usageMAY be picked up and submitted on the next 5-minute submitter run. Write a record ONLY when it is FINAL for its(licenseArn, customerAWSAccountId, dimension, hour)— a second write for an already-submitted group returnsDuplicateRecord(first-write-wins) and the later quantity is not billed (under-billing risk). The seller owns the upstream aggregation/idempotency in this mode; the skill provides no raw-row aggregation.
The alarms and health dashboard are created to match the deployed components — direct-submit mode omits the aggregation-stage alarms/widgets (aggregator/cleanup/discoverer, work/cleanup DLQ, UsageAggregationExpired, UsageRecordRejected) and shows a submitter/expiry-focused dashboard.
Mode is not a one-way door. A seller can start in either mode and switch later (change DEPLOYMENT_MODE and re-deploy), and the skill can layer on additional implementation as needs evolve. Switching an actively-metering live stack is a stack UPDATE that adds/removes the mode-specific resources: the raw usage table is Retain (its data survives), but moving to direct-submit REMOVES the work/cleanup queues + discoverer/aggregator/cleanup — confirm the blast radius with the seller before flipping the mode on a live stack.
VMT lets sellers break usage down by seller-defined tags. The allocation breakdown is the seller’s responsibility, provided on each usage row — there is NO enable flag. A usage row participates in VMT simply by carrying a usageAllocations attribute (a list of {AllocatedUsageQuantity, Tags:[{Key,Value}]} objects in the exact BatchMeterUsage UsageAllocation shape). The aggregator MERGES the per-row allocations of a group into the hour’s single UsageRecord (summing identical tag sets); it never decides or derives the tag split.
Rules the usage-writer must follow (the skill explains these at initial integration):
AllocatedUsageQuantity values SHALL sum to that row’s own quantity.TooManyAllocations).(licenseArn, customerAWSAccountId, dimension, hour), either every non-zero row carries usageAllocations or none does. Mixing tagged and untagged non-zero rows is rejected client-side (MixedAllocatedAndUnallocated). Zero-quantity rows are ignored for this check. Different groups in the same hour/batch may independently have or omit allocations.Quantity (else AllocationSumMismatch) — this is automatic if each row’s allocations sum to its quantity.UsageAllocations is omitted.Ask the seller for their real tag keys/values — the packaged code carries no illustrative tag and no VmtEnabled/VmtTagAttribute parameter.
Usage quantities MUST be non-negative integers. The aggregator rejects a negative (NegativeQuantity), fractional/decimal (NonIntegerQuantity), or non-numeric (NonNumericQuantity) quantity client-side — it never truncates (e.g. 2.5→2) or clamps a negative to 0. Aggregate/round in your writer before persisting. A 0 quantity is valid (not a rejection).
ExpiredTokenException → the buyer re-does “Set up your account” in Marketplace for a fresh token (a full resubscribe is NOT required).CustomerAWSAccountId + LicenseArn + ProductCode and append the invocation Region to registeredRegions (idempotent). Do NOT persist CustomerIdentifier for a new integration (deprecated, legacy-only).TimestampOutOfBoundsException, and one bad timestamp rejects the ENTIRE batch.now-23h … now-1h catch-up window recovers a missed/failed run within 24 hours (the discoverer re-enqueues pending hours; the submitter re-drains aggregated_usage). A 6-hour month-boundary grace period applies (previous-month records accepted until 06:00 UTC on the 1st).License Updated → agreementStatus = active and subscriptionStatus = active.License Deprovisioned → subscriptionStatus = deprovisioning (+ a deprovisioningExpiry); OPENS the ~1-hour flush window; the pipeline flushes any remaining aggregated usage, and the events-stack deprovision-cleanup Lambda (rate(15m)) sets subscriptionStatus = inactive once deprovisioningExpiry passes (time-based, not the submitter).Purchase Agreement Ended → agreementStatus = inactive ONLY (does NOT stop metering or flush).Purchase Agreement Amended → update agreement metadata (stays active).Purchase Agreement Created → not consumed by metering; the seller handles it where needed.