Subchapter 65.1
references/cost-model-schema.mdMarkdown24 KBView on GitHub
The cost calculator at ${SKILL_DIR}/scripts/calculate_costs.py reads a JSON file describing the data model and access patterns, and writes a monthly-cost report in Markdown. This document specifies the JSON shape. (${SKILL_DIR} is defined in SKILL.md “Resolving the skill’s own paths” — substitute the absolute path of the directory containing SKILL.md.)
Scripts
Benchmark LambdaThe agent generates this JSON from its own in-context design — it is not authored by hand. Everything the calculator needs (RPS, item size, operation, consistency, GSI projection) is already part of a properly produced access-pattern list and schema (per Mechanics #2 and #18). Assembling the JSON is a serialization step, not a separate design exercise.
{
"tables": [ /* Table[] */ ],
"access_patterns": [ /* AccessPattern[] */ ]
}Only these two arrays are required. Everything else is optional and has sensible defaults.
{
"table_name": "Orders",
"table_class": "STANDARD",
"global_tables": false,
"key_schema": {
"partition_key": "customer_id",
"sort_key": "order_id"
},
"entities": [ /* Entity[] */ ],
"gsis": [ /* Gsi[] */ ],
"vector_indexes": [ /* VectorIndex[] */ ]
}| Field | Required | Default | Notes |
|---|---|---|---|
table_name | yes | — | Must match the table value on every access pattern that targets it. |
table_class | no | "STANDARD" | "STANDARD" or a non-standard value. Storage is priced at the full public Standard rate either way — the 25 GB free tier is not applied (it is account-wide and often already consumed). |
global_tables | no | false | If true, adds 48 bytes per item to storage calculations. |
key_schema | no for calculator, yes for live deploy | — | {"partition_key": "<attr>", "sort_key": "<attr>?"}. The calculator ignores this; it sizes items generically. The live-deploy path (${SKILL_DIR}/scripts/deploy_model.py) requires it to build CreateTable kwargs and refuses with a clear error if any table is missing it. Populate from the schema you already derived — the attribute names are the ones you chose per Data modeling #7. key_schema names the key attributes but does NOT carry their type — the deploy resolves each key’s AttributeType (S/N/B) by looking the name up in entities[].attributes[] or attribute_definitions (below). A key whose type is not declared anywhere deploys as S — so a numeric (epoch sort key, numeric id) or binary key MUST be declared N/B, or real writes fail with ValidationException in production. |
attribute_definitions | no | [] | Optional table-level list declaring key attribute types, in the raw-CreateTable-API shape: [{"attribute_name": "order_date", "attribute_type": "N"}]. The {"name","type"} shorthand is also accepted. Read by both deploy_model.py and scripts/benchmark_lambda.py. An entry here wins over an entities[].attributes[] type for the same attribute. Use this when you’d rather declare key types at the table level than inside an entity; declaring them in entities[].attributes[] is equally valid. |
entities | no | [] | Used (a) to size attributes for query projection caps, (b) as a storage fallback when no write pattern targets the table, and (c) as a source of key attribute types (attributes[].type) for the live deploy. |
gsis | no | [] | Drives write amplification and storage sizing. |
vector_indexes | no | [] | Vector indexes on the table. Drives vector write capacity and vector index storage, and is required for a SearchVectors access pattern to resolve. See Vector index below. |
{
"entity_name": "Order",
"estimated_item_size_bytes": 2048,
"estimated_item_count": 500000,
"attributes": [
{ "name": "order_id", "type": "S" },
{ "name": "status", "type": "S" },
{ "name": "total_cents", "type": "N" }
]
}| Field | Required | Default | Notes |
|---|---|---|---|
entity_name | yes | — | Free-form identifier; used only in cross-references. |
estimated_item_size_bytes | no | 1024 | Used in the storage fallback path. Excludes vector attributes — declare the item size without its embeddings and let the calculator derive those from dimensions, or an ALL-projection vector index double-counts them. See Vector capacity is metered in BYTES. |
estimated_item_count | no | 100,000 | Used in the storage fallback path. |
attributes | no | [] | Each attribute has a name and type. Types map to sizes: S=100, N=8, BOOL=1, B=256, L=200, M=200, SS=200, NS=200, BS=200, NULL=1. These sizes are heuristics — use them as-is unless you have a stronger reason. For live deploy this is also where key attribute types come from: every attribute named in key_schema (partition/sort) or in any GSI key MUST appear here (or in the table’s attribute_definitions) with its true type, or the deploy defaults it to S. |
Entities are used mainly when access patterns specify a projection list. The calculator caps Query/Scan items_per_request at the 900 KB page limit based on the sum of projected attribute sizes. If you never use projection, the entity list is only consulted for the storage fallback.
Key attribute types are load-bearing for live deploy. DynamoDB types only key attributes (PK/SK on the table and every GSI); non-key attributes are schemaless. The deploy builds each table’s
AttributeDefinitionsby looking up every key attribute’s type inentities[].attributes[]and the table-levelattribute_definitionsblock (the latter wins on conflict), defaulting toSfor any it can’t find. A key declared (or defaulted) to the wrong type is the one schema error DynamoDB enforces at write time: a numeric epoch sort key left asSaccepts the seed but rejects real integer writes withValidationException. So whenever a key is an epoch timestamp, a numeric id, or binary, declare itN/B— in either place — and the benchmark/seed path (which reads the same two sources) will generate matching values automatically.
{
"index_name": "OrdersByCustomer",
"partition_key": "customer_id",
"sort_key": "created_at",
"projection": {
"type": "ALL"
}
}With an INCLUDE projection:
{
"index_name": "OrdersByStatusSparse",
"partition_key": "status_active",
"sort_key": "created_at",
"projection": {
"type": "INCLUDE",
"attributes": ["order_id", "customer_id", "total_cents"]
}
}| Field | Required | Default | Notes |
|---|---|---|---|
index_name | yes | — | Must match the index value on any access pattern that targets it. |
partition_key | no | — | Attribute name of the GSI PK. Used for sparse-GSI detection: if an access pattern provides attributes_written and the GSI’s PK is not in that list, amplification is skipped for that write. |
sort_key | no | — | Attribute name of the GSI SK. Used together with INCLUDE/KEYS_ONLY projection checks on UpdateItem. |
projection.type | no | "ALL" | "ALL", "INCLUDE", or "KEYS_ONLY". Drives both write-amplification size and storage size. |
projection.attributes | no | [] | Only for INCLUDE projections. The list is used in UpdateItem amplification checks — if none of the written attributes intersects this list, amplification is skipped. |
Projection ratios (applied to base item size):
ALL → 1.0 (full item written to index; full base-table storage footprint)INCLUDE → 0.3 (conservative default when the include list is unknown)KEYS_ONLY → 0.1 (keys + SK only)GSI sparseness (write amplification AND storage). A DynamoDB item is written to — and stored in — a GSI only if it carries that GSI’s key (the partition key, plus the sort key if the GSI is composite). This is verified live against
ReturnConsumedCapacity=INDEXES: an item missing a composite GSI’s sort key fired zero writes to it. In a single-table design with heterogeneous entities, each GSI is keyed on an attribute only some entities have, so most items belong to only one or two of the table’s GSIs — not all of them. The calculator models this: write amplification usesentities_written(above) to bill only the member indexes; GSI storage is sized by the byte-share of entities that carry the index’s key, not the whole base table. When the model doesn’t declare enough to establish membership (noentities_written, or a GSI key declared on no entity), the calculator stays at the conservative full-fan-out / full-size upper bound rather than guessing low.
{
"index_name": "ProductEmbeddingIndex",
"vector_attribute": "Embedding",
"dimensions": 1024,
"distance_function": "COSINE",
"projection": { "type": "INCLUDE", "attributes": ["Title"] },
"search_schema": { "partition_key": "Category", "inline_filters": ["Brand"] },
"estimated_indexed_items": 40000000
}| Field | Required | Default | Notes |
|---|---|---|---|
index_name | yes | — | Must match the index value on any SearchVectors access pattern targeting it. |
vector_attribute | yes | — | Attribute holding the embedding. Indexes sharing an attribute MUST agree on dimensions — DynamoDB rejects otherwise with Attributes cannot be redefined. |
dimensions | yes | — | 1–4096. Fixed at creation; changing it means a new index and a migration. |
distance_function | yes | — | COSINE, EUCLIDEAN, or DOT_PRODUCT. Fixed at creation. |
projection.type | no | "ALL" | ALL, INCLUDE, or KEYS_ONLY. This is the dominant cost lever — see below. Fixed at creation. The bare-string shorthand "projection": "KEYS_ONLY" is accepted and is equivalent to {"type": "KEYS_ONLY"}, the same way attribute_definitions accepts the {"name","type"} shorthand. |
projection.attributes | no | [] | For INCLUDE. Sized from entities[].attributes[] where declared, else 100 B each. |
search_schema.partition_key | no | — | The vector index partition key (HASH). Must also appear in attribute_definitions or entities[].attributes[], exactly as a GSI key must. An item missing this attribute is written to the base table without error and silently excluded from the index — see SKILL.md Fact #10 and vector-search.md. |
search_schema.inline_filters | no | [] | Up to 18 INLINE_FILTER attributes, optional at search time. |
estimated_indexed_items | no | sum of entity counts | Items actually replicated into the index — i.e. those carrying the vector attribute (and the partition key, if defined). Declare it when only a subset of the table carries embeddings; otherwise the calculator uses the table’s whole population, which is a conservative upper bound and is flagged in the report with *. |
The calculator rejects a model where a vector-index table declares provisioned_capacity,
where dimensions or top_k are out of range, where more than 5 vector indexes are declared
on one table, or where indexes on the same attribute disagree on dimensions. These are
service constraints, so a violation means the design cannot be deployed — better to fail than
to price something impossible.
Vector indexes are billed on three dimensions, separately from base-table RCU/WCU. Rates are
from the AWS Pricing API (us-east-1 Standard); Standard-IA is 125% on requests, 40% on storage.
Rates change and vary by region: these are the values the calculator ships with, and before
quoting a dollar figure to a user in another region — or a long time after this was written —
confirm them against aws pricing get-products --service-code AmazonDynamoDB or the DynamoDB
pricing page. The shape of the model below is what matters and does not change with the rates.
| Dimension | Rate | How the calculator treats it |
|---|---|---|
| Index storage | $0.25/GB-month (rolls into table storage) | Derived from the design |
VectorWriteRequest | $0.52/GB | Derived — validated to within 2.1% against live measurement |
VectorSearch | $0.002/GB | Not priced. Drivers reported instead — see below |
Write cost, measured semantics (controlled probe: two indexes on the same attribute and dimensions, projection the only variable):
KEYS_ONLY is flat — dimensions × 4 plus ~35 B of key overhead, and independent of
item size. Measured 4,105 / 4,104 / 4,104 B on a 1024-dim index across a minimal item, a
+10 KB item, and an item carrying a second vector.ALL adds the entire rest of the item to every vector write, including other vector
attributes (carried in base-table form, ~5.5 B per number rather than 4). A 10 KB payload
added 10,246 B.ALL costs exactly the same as
KEYS_ONLY. The projection only costs what the rest of the item costs. Choose it on item
size, not on the vector.
estimated_item_size_bytesEXCLUDES the vector attributes. Declare the size of the item without its embeddings; the calculator adds each vector’s own footprint fromdimensions. This matters only forALL(andINCLUDEnaming a vector), where the model isdimensions × 4 + key overhead + the rest of the item— if the declared size already counted the embedding, the vector is billed twice and anALLindex looks roughly twice as expensive as it is. The convention is this way round becausedimensionsis exact and an item-size estimate is not, so deriving the vector’s bytes fromdimensionsis strictly more accurate than trusting it to be included. A 1024-dim embedding is ~4 KB in the index and ~5.5 KB in base-table form, so on a small item this is the difference between a right answer and a doubled one.
Search cost is deliberately not priced. It bills on the vector data a search examines
inside the index plus the data returned, and the examined fraction varies by an order of
magnitude with index configuration (measured: 12.8% of all vector bytes on a 60-item
unpartitioned index against 1.3% on a 200-item partitioned one). A calibration fitted across
configurations came out 51–69% low at 256/1536/3072 dimensions — exactly where real embedding
models sit. The report gives the measured drivers instead: TopK is exactly linear, the
projection is a ~170× multiplier on bytes returned per result (~89 B for KEYS_ONLY against
the whole item for ALL; a TopK=100 search on an ALL index measured 1.52 MB), cost is
linear in dimensions at fixed configuration, and index size is not a linear driver because
ANN prunes — measured between 10 and 200 vectors in the searched partition. That is not the
same as “population never matters”: an empty 1024-dimension index measured about twice
a populated one, so do not extrapolate the shape down to a sparse or freshly-created index.
Obtain the real figure from ReturnConsumedCapacity or the VectorSearchRequestBytes
CloudWatch metric — a live validation run measures it for you.
{
"pattern_id": "Q8",
"description": "Get order by ID",
"operation": "GetItem",
"table": "Orders",
"index": null,
"peak_rps": 800,
"estimated_item_size_bytes": 2048,
"items_per_request": 1,
"consistency": "strong"
}| Field | Required | Default | Notes |
|---|---|---|---|
pattern_id | yes | — | Short identifier used in the cost report (e.g. Q8, U1). |
description | no | "" | One-line human description. |
operation | yes | — | One of: GetItem, Query, Scan, BatchGetItem, TransactGetItems, PutItem, UpdateItem, DeleteItem, BatchWriteItem, TransactWriteItems, SearchVectors. |
table | yes | — | Name of the target table. |
index | no | null | GSI name, if the operation targets a secondary index. |
peak_rps | yes | 0 | Peak requests per second. This multiplies capacity into monthly cost. If you write 0 the pattern contributes nothing to the bill — treat this as a mandatory field per Mechanics #2. |
avg_rps | no | = peak_rps | Average/typical requests per second, for the expected-volume scenario. When any pattern sets it, the report adds a second “Expected Monthly Cost” headline computed at avg_rps (the per-op capacity is scale-invariant, so this drives the same CU at a lower rate and also scales storage growth). Defaults to peak_rps, so omitting it on every pattern yields the same single peak-sustained headline as before. Translate a stated daily/monthly volume to an average rate (e.g. 15,000 orders/day ÷ 86,400 s ≈ 0.17 rps); do not hand-compute the dollar figure — set this and let the calculator produce it. |
estimated_item_size_bytes | no | 1024 | Per-item size for single-item operations; per-item size for multi-item reads. |
items_per_request | no | 1 | Items returned (Query/Scan) or operated on (BatchGetItem, BatchWriteItem, TransactWriteItems, TransactGetItems). |
consistency | no | "eventual" | "eventual", "strong", or "transactional". transactional is implied by operation name for Transact* operations. |
A SearchVectors pattern is shaped a little differently: it targets a vector index by name,
carries top_k instead of items_per_request, and takes no consistency (vector search is
always eventually consistent, and consumes no RCU).
{
"pattern_id": "V1",
"description": "Find similar products",
"operation": "SearchVectors",
"table": "Products",
"index": "ProductEmbeddingIndex",
"peak_rps": 40,
"top_k": 10
}The index MUST name a vector index declared in that table’s vector_indexes, or the
calculator refuses the model rather than silently costing nothing.
Optional fields that tighten the estimate:
| Field | Applies to | Purpose |
|---|---|---|
top_k | SearchVectors | Results requested, 1-100. Defaults to 10. Exactly linear in the bytes a search returns, so it is the main lever you control at query time. |
filter | Query, Scan | Free-form string; presence triggers a note that FilterExpression doesn’t reduce cost. |
projection | Query, Scan | Array of attribute names; used to cap items_per_request at the 900 KB page limit. |
item_sizes | Batch*, Transact* | Array of per-item sizes when items vary; overrides estimated_item_size_bytes for batch/transact ops. |
previous_item_size_bytes | PutItem, UpdateItem | Billed size is max(before, after). Provide the before-size if you know updates shrink items. |
attributes_written | writes | Array of attribute names the write touches. Drives the UpdateItem GSI gate (verified live): an ALL-projection GSI rewrites on any change to an indexed item; an INCLUDE GSI only when a projected attribute changed; a KEYS_ONLY GSI only when its key changed. Changing an attribute that is a GSI key costs 2× on that index (DynamoDB deletes the old index entry and inserts a new one) — set this so the calculator can tell a key-moving update from a plain one. This field also gates VECTOR write cost, on the same convention: when it is provided and does not contain the index’s vector_attribute, that index is skipped and its vector write line prices at $0 while every other number looks right — so a write that stores an embedding must list it, "attributes_written": ["Embedding"]. When it is omitted the calculator cannot tell, so it assumes the vector attribute was touched (conservative upper bound) and says so in the report. Note there is no boolean form of this: a field like writes_vector_attribute is not in the schema and is silently ignored. |
entities_written | writes | Which entities a single write call creates/updates, for a single-table design with heterogeneous entities. A list of entity names (repeats allowed) or [{"entity": "Order", "count": 1}, {"entity": "LineItem", "count": 3}]. This is what makes GSI write-amplification accurate on a multi-entity table: a write amplifies only to the GSIs whose key the written entity actually carries (verified live), so a transaction writing 1 order + 3 line items is billed only the indexes those entities belong to — not every GSI on the table. Omit it and the calculator falls back to a conservative UPPER BOUND (every written item charged against every GSI) and says so in the report. On a single-entity table it is unnecessary (the lone entity is the writer). |
conditional_fail_rate | writes | Fraction (0.0–1.0). Failed conditional writes are still billed — this adds their cost on top. |
non_existent_rate | GetItem | Fraction of calls that miss. Attaches a note only; the minimum-RRU cost is already modeled. |
write_action | writes | "create", "update", "delete", or "mixed". Drives storage growth — only creates add storage. Defaults are operation-sensible: PutItem=create, UpdateItem=update, DeleteItem=delete, Batch/Transact=mixed. |
retention_days | writes | Days that created items persist before deletion/expiration. Default 30. Used for steady-state storage sizing. Honored per pattern — set it directly on the write pattern and it wins over any requirements default. The resulting storage is a bounded-retention snapshot at this window; a no-TTL / “kept forever” table accumulates past it, so do not read the figure as lifetime storage. |
source_requirement | writes | Cross-reference back to a requirements document; only used when a separate requirements JSON is provided. Ignore in normal use. |
A tiny Orders table with two patterns — GetItem by id (strong), Query by customer (eventual) — and one GSI:
{
"tables": [
{
"table_name": "Orders",
"key_schema": {
"partition_key": "customer_id",
"sort_key": "order_id"
},
"gsis": [
{
"index_name": "OrdersByCustomer",
"partition_key": "customer_id",
"sort_key": "created_at",
"projection": { "type": "ALL" }
}
]
}
],
"access_patterns": [
{
"pattern_id": "Q8",
"description": "Get order by ID",
"operation": "GetItem",
"table": "Orders",
"peak_rps": 800,
"estimated_item_size_bytes": 2048,
"consistency": "strong"
},
{
"pattern_id": "Q9",
"description": "List user orders",
"operation": "Query",
"table": "Orders",
"index": "OrdersByCustomer",
"peak_rps": 400,
"estimated_item_size_bytes": 2048,
"items_per_request": 20,
"consistency": "eventual"
}
]
}The access-pattern list described in SKILL.md already carries the data the calculator needs — the JSON is a literal translation of that table plus the schema:
| Skill artifact | JSON field |
|---|---|
| Access-pattern list row: RPS | peak_rps |
| Access-pattern list row: “items returned per call and approximate item size” | items_per_request, estimated_item_size_bytes |
| Access-pattern list row: consistency | consistency |
| Per-pattern plan: API call | operation |
| Per-pattern plan: target table or GSI | table, index |
| Schema: primary key (PK + optional SK) | tables[].key_schema — required for live deploy, ignored by the calculator |
| Schema: GSI key attributes and projection | tables[].gsis[] |
| Schema: vector index (dimensions, distance function, SearchSchema, projection) | tables[].vector_indexes[] |
| Per-pattern plan: a similarity search | operation: "SearchVectors" + index + top_k |
| Schema: Global Tables replication | tables[].global_tables |
If a field you would need is not yet in the design, the design itself has a gap — filling the JSON is how you find out.
python3 ${SKILL_DIR}/scripts/calculate_costs.py --model /path/to/dynamodb_data_model.json --output /path/to/cost_report.mdThe calculator prints where it wrote the report and how many patterns and tables it analyzed. Pricing uses DynamoDB Standard table-class, on-demand public rates; rates vary by region, so confirm the figure against the AWS DynamoDB pricing page (or aws pricing get-products --service-code AmazonDynamoDB) for the region you will deploy in before quoting it. Read the disclaimer at the top of the generated report before sharing the number.