Subchapter 116.7

Transforming Data

references/transforming-data.mdMarkdown6 KBView on GitHub

Data Transformation Patterns

JSONata Expression Syntax

Running Release Tests

Scanning With AWS Security Agent

Setup DevOps Agent

Setup Security Agent

Setup

Threat Modeling With AWS Security Agent

Messaging And Streaming

Well Architected Review

Launch With AWS

Setting Up CloudWatch Observability

Signing In To AWS

Creating Amazon Aurora Db Cluster With Instances

Routing Traffic With Route53 And CloudFront

Resilience Program Design

Migrating From Jsonpath To Jsonata

Processing State Inputs And Outputs

Service Integrations

Transforming Data

Validation And Testing

Assets

Compensation Saga Pattern Asl

Express Standard Handoff Asl

Human In The Loop With Timeout Escalation Asl

Nested Map Parallel Structures Asl

Polling Loop Wait Check Choice Asl

2 more in the appendix

Connecting Lambda To API Gateway

Connecting Lambda To DynamoDB

Creating API Gateway Stage

Debugging Lambda Timeouts

Deploying Custom Domain REST API

Processing S3 Uploads With Step Functions

JSONata expressions are written inside {% %} delimiters in string values:

"Output": "{% $states.input.customer.name %}"
"TimeoutSeconds": "{% $timeout %}"
"Condition": "{% $states.input.age >= 18 %}"

Rules:

The string must start with {% (no leading spaces) and end with %} (no trailing spaces).
Not all fields accept JSONata — Type and Resource must be constant strings.
JSONata expressions can appear in string values within objects and arrays at any nesting depth.
A string without {% %} is treated as a literal value.
All string literals inside JSONata expressions must use single quotes ('text'), not double quotes. The expression is already inside a JSON double-quoted string, so double quotes would break the JSON.
Complex logic is wrapped in ( expr1; expr2; ...; finalExpr ) where semicolons separate sequential expressions and the last expression is the return value.
You cannot use $ or $$ at the top level.
You cannot use unqualified field names at top level.
Use $parse() instead $eval for deserializing JSON strings.
Expressions must produce a defined value because JSON cannot represent undefined.

String Quoting

"Output": "{% 'Hello ' & $states.input.name %}"
"Condition": "{% $states.input.status = 'active' %}"

Never use double quotes inside the expression:

❌  "Output": "{% "Hello" %}"
✓  "Output": "{% 'Hello' %}"

Local Variable Binding with `:=`

Use := inside ( ... ) blocks to bind intermediate values within a single JSONata expression. Semicolons separate each binding, and the last expression is the return value:

"Output": "{% ( $subtotal := $sum($states.input.items.price); $tax := $subtotal * 0.1; $discount := $exists($couponValue) ? $couponValue : 0; {'subtotal': $subtotal, 'tax': $tax, 'discount': $discount, 'total': $subtotal + $tax - $discount} ) %}"

You can also define local helper functions:

"Assign": {
  "summary": "{% ( $formatPrice := function($amt) { '$' & $formatNumber($amt, '#,##0.00') }; $subtotal := $sum($states.input.items.price); {'itemCount': $count($states.input.items), 'subtotal': $formatPrice($subtotal), 'total': $formatPrice($subtotal * 1.1)} ) %}"
}

Local variables bound with := exist only within the ( ... ) block. They do not affect state machine variables. To persist values across states, use the Assign field.

Filtering Arrays

"Output": {
  "expensiveItems": "{% $states.input.items[price > 100] %}"
}

Aggregation

"Output": {
  "total": "{% $sum($states.input.items.price) %}",
  "average": "{% $average($states.input.items.price) %}",
  "count": "{% $count($states.input.items) %}"
}

String Operations

"Output": {
  "fullName": "{% $states.input.firstName & ' ' & $states.input.lastName %}",
  "upper": "{% $uppercase($states.input.name) %}",
  "trimmed": "{% $trim($states.input.rawInput) %}"
}

Object Merging

"Output": "{% $merge([$states.input, {'processedAt': $now(), 'status': 'complete'}]) %}"

Building Lookup Maps with `$reduce`

Use $reduce to transform an array into a key-value object:

"Assign": {
  "priceByProduct": "{% $reduce($states.input.items, function($acc, $item) { $merge([$acc, {$item.productId: $item.price}]) }, {}) %}"
}

Given [{"productId": "A1", "price": 10}, {"productId": "B2", "price": 25}], this produces {"A1": 10, "B2": 25}.

Dynamic Key Access with `$lookup`

Use $lookup to access an object property by a variable key:

"Output": {
  "price": "{% $lookup($priceByProduct, $states.input.productId) %}"
}

This is essential when you’ve built a mapping object with $reduce and need to retrieve values dynamically. Standard dot notation ($priceByProduct.someKey) only works with literal key names.

Conditional Values

"Output": {
  "tier": "{% $states.input.total > 1000 ? 'gold' : 'standard' %}",
  "discount": "{% $exists($states.input.coupon) ? 0.1 : 0 %}"
}

Array Membership with `in` and Concatenation with `$append`

Test if a value exists in an array with in:

"Condition": "{% $states.input.status in ['pending', 'processing', 'shipped'] %}"

Concatenate arrays with $append:

"Assign": {
  "allIds": "{% $append($states.input.orderIds, $states.input.returnIds) %}"
}

Array Mapping

"Output": {
  "names": "{% $states.input.users.(firstName & ' ' & lastName) %}"
}

Generating UUIDs and Random Values

"Assign": {
  "requestId": "{% $uuid() %}",
  "randomValue": "{% $random() %}"
}

Partitioning Arrays

"Assign": {
  "batches": "{% $partition($states.input.items, 10) %}"
}

Parsing JSON Strings

"Assign": {
  "parsed": "{% $parse($states.input.jsonString) %}"
}

Hashing

"Assign": {
  "hash": "{% $hash($states.input.content, 'SHA-256') %}"
}

Timestamp Comparison with `$toMillis`

JSONata timestamps are strings, so you can’t compare them directly with < or >. Use $toMillis to convert to numeric milliseconds:

"Condition": "{% $toMillis($states.input.orderDate) > $toMillis($states.input.cutoffDate) %}"

Useful for sorting timestamps, calculating durations, or finding the most recent entry:

"Assign": {
  "ageMinutes": "{% $round(($toMillis($now()) - $toMillis($states.input.createdAt)) / 60000, 2) %}",
  "mostRecent": "{% $sort($states.input.timestamps, function($a, $b) { $toMillis($a) < $toMillis($b) })[0] %}"
}

Step Functions Built-in functions

Function	Purpose
`$partition(array, size)`	Partition array into chunks
`$range(start, end, step)`	Generate array of values
`$hash(data, algorithm)`	Calculate hash (MD5, SHA-1, SHA-256, SHA-384, SHA-512)
`$random([seed])`	Random number 0 ≤ n < 1, optional seed
`$uuid()`	Generate v4 UUID
`$parse(jsonString)`	Deserialize JSON string

references/transforming-data.md

Raw file

aws/agent-toolkit-for-aws