Setting the file. One moment.
Subchapter 18.1
references/api-monitoring.mdMarkdown16 KBView on GitHub
Complete guide for monitoring Terraform Stack deployments using the HCP Terraform API. Use this approach for automation, CI/CD pipelines, and non-interactive environments like AI agents.
Use the HCP Terraform API instead of CLI commands when:
CLI commands that don’t work in automation:
terraform stacks deployment-run watch - Streams output, blocks indefinitelyterraform stacks deployment-group watch - Streams output, blocks indefinitelyterraform stacks configuration watch - Streams output, blocks indefinitelyTOKEN=$(jq -r '.credentials["app.terraform.io"].token' ~/.terraform.d/credentials.tfrc.json)export TFC_TOKEN="your-token-here"
TOKEN=$TFC_TOKENAll API requests require these headers:
-H "Authorization: Bearer $TOKEN"
-H "Content-Type: application/vnd.api+json"After uploading a configuration with terraform stacks configuration upload, follow this sequence to monitor deployment progress:
Endpoint: GET /api/v2/stack-configurations/{configuration-id}
Purpose: Verify configuration upload completed successfully and get the configuration details.
Request:
curl -s -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/vnd.api+json" \
"https://app.terraform.io/api/v2/stack-configurations/{configuration-id}" | jq '.'Response Fields:
attributes.status - Configuration processing status (pending/completed)attributes.sequence-number - Version number of this configurationattributes.components-detected - Number of components foundattributes.deployments-detected - Number of deployments foundExample Response:
{
"data": {
"id": "stc-ABC123",
"type": "stack-configurations",
"attributes": {
"status": "completed",
"sequence-number": 5,
"components-detected": 3,
"deployments-detected": 2,
"created-at": "2024-01-15T10:30:00.000Z",
"updated-at": "2024-01-15T10:30:45.000Z"
}
}
}Endpoint: GET /api/v2/stack-configurations/{configuration-id}/stack-deployment-group-summaries
Purpose: Get list of deployment groups, their IDs, and current status summary.
Request:
curl -s -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/vnd.api+json" \
"https://app.terraform.io/api/v2/stack-configurations/{configuration-id}/stack-deployment-group-summaries" | jq '.'Response Fields:
id - Deployment group ID (needed for next step)attributes.name - Deployment group name (e.g., dev_default)attributes.status - Overall status (running/succeeded/failed)attributes.status-counts - Breakdown of deployment statusesExample Response:
{
"data": [
{
"id": "sdg-XYZ789",
"type": "stack-deployment-group-summaries",
"attributes": {
"name": "dev_default",
"status": "running",
"status-counts": {
"pending": 0,
"running": 1,
"succeeded": 1,
"failed": 0
}
}
}
]
}Endpoint: GET /api/v2/stack-deployment-groups/{group-id}/stack-deployment-runs
Purpose: Get list of deployment runs for a specific group with their current status.
Request:
curl -s -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/vnd.api+json" \
"https://app.terraform.io/api/v2/stack-deployment-groups/{group-id}/stack-deployment-runs" | jq '.'Response Fields:
id - Deployment run ID (needed for next step)attributes.status - Current status (planning/planned/applying/applied/failed)attributes.created-at - Run start timeattributes.updated-at - Last update timeExample Response:
{
"data": [
{
"id": "sdr-123ABC",
"type": "stack-deployment-runs",
"attributes": {
"status": "planning",
"created-at": "2024-01-15T10:31:00.000Z",
"updated-at": "2024-01-15T10:31:15.000Z"
}
}
]
}Endpoint: GET /api/v2/stack-deployment-runs/{run-id}/stack-deployment-steps
Purpose: Get detailed information about individual plan and apply steps.
Request:
curl -s -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/vnd.api+json" \
"https://app.terraform.io/api/v2/stack-deployment-runs/{run-id}/stack-deployment-steps" | jq '.'Response Fields:
id - Step ID (needed for diagnostics and outputs)attributes.operation-type - Type of operation (plan/apply)attributes.status - Step status (running/completed/failed)attributes.component-name - Which component is being processedExample Response:
{
"data": [
{
"id": "sds-PlanStep123",
"type": "stack-deployment-steps",
"attributes": {
"operation-type": "plan",
"status": "completed",
"component-name": "vpc",
"created-at": "2024-01-15T10:31:05.000Z",
"completed-at": "2024-01-15T10:31:30.000Z"
}
},
{
"id": "sds-ApplyStep456",
"type": "stack-deployment-steps",
"attributes": {
"operation-type": "apply",
"status": "running",
"component-name": "vpc",
"created-at": "2024-01-15T10:32:00.000Z"
}
}
]
}Endpoint: GET /api/v2/stack-deployment-steps/{step-id}/stack-diagnostics
Purpose: Retrieve detailed error messages when a deployment step fails.
Critical: The stack_deployment_step_id query parameter is required. Without it, the API returns empty results.
Request:
curl -s -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/vnd.api+json" \
"https://app.terraform.io/api/v2/stack-deployment-steps/{step-id}/stack-diagnostics?stack_deployment_step_id={step-id}" | jq '.'Response Fields:
attributes.severity - Diagnostic level (error/warning)attributes.summary - Brief error descriptionattributes.detail - Detailed error messageattributes.diags - Array of diagnostic objects with file locations and code snippetsExample Response (Error with Details):
{
"data": [
{
"id": "stf-ErrorExampleId",
"type": "stack-diagnostics",
"attributes": {
"severity": "error",
"summary": "Diagnostics reported",
"detail": "2 errors",
"diags": [
{
"summary": "Unsupported attribute",
"detail": "This object does not have an attribute named \"target_id\".",
"range": {
"filename": "main.tf",
"start": {
"line": 634,
"column": 33
},
"end": {
"line": 634,
"column": 43
},
"source": "registry.terraform.io/terraform-aws-modules/alb/aws@9.17.0//main.tf"
},
"snippet": {
"code": " target_id = each.value.target_id",
"context": "resource \"aws_lb_target_group_attachment\" \"this\""
}
},
{
"summary": "Invalid reference",
"detail": "A reference to a resource type must be followed by at least one attribute access.",
"range": {
"filename": "main.tf",
"start": {
"line": 142,
"column": 15
},
"end": {
"line": 142,
"column": 28
},
"source": "local-module//main.tf"
},
"snippet": {
"code": " vpc_id = aws_vpc.main",
"context": "resource \"aws_subnet\" \"private\""
}
}
],
"acknowledged": false,
"created-at": "2024-01-15T10:32:15.000Z"
}
}
]
}Parsing Diagnostics:
Extract error information with jq:
# Get error summaries
curl -s -H "Authorization: Bearer $TOKEN" \
"https://app.terraform.io/api/v2/stack-deployment-steps/{step-id}/stack-diagnostics?stack_deployment_step_id={step-id}" | \
jq -r '.data[].attributes.diags[]? | "\(.summary): \(.detail)"'
# Get file locations
curl -s -H "Authorization: Bearer $TOKEN" \
"https://app.terraform.io/api/v2/stack-deployment-steps/{step-id}/stack-diagnostics?stack_deployment_step_id={step-id}" | \
jq -r '.data[].attributes.diags[]? | "\(.range.filename):\(.range.start.line)"'Endpoint: GET /api/v2/stack-deployment-steps/{final-apply-step-id}/artifacts?name=apply-description
Purpose: Retrieve Stack outputs after a successful deployment completes.
Important Notes:
curl -L to follow redirects automaticallyRequest:
curl -L -s -H "Authorization: Bearer $TOKEN" \
"https://app.terraform.io/api/v2/stack-deployment-steps/{final-apply-step-id}/artifacts?name=apply-description"Response Structure:
The artifact response includes an .outputs object where each output contains a change.after property with the actual output value:
{
"outputs": {
"alb_url": {
"change": {
"actions": ["no-op"],
"before": "http://my-alb-123456789.us-west-2.elb.amazonaws.com",
"after": "http://my-alb-123456789.us-west-2.elb.amazonaws.com",
"after_unknown": false,
"before_sensitive": false,
"after_sensitive": false
},
"type": "string"
},
"ecr_repository_url": {
"change": {
"actions": ["no-op"],
"before": "123456789.dkr.ecr.us-west-2.amazonaws.com/my-repo",
"after": "123456789.dkr.ecr.us-west-2.amazonaws.com/my-repo",
"after_unknown": false,
"before_sensitive": false,
"after_sensitive": false
},
"type": "string"
}
}
}Extract Only Output Values:
curl -L -s --header "Authorization: Bearer $TOKEN" \
"https://app.terraform.io/api/v2/stack-deployment-steps/{final-apply-step-id}/artifacts?name=apply-description" | \
jq -r '.outputs | to_entries | .[] | "\(.key): \(.value.change.after)"'Example Output:
alb_url: http://my-alb-123456789.us-west-2.elb.amazonaws.com
ecr_repository_url: 123456789.dkr.ecr.us-west-2.amazonaws.com/my-repoThe artifacts endpoint accepts these name parameter values:
plan-description - Terraform plan output in JSON formatplan-debug-log - Detailed debug logs from plan operationapply-description - Terraform apply output including outputs (JSON format)apply-debug-log - Detailed debug logs from apply operationRecommended polling intervals:
Implement exponential backoff:
# Example polling script with backoff
RETRY_COUNT=0
MAX_RETRIES=30
BACKOFF=5
while [ $RETRY_COUNT -lt $MAX_RETRIES ]; do
STATUS=$(curl -s -H "Authorization: Bearer $TOKEN" \
"https://app.terraform.io/api/v2/stack-deployment-runs/{run-id}" | \
jq -r '.data.attributes.status')
if [ "$STATUS" = "applied" ] || [ "$STATUS" = "failed" ]; then
echo "Deployment finished with status: $STATUS"
break
fi
echo "Current status: $STATUS. Waiting ${BACKOFF}s..."
sleep $BACKOFF
RETRY_COUNT=$((RETRY_COUNT + 1))
doneThese CLI commands DO NOT work in automation:
terraform stacks deployment-run watch - Streams output, blocks indefinitelyterraform stacks deployment-group watch - Streams output, blocks indefinitelyterraform stacks configuration watch - Streams output, blocks indefinitelySolution: Use API polling instead of watch commands.
There is currently no CLI command to retrieve Stack outputs. You must:
apply-description artifactThe artifacts endpoint returns HTTP 307 redirect to the actual artifact location. Ensure your HTTP client follows redirects:
curl: Use -L flag
Python requests: Set allow_redirects=True (default)
Node.js fetch: Set redirect: 'follow' (default)
Common API errors:
stack_deployment_step_id query parameter#!/bin/bash
# Configuration
TOKEN=$(jq -r '.credentials["app.terraform.io"].token' ~/.terraform.d/credentials.tfrc.json)
CONFIG_ID="stc-ABC123"
BASE_URL="https://app.terraform.io/api/v2"
# Helper function
api_get() {
curl -s -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/vnd.api+json" \
"$1"
}
# 1. Wait for configuration to complete
echo "Checking configuration status..."
while true; do
STATUS=$(api_get "$BASE_URL/stack-configurations/$CONFIG_ID" | jq -r '.data.attributes.status')
[ "$STATUS" = "completed" ] && break
echo "Configuration status: $STATUS. Waiting..."
sleep 5
done
# 2. Get deployment groups
echo "Getting deployment groups..."
GROUP_ID=$(api_get "$BASE_URL/stack-configurations/$CONFIG_ID/stack-deployment-group-summaries" | \
jq -r '.data[0].id')
# 3. Get deployment run
echo "Getting deployment run..."
RUN_ID=$(api_get "$BASE_URL/stack-deployment-groups/$GROUP_ID/stack-deployment-runs" | \
jq -r '.data[0].id')
# 4. Monitor deployment run
echo "Monitoring deployment run: $RUN_ID"
while true; do
STATUS=$(api_get "$BASE_URL/stack-deployment-runs/$RUN_ID" | jq -r '.data.attributes.status')
echo "Deployment status: $STATUS"
if [ "$STATUS" = "applied" ]; then
echo "Deployment succeeded!"
# 5. Get outputs from final apply step
APPLY_STEP=$(api_get "$BASE_URL/stack-deployment-runs/$RUN_ID/stack-deployment-steps" | \
jq -r '.data[] | select(.attributes["operation-type"] == "apply") | .id' | tail -1)
echo "Retrieving outputs from step: $APPLY_STEP"
curl -L -s -H "Authorization: Bearer $TOKEN" \
"$BASE_URL/stack-deployment-steps/$APPLY_STEP/artifacts?name=apply-description" | \
jq -r '.outputs | to_entries | .[] | "\(.key): \(.value.change.after)"'
break
fi
if [ "$STATUS" = "failed" ]; then
echo "Deployment failed!"
# Get error diagnostics
FAILED_STEP=$(api_get "$BASE_URL/stack-deployment-runs/$RUN_ID/stack-deployment-steps" | \
jq -r '.data[] | select(.attributes.status == "failed") | .id' | head -1)
echo "Error diagnostics from step: $FAILED_STEP"
api_get "$BASE_URL/stack-deployment-steps/$FAILED_STEP/stack-diagnostics?stack_deployment_step_id=$FAILED_STEP" | \
jq -r '.data[].attributes.diags[]? | "\(.summary): \(.detail)"'
exit 1
fi
sleep 10
doneThis script demonstrates a complete monitoring workflow from configuration upload to output retrieval with error handling.