Subchapter 37.15
references/cloudwatch/appsignals-guides/ec2-nodejs.mdMarkdown21 KBView on GitHub
Your task is to modify Infrastructure as Code (IaC) files to enable AWS Application Signals for a Node.js application running on EC2 instances. You will update IAM permissions, install monitoring agents, and configure OpenTelemetry instrumentation through UserData scripts.
After completing this task:
Error Handling:
Do NOT:
cdk deploy, terraform apply, etc.)Code examples use CDK TypeScript syntax. If you are working with Terraform or CloudFormation, translate the CDK syntax to the appropriate format while keeping all bash commands identical. The UserData bash commands (CloudWatch Agent installation, ADOT installation, environment variables) are universal across all IaC tools - only the wrapper syntax differs.
Execute these steps to collect the information needed for configuration:
Read the UserData script and look for the application startup command. This is typically one of the last commands in UserData.
If you see:
docker run or docker start → Docker deploymentnode, npm start, yarn start, or similar → Non-Docker deploymentIf unclear:
Critical distinction: Where does the Node.js process run?
Analyze the existing IaC to determine these values for Application Signals enablement:
{{SERVICE_NAME}}
OTEL_RESOURCE_ATTRIBUTES=service.name={{SERVICE_NAME}}my-nodejs-app{{ENTRY_POINT}}
node --require ... {{ENTRY_POINT}}node commands in UserData)server.js, index.js, or app.js{{APP_DIR}}
cd, git clone, or file copy commands in UserData)/opt/myappFor Docker-based deployments you will also need to find these additional values:
{{APP_NAME}}
docker logs {{APP_NAME}}, docker exec, health checks, etc.docker run --name or use {{SERVICE_NAME}}-containernodejs-express-app{{IMAGE_URI}}
docker run or docker pull commands123456789012.dkr.ecr.us-east-1.amazonaws.com/my-app:latestIf you cannot determine a value: Ask the user for clarification before proceeding. Do not guess or make up values.
Determine the operating system to use the correct package manager and installation commands.
Amazon Linux:
yum package managerdnf package manageryum or dnf), or look for AMI references containing al2 or al2023Other Linux distributions:
apt package managerdnf or yum package managerIf unclear: Look for AMI name/ID in the IaC or ask the user which OS the EC2 instance is running. Do not guess or make up values.
Determine if the Node.js application uses CommonJS or ESM module format. This affects which ADOT dependencies to install and which node flags to use.
Check the application’s package.json file:
"type": "module" → ESM format"type": "commonjs" or no type field → CommonJS format (default)Alternative checks:
.mjs extension → ESM format.cjs extension → CommonJS format.js extension → Depends on package.json type fieldIf unclear:
Follow these steps in sequence:
Search for EC2 instance definitions using these patterns:
CDK:
new ec2.Instance(
ec2.Instance(
CfnInstance(Terraform:
resource "aws_instance"CloudFormation:
AWS::EC2::InstanceRead the file(s) containing the EC2 instance definition. You need to identify:
Find the IAM role attached to the EC2 instance
CDK:
role: someRole
new iam.Role(this, 'RoleName'What this policy is for, and its scope.
CloudWatchAgentServerPolicygoes on the instance role for the CloudWatch agent, which receives telemetry locally and forwards it to CloudWatch and X-Ray — so these permissions are what let the agent reach those destinations, not something the instrumentation itself needs. On EC2, ECS, and EKS, an ADOT-SDK-only setup adds no IAM to the workload at all (see thesetting-up-cloudwatch-observabilityskill’sreferences/cloudwatch-omni/instrumentation/instrumentation.md, which forbids attaching this policy on that path). Lambda is the exception — that path does grant its execution role X-Ray write permissions.The policy is broader than this configuration needs: it grants 14 actions, all on
Resource: "*", includingec2:DescribeVolumesandlogs:PutRetentionPolicy, which an Application-Signals-only agent config does not use. If the customer wants to trim it, resource scoping is the more valuable axis than pruning actions — dropping actions still leaveslogs:PutLogEventsandlogs:CreateLogGroupon every log group in the account andcloudwatch:PutMetricDataon every namespace, so the instance role can still write over unrelated services’ logs. Scope the resources and add anaws:ResourceAccountcondition, the wayCloudWatchLambdaApplicationSignalsExecutionRolePolicydoes for the Lambda path.Read the live document before changing anything — it takes two calls, since the version id is required and is not knowable up front:
aws iam get-policy --policy-arn arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy --query Policy.DefaultVersionIdthenaws iam get-policy-version --policy-arn <same> --version-id <that>. Do not hand-roll an action list from this page: the agent still creates the Application Signals log group (/aws/application-signals/data), sologs:CreateLogGroupandlogs:CreateLogStreammust survive any trim. A denial does not surface in the console — the agent recordsAccessDeniedin its own log (/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.logon Linux), and the symptom is that telemetry never starts arriving. Check that file first.
Add the CloudWatch Agent Server Policy to the IAM role’s managed policies.
CDK:
const role = new iam.Role(this, 'AppRole', {
assumedBy: new iam.ServicePrincipal('ec2.amazonaws.com'),
managedPolicies: [
iam.ManagedPolicy.fromAwsManagedPolicyName('CloudWatchAgentServerPolicy'),
// ... keep existing policies
],
});Add a CloudWatch Agent installation command to the UserData script.
CRITICAL for Terraform Users: When modifying Terraform user_data heredocs, you MUST preserve the EXACT indentation of existing lines. Terraform’s <<-EOF syntax strips leading whitespace, but only if indentation is consistent. When adding new bash commands:
If indentation is inconsistent, Terraform will NOT strip the whitespace, causing the deployed script to have leading spaces before #!/bin/bash, which will cause cloud-init to fail.
CDK TypeScript example:
instance.userData.addCommands(
'dnf install -y amazon-cloudwatch-agent', // Use dnf for AL2023, yum for AL2
// ... rest of UserData follows
);Placement: Add this command early in the UserData script:
dnf update -y, apt-get update), add it immediately after thoseFor other Linux distributions: CloudWatch Agent may not be available via the OS package manager. Refer to AWS CloudWatch Agent installation docs (opens in a new tab) for distribution-specific instructions.
The CloudWatch Agent was installed in Step 4. Now configure it for Application Signals:
CDK TypeScript example:
instance.userData.addCommands(
'# Create CloudWatch Agent configuration for Application Signals',
"cat > /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json << 'EOF'",
'{',
' "traces": {',
' "traces_collected": {',
' "application_signals": {}',
' }',
' },',
' "logs": {',
' "metrics_collected": {',
' "application_signals": {}',
' }',
' }',
'}',
'EOF',
'',
'# Start CloudWatch Agent with Application Signals configuration',
'/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \\',
' -a fetch-config \\',
' -m ec2 \\',
' -s \\',
' -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json',
);Choose based on deployment type AND module format identified in “Before You Start”.
For Docker deployments, modify the Dockerfile in the application directory.
Add the ADOT Node.js SDK installation AFTER any existing npm install or dependency installation commands:
For CommonJS applications:
# Install ADOT Node.js auto-instrumentation (use latest; ServiceEvents requires >=0.12.0)
RUN npm install @aws/aws-distro-opentelemetry-node-autoinstrumentationFor ESM applications:
# Install ADOT Node.js auto-instrumentation with ESM support (use latest; ServiceEvents requires >=0.12.0)
RUN npm install @aws/aws-distro-opentelemetry-node-autoinstrumentation @opentelemetry/instrumentationWhy modify Dockerfile, not UserData: The ADOT package must be installed inside the container image, not on the EC2 host. UserData commands run on the host and won’t affect the containerized application.
For non-Docker deployments, add to UserData AFTER CloudWatch Agent configuration:
For CommonJS applications:
instance.userData.addCommands(
'# Install ADOT Node.js auto-instrumentation (must run in the app directory so the',
'# package lands in {{APP_DIR}}/node_modules where Node module resolution finds it)',
'cd {{APP_DIR}} && npm install @aws/aws-distro-opentelemetry-node-autoinstrumentation',
);For ESM applications:
instance.userData.addCommands(
'# Install ADOT Node.js auto-instrumentation with ESM support (run in the app directory)',
'cd {{APP_DIR}} && npm install @aws/aws-distro-opentelemetry-node-autoinstrumentation @opentelemetry/instrumentation',
);Choose based on deployment type AND module format identified in “Before You Start”.
For Docker deployments, you need to modify both the Dockerfile CMD and the UserData docker run command.
1. Modify Dockerfile CMD to load ADOT agent:
Find the CMD line in your Dockerfile and modify it based on module format:
For CommonJS applications:
# Before:
CMD ["node", "app.js"]
# After:
CMD ["node", "--require", "@aws/aws-distro-opentelemetry-node-autoinstrumentation/register", "app.js"]For ESM applications:
# Before:
CMD ["node", "app.js"]
# After:
CMD ["node", "--import", "@aws/aws-distro-opentelemetry-node-autoinstrumentation/register", "--experimental-loader=@opentelemetry/instrumentation/hook.mjs", "app.js"]2. Add environment variables to docker run command in UserData:
Container networking — match the customer’s existing setup (minimal change). The example below uses --network host with localhost:4316 endpoints. That pairing is one option, not a hard requirement — the right choice depends on how the container already reaches the host-installed CloudWatch Agent. Don’t change the customer’s networking model just to instrument; instead pick the variant that fits theirs:
--network host (or willing to): keep it, and the localhost:4316 / localhost:2000 endpoints in the example work as-is. Trade-off: host networking shares the host’s network namespace (no container isolation), though the agent’s ports can stay bound to loopback, unreachable off-host. For production, it is recommended to restrict the OTLP 4316 / proxy 2000 ports via EC2 security groups / host firewall and to avoid co-locating untrusted containers; this guide does not apply those controls, so assess and configure them for your environment.--network host. Point the endpoints at the host instead — host.docker.internal:4316/:2000 (add --add-host=host.docker.internal:host-gateway on Linux) or the bridge gateway IP. This requires the CloudWatch Agent to listen on a non-loopback address, so it is recommended to restrict those ports with security groups / host firewall.cwagent:4316). Nothing binds to host interfaces. This is the same model the ECS guides use; choose it if the customer prefers full container isolation over a host-installed agent.Find the existing docker run command in UserData. Replace it with (this shows the --network host example — adapt per the networking variant you chose above):
instance.userData.addCommands(
'# Run container with Application Signals environment variables',
`docker run -d --name {{APP_NAME}} \\`,
` -e OTEL_METRICS_EXPORTER=none \\`,
` -e OTEL_LOGS_EXPORTER=none \\`,
` -e OTEL_AWS_APPLICATION_SIGNALS_ENABLED=true \\`,
` -e OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf \\`,
` -e OTEL_TRACES_SAMPLER=xray \\`,
` -e OTEL_TRACES_SAMPLER_ARG=endpoint=http://localhost:2000 \\`,
` -e OTEL_AWS_APPLICATION_SIGNALS_EXPORTER_ENDPOINT=http://localhost:4316/v1/metrics \\`,
` -e OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:4316/v1/traces \\`,
` -e OTEL_RESOURCE_ATTRIBUTES=service.name={{SERVICE_NAME}} \\`,
` --network host \\`,
` {{IMAGE_URI}}`,
);For non-Docker deployments, set environment variables and modify the node startup command based on module format.
Find the existing command that starts the Node.js application. Add the environment variables BEFORE it and modify the startup command:
For CommonJS applications:
instance.userData.addCommands(
'# Set OpenTelemetry environment variables',
'export OTEL_METRICS_EXPORTER=none',
'export OTEL_LOGS_EXPORTER=none',
'export OTEL_AWS_APPLICATION_SIGNALS_ENABLED=true',
'export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf',
'export OTEL_TRACES_SAMPLER=xray',
'export OTEL_TRACES_SAMPLER_ARG=endpoint=http://localhost:2000',
'export OTEL_AWS_APPLICATION_SIGNALS_EXPORTER_ENDPOINT=http://localhost:4316/v1/metrics',
'export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:4316/v1/traces',
'export OTEL_RESOURCE_ATTRIBUTES=service.name={{SERVICE_NAME}}',
'',
'# Start application with ADOT agent',
'cd {{APP_DIR}}',
'node --require "@aws/aws-distro-opentelemetry-node-autoinstrumentation/register" {{ENTRY_POINT}}',
);For ESM applications:
instance.userData.addCommands(
'# Set OpenTelemetry environment variables',
'export OTEL_METRICS_EXPORTER=none',
'export OTEL_LOGS_EXPORTER=none',
'export OTEL_AWS_APPLICATION_SIGNALS_ENABLED=true',
'export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf',
'export OTEL_TRACES_SAMPLER=xray',
'export OTEL_TRACES_SAMPLER_ARG=endpoint=http://localhost:2000',
'export OTEL_AWS_APPLICATION_SIGNALS_EXPORTER_ENDPOINT=http://localhost:4316/v1/metrics',
'export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:4316/v1/traces',
'export OTEL_RESOURCE_ATTRIBUTES=service.name={{SERVICE_NAME}}',
'',
'# Start application with ADOT agent (ESM)',
'cd {{APP_DIR}}',
'node --import "@aws/aws-distro-opentelemetry-node-autoinstrumentation/register" \\',
' --experimental-loader=@opentelemetry/instrumentation/hook.mjs \\',
' {{ENTRY_POINT}}',
);Note for systemd services: If the application uses systemd (look for .service files or systemctl commands in UserData), translate the export statements to Environment= directives in the service file, set WorkingDirectory={{APP_DIR}}, and update ExecStart= to use the appropriate node flags. After modifying the service file, add systemctl daemon-reload and systemctl restart <service> to UserData
Tell the user:
“I’ve completed the Application Signals enablement for your Node.js application. Here’s what I modified:
Files Changed:
Next Steps:
git diffcdk deployterraform applyVerification: Once deployed, you can verify Application Signals is working by:
Monitor Application Health: After enablement, you can monitor your application’s operational health using Application Signals dashboards. For more information, see Monitor the operational health of your applications with Application Signals (opens in a new tab).
Troubleshooting If you encounter any other issues, refer to the CloudWatch APM troubleshooting guide (opens in a new tab).
Let me know if you’d like me to make any adjustments before you deploy!”