Skills

Skill 20 of 48

Verify SSI

Verify Single Step Instrumentation (SSI) is working end-to-end on Linux hosts — SSI automatically instruments applications for APM without code changes.

2 minutes · 537 words · 18 sections

Install

$ npx skills add datadog-labs/agent-skills --skill verify-ssi

$ npx skills add datadog-labs/agent-skills

The first command installs just this skill, by the name in its SKILL.md; the second installs the whole repository.

Verify APM SSI on Linux

Before doing anything else: Fully resolve all variables in ## Context to resolve before acting. Do not begin Step 1 until every variable has a concrete value.

Triggers

Invoke this skill when the user expresses intent to:

Confirm SSI is working after installing the Datadog Agent on Linux
Check whether a Linux process is being instrumented
Verify the tracer is running and reporting telemetry

Do NOT invoke this skill if:

SSI has not been enabled yet — run agent-install first
Services have not been restarted since the agent was installed — restart them first, then verify

Prerequisites

Not done: agent-install is complete
Not done: Application services have been restarted since the agent was installed

pup-cli: check, install, and authenticate

Claude runs

pup auth login --site <DD_SITE>

This opens a browser tab for OAuth. Complete the login there — Claude will continue once the command exits.

If valid token — proceed. ERROR: No browser available: export DD_APP_KEY=<your-app-key>

Context to resolve before acting

Variable	How to resolve
`DD_HOSTNAME`	Hostname as Datadog sees it — from `sudo datadog-agent status` output
`SERVICE_NAME`	Expected service name in APM — ask the user
`ENV`	Environment tag — ask the user
`DD_SITE`	`grep "^site:" /etc/datadog-agent/datadog.yaml` via SSH, or ask the user
`SSH_KEY`	Path to SSH private key
`SSH_USER`	SSH username
`SSH_HOST`	Hostname or IP of the target host

Step 1: Confirm the Process is Injected

Use /proc/<pid>/maps — this is the authoritative check. It shows the actual shared libraries loaded into the running process, which is the only way to confirm the launcher and tracer .so files were actually loaded.

Claude runs

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST> \
  "pgrep -a -f '<SERVICE_NAME>' | head -5"

Use the PID from above:

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST> \
  "sudo cat /proc/<PID>/maps | grep -E 'launcher|apm-library|datadog'"

If the output includes both the launcher (e.g. launcher.preload.so) and a language library (e.g. apm-library-python) — injection succeeded for this process.

ERROR: Launcher present but no language library — launcher ran but couldn’t inject. Check for injection errors:

Claude runs

pup apm troubleshooting list --hostname <DD_HOSTNAME> --timeframe 1h

ERROR: Neither present — process was not injected. Check /etc/ld.so.preload:

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST> "cat /etc/ld.so.preload"

If empty — install did not set up the launcher. Re-run the install script with DD_APM_INSTRUMENTATION_ENABLED=host. If non-empty but the process still isn’t injected — the process was started before the launcher was installed. Restart the service and recheck.

Step 2: Confirm the Agent is Receiving Traces

Claude runs

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST> \
  "sudo datadog-agent status 2>&1 | grep -A 15 'APM Agent'"

Healthy output shows:

feature_auto_instrumentation_enabled: true
Receiver (previous minute) with > 0 traces

ERROR: feature_auto_instrumentation_enabled: false — SSI not active on the agent. Check apm_config in /etc/datadog-agent/datadog.yaml.

ERROR: Receiver (previous minute): 0 — agent running but no traces yet. Generate traffic first (see Step 3), then recheck.

Step 3: Confirm the Service is Visible in Datadog

Claude runs

DD_SITE=<DD_SITE> pup apm services list --env <ENV> --from 1h

If <SERVICE_NAME> appears with isTraced: true — traces are reaching the Datadog backend.

Flask / ddtrace v3 naming note: With ddtrace >=3.x, Flask spans are emitted as service:flask rather than service:<DD_SERVICE>. The DD_SERVICE value appears as base_service on the spans. If you set DD_SERVICE=my-app, search for service:flask in the APM UI — the service list will show flask, not my-app. Check the base_service tag to confirm it matches your DD_SERVICE.

ERROR: Service missing — generate traffic to trigger trace creation:

Claude runs

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST> \
  "sudo ss -tlnp 2>/dev/null | grep <PID> || sudo netstat -tlnp 2>/dev/null | grep <PID>"

Use the port from above:

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST> \
  "for i in \$(seq 1 10); do curl -s -o /dev/null http://localhost:<PORT>/; done"

Wait 30 seconds, then retry:

DD_SITE=<DD_SITE> pup apm services list --env <ENV> --from 10m
DD_SITE=<DD_SITE> pup traces search --query "service:<SERVICE_NAME>" --from 10m --limit 5

ERROR: Still missing — check for injection errors and go to troubleshoot-ssi:

pup apm troubleshooting list --hostname <DD_HOSTNAME> --timeframe 1h

Done

Exit when ALL of the following are true:

Not done: Step 1: launcher + language library both visible in /proc/<PID>/maps
Not done: Step 2: agent APM receiver shows > 0 traces/min
Not done: Step 3: service appears in pup apm services list

If any check fails, go to troubleshoot-ssi.

When all steps pass, automatically proceed to onboarding-summary now — do not ask the user for permission.

Security constraints

Never write a raw API key into any file or chat message
Always confirm before restarting production services

About this skill

Trigger

Verify Single Step Instrumentation (SSI) is working end-to-end on Linux hosts — SSI automatically instruments applications for APM without code changes. Only use after enable-ssi has run.

The verbatim description from this skill’s front matter — the string an agent matches on to decide whether to load it.

Colophon for this issue

Colophon

How this issue was made

Source

github.com/datadog-labs/agent-skills, branch main, last pushed 17 September 2026.

Licence

MIT — the text of every skill is reproduced unmodified, frontmatter included, under the upstream licence.

Discovery

48 skills found by walking the repository tree for SKILL.md, not by matching a directory convention. 11 distinct layouts observed: agent-observability/*/SKILL.md, */SKILL.md, dd-apm/k8s-ssi/*/SKILL.md, dd-apm/linux-ssi/*/SKILL.md, dd-apm/*/SKILL.md, dd-apps/*/SKILL.md, dd-audit/*/SKILL.md, dd-browser-sdk/*/SKILL.md, dd-security/csm/*/SKILL.md, dd-software-delivery/*/SKILL.md, SKILL.md (repo root).

Issue colours

Resolved from a curated brand profile — hue 299°, chroma 0.183. Two accent tones are generated per issue and each is proven against its own ground before it ships: a single accent that passes AA on both light and dark paper is arithmetically impossible.

Heading repairs

1 repair applied to this skill so the document has one h1 and no skipped levels:

Shifted “18 headings” from h1 to h2 so the skill title is the only h1.

Spec compliance

12 editorial notes across 12 of 48 skills. They are printed in the margin of each skill rather than as errors here.

Agent Observability Build Eval From Annotations — 1 note
Agent Observability Eval Pipeline — 1 note
Agent Observability Replay Trace — 1 note
Dd Audit AI Activity — 1 note
Dd Audit Compliance Report — 1 note
Dd Audit Cost Spike Investigation — 1 note
Dd Audit Key Compromise — 1 note
Dd Audit Security Investigation — 1 note
Upgrade Browser SDK V5 — 1 note
Upgrade Browser SDK V6 — 1 note

and 2 more.

Images

Images inside a skill come from the upstream repository. Where the author gave no alternative text we mark the image decorative rather than inventing a description — a plausible caption we made up is worse than none for the reader who depends on it.

Signal

Install counts come from skills.sh. They measure downloads, not quality, and an unranked repository is not an unread one.

Agent surfaces

The whole issue is available as one markdown document at /datadog-labs/agent-skills.md, and each skill at its own .md URL.

Publication

Set by Skills Docs from the source repository. Body text is Literata at the reader’s chosen size and measure; code is Geist Mono. Nothing on this page was written by us except this paragraph.

Verify APM SSI on Linux

Before doing anything else: Fully resolve all variables in ## Context to resolve before acting. Do not begin Step 1 until every variable has a concrete value.

Triggers

Invoke this skill when the user expresses intent to:

Confirm SSI is working after installing the Datadog Agent on Linux
Check whether a Linux process is being instrumented
Verify the tracer is running and reporting telemetry

Do NOT invoke this skill if:

SSI has not been enabled yet — run agent-install first
Services have not been restarted since the agent was installed — restart them first, then verify

Prerequisites

Not done: agent-install is complete
Not done: Application services have been restarted since the agent was installed

pup-cli: check, install, and authenticate

Claude runs

pup --version

If not found:

Claude runs

brew tap datadog-labs/pack
brew install pup

Check auth:

pup auth status --site <DD_SITE>

If not authenticated:

Claude runs

pup auth login --site <DD_SITE>

This opens a browser tab for OAuth. Complete the login there — Claude will continue once the command exits.

If valid token — proceed. ERROR: No browser available: export DD_APP_KEY=<your-app-key>

Context to resolve before acting

Variable	How to resolve
`DD_HOSTNAME`	Hostname as Datadog sees it — from `sudo datadog-agent status` output
`SERVICE_NAME`	Expected service name in APM — ask the user
`ENV`	Environment tag — ask the user
`DD_SITE`	`grep "^site:" /etc/datadog-agent/datadog.yaml` via SSH, or ask the user
`SSH_KEY`	Path to SSH private key
`SSH_USER`	SSH username
`SSH_HOST`	Hostname or IP of the target host

Step 1: Confirm the Process is Injected

Claude runs

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST> \
  "pgrep -a -f '<SERVICE_NAME>' | head -5"

Use the PID from above:

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST> \
  "sudo cat /proc/<PID>/maps | grep -E 'launcher|apm-library|datadog'"

If the output includes both the launcher (e.g. launcher.preload.so) and a language library (e.g. apm-library-python) — injection succeeded for this process.

ERROR: Launcher present but no language library — launcher ran but couldn’t inject. Check for injection errors:

Claude runs

pup apm troubleshooting list --hostname <DD_HOSTNAME> --timeframe 1h

ERROR: Neither present — process was not injected. Check /etc/ld.so.preload:

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST> "cat /etc/ld.so.preload"

Step 2: Confirm the Agent is Receiving Traces

Claude runs

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST> \
  "sudo datadog-agent status 2>&1 | grep -A 15 'APM Agent'"

Healthy output shows:

feature_auto_instrumentation_enabled: true
Receiver (previous minute) with > 0 traces

ERROR: feature_auto_instrumentation_enabled: false — SSI not active on the agent. Check apm_config in /etc/datadog-agent/datadog.yaml.

ERROR: Receiver (previous minute): 0 — agent running but no traces yet. Generate traffic first (see Step 3), then recheck.

Step 3: Confirm the Service is Visible in Datadog

Claude runs

DD_SITE=<DD_SITE> pup apm services list --env <ENV> --from 1h

If <SERVICE_NAME> appears with isTraced: true — traces are reaching the Datadog backend.

Flask / ddtrace v3 naming note: With ddtrace >=3.x, Flask spans are emitted as service:flask rather than service:<DD_SERVICE>. The DD_SERVICE value appears as base_service on the spans. If you set DD_SERVICE=my-app, search for service:flask in the APM UI — the service list will show flask, not my-app. Check the base_service tag to confirm it matches your DD_SERVICE.

ERROR: Service missing — generate traffic to trigger trace creation:

Claude runs

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST> \
  "sudo ss -tlnp 2>/dev/null | grep <PID> || sudo netstat -tlnp 2>/dev/null | grep <PID>"

Use the port from above:

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST> \
  "for i in \$(seq 1 10); do curl -s -o /dev/null http://localhost:<PORT>/; done"

Wait 30 seconds, then retry:

DD_SITE=<DD_SITE> pup apm services list --env <ENV> --from 10m
DD_SITE=<DD_SITE> pup traces search --query "service:<SERVICE_NAME>" --from 10m --limit 5

ERROR: Still missing — check for injection errors and go to troubleshoot-ssi:

pup apm troubleshooting list --hostname <DD_HOSTNAME> --timeframe 1h

Done

Exit when ALL of the following are true:

Not done: Step 1: launcher + language library both visible in /proc/<PID>/maps
Not done: Step 2: agent APM receiver shows > 0 traces/min
Not done: Step 3: service appears in pup apm services list

If any check fails, go to troubleshoot-ssi.

When all steps pass, automatically proceed to onboarding-summary now — do not ask the user for permission.

Security constraints

Never write a raw API key into any file or chat message
Always confirm before restarting production services