Setting the chapter. One moment.
AWS Agents For Devsecops
AWS Core
Core Skills · AWS…
Messaging And Streaming Skills
Migration And Modernization Skills
Networking And Content Delivery Skills
Security And Identity Skills
System Table Skills
Web And Mobile Development
120 chapters · 648 min
Core Skills
Chapter 24 of 120
Selects, deploys, and customizes AI models on Amazon SageMaker.
3 minutes · 692 words · 3 sections
Domain expertise for fine-tuning and deploying models on Amazon SageMaker. Covers the full model customization lifecycle from planning through production deployment.
Match the user’s intent to the appropriate reference folder and load only that content.
| User intent | Reference | When to use |
|---|---|---|
| Plan a model customization project, discover scope of work, resume or modify a plan | references/planning/ (opens in a new tab) | User’s request relates to model customization or deployment (fine-tuning, training, building, customizing, reviewing data, deploying or standing up a model — including selecting or deploying an off-the-shelf or base model with no training — or getting advice on approach). Always co-activate with other intents to discover full scope. Load this reference FIRST when the request matches multiple rows in this table — read its plan templates before routing to a single-action reference. |
| Define the business problem, success criteria, or use case spec | references/use-case-specification/ (opens in a new tab) | User says “define my use case”, “capture requirements”, “what should I decide up front”, or as default first step in any plan. Skip only if user explicitly declines. |
| Select or change a base model | references/model-selection/ |
Install this repository
npx skills add aws/agent-toolkit-for-aws/plugin marketplace add aws/agent-toolkit-for-awsSkills install per repository, not per chapter — the CLI has no documented per-skill form, so we do not print one.
Selects, deploys, and customizes AI models on Amazon SageMaker. Fine-tuning (SFT, DPO, RLVR, RLAIF), model selection, dataset preparation, evaluation, deployment to SageMaker endpoints or Bedrock, and endpoint diagnostics. Covers the full lifecycle from planning through production. Use when fine-tuning models on SageMaker, selecting base models from SageMaker Hub, finding a model to deploy without fine-tuning, transforming datasets for training, checking data readiness, evaluating model quality, deploying to endpoints, setting up IAM roles and S3 buckets for training jobs, or managing a SageMaker Managed MLflow app. Also use to check endpoint health, diagnose failures, debug latency or errors, or view container logs and CloudWatch metrics. Covers Serverless Model Customization, Nova and OSS deployment paths, and PySDK v3 usage. NOT for Ground Truth labeling, Feature Store, or general-purpose AWS infrastructure.
The verbatim description from this skill’s front matter — the string an agent matches on to decide whether to load it.
skills/core-skills/aws-ai-ml/SKILL.mdmain, last pushed 10 August 2026.SKILL.md, not by matching a directory convention. 17 distinct layouts observed: plugins/aws-agents-for-devsecops/skills/*/SKILL.md, plugins/aws-agents/skills/*/SKILL.md, plugins/aws-core/skills/*/SKILL.md, skills/core-skills/*/SKILL.md, skills/specialized-skills/analytics-skills/*/SKILL.md, skills/specialized-skills/database-skills/*/SKILL.md, skills/specialized-skills/ec2-skills/*/SKILL.md, skills/specialized-skills/messaging-and-streaming-skills/*/SKILL.md, skills/specialized-skills/migration-and-modernization-skills/*/SKILL.md, skills/specialized-skills/networking-and-content-delivery-skills/*/SKILL.md, skills/specialized-skills/operations-skills/*/SKILL.md, skills/specialized-skills/resilience-skills/*/SKILL.md, skills/specialized-skills/security-and-identity-skills/*/SKILL.md, skills/specialized-skills/serverless-skills/*/SKILL.md.| User asks which model to use, mentions a model name or family, or wants to evaluate what’s available. Always activate model-selection even for known model names because the exact Hub model ID must be resolved. Recommended: route to use-case-specification first to capture requirements — this produces better filtering results. Routing to use-case-specification first is not required if user provides a specific model name/ID or declines. If intent is ambiguous (fine-tune vs deploy as-is), model-selection MUST confirm which path before proceeding. Base model filtering for deployment MUST go through select-for-deployment.md and its scripts for any final recommendation. |
| Choose a fine-tuning technique (SFT, DPO, RLVR, RLAIF) | references/finetuning-technique/ (opens in a new tab) | User has decided to fine-tune and needs to choose a technique, or technique needs validation against the selected model’s recipes. Requires a base model to be selected first. |
| Validate dataset quality and format | references/dataset-evaluation/ (opens in a new tab) | User says “is my dataset okay”, “check my training data”, “I have my own data”, or before starting any fine-tuning job. |
| Transform or convert a dataset between formats | references/dataset-transformation/ (opens in a new tab) | User says “transform”, “convert”, “reformat”, or dataset schema needs to change. Always use this rather than writing inline transformation code. |
| Generate fine-tuning code and start training | references/finetuning/ (opens in a new tab) | User says “start training”, “fine-tune my model”, “I’m ready to train”, or plan reaches the finetuning step. Supports SFT, DPO, RLVR, RLAIF trainers. |
| Evaluate or benchmark a trained model | references/model-evaluation/ (opens in a new tab) | User says “evaluate my model”, “run a benchmark”, “test model performance”, “compare models”. Supports LLM-as-Judge and Custom Scorer. |
| Deploy a model to endpoint or Bedrock | references/model-deployment/ (opens in a new tab) | User says “deploy my model”, “create an endpoint”, “make it available”. Handles Nova vs OSS deployment pathways. |
| Set up IAM roles, S3 buckets, SDK configuration | references/sdk-getting-started/ (opens in a new tab) | User says “set up”, “getting started”, “check my environment”, “configure SDK”, or as first step in any plan involving SageMaker training/evaluation/deployment. |
| Manage project directory and artifacts | references/directory-management/ (opens in a new tab) | Starting a new project, resuming existing one, or when PLAN.md needs to be associated with a project directory. |
| Set up, update, or delete a SageMaker Managed MLflow app | references/manage-mlflow/ (opens in a new tab) | User says “set up MLflow”, “create MLflow app”, “update my MLflow app”, “delete my MLflow app”, “I need an MLflow server”, asks “what is SageMaker MLflow”, or a workflow needs an MLflow backend and none is connected. |
| Diagnose a failing or unhealthy SageMaker endpoint | references/endpoint-diagnostics/ (opens in a new tab) | User reports endpoint errors, latency, inference failures, or a deployment that failed. “What’s the status of my endpoint?”, “Is my endpoint erroring?”, “My endpoint failed — why?”, “How many instances are running behind my endpoint?”, “Is the latency my model or SageMaker?”, “Show me the container logs for my endpoint.” NOT for training-job issues, endpoint deletion, scaling changes, or new deployments. |
skills/specialized-skills/storage-skills/*/SKILL.mdskills/specialized-skills/system-table-skills/*/SKILL.mdskills/specialized-skills/web-and-mobile-development/*/SKILL.mdh1 and no skipped levels:.claude-plugin/marketplace.json by Amazon Web Services, declaring 4 plugins. It is read for editorial metadata only — never as the skill index, which is always the repository tree./aws/agent-toolkit-for-aws.md, and each chapter at its own .md URL.92 files · 485 KB
Everything this skill ships beside its prose. All of it is set here, as subchapters of chapter 24.
Documentation the agent loads on demand, rather than up front.
references/dataset-evaluation/1 file · 5 KB
references/dataset-evaluation/references/2 files · 8 KB
references/dataset-evaluation/scripts/1 file · 27 KB
references/dataset-transformation/code_templates/1 file · 2 KB
references/dataset-transformation/1 file · 13 KB
references/dataset-transformation/references/3 files · 11 KB
references/dataset-transformation/scripts/1 file · 5 KB
references/directory-management/1 file · 2 KB
references/endpoint-diagnostics/1 file · 4 KB
references/endpoint-diagnostics/references/1 file · 1 KB
references/endpoint-diagnostics/scripts/1 file · 13 KB
references/finetuning-technique/1 file · 2 KB
references/finetuning-technique/references/1 file · 2 KB
references/finetuning-technique/scripts/1 file · 741 B
references/finetuning/code_templates/5 files · 30 KB
references/finetuning/1 file · 9 KB
references/finetuning/references/5 files · 33 KB
references/finetuning/scripts/1 file · 841 B
references/finetuning/templates/2 files · 22 KB
references/manage-mlflow/1 file · 5 KB
references/manage-mlflow/references/5 files · 10 KB
references/model-deployment/code_templates/5 files · 18 KB
references/model-deployment/1 file · 9 KB
references/model-deployment/references/7 files · 58 KBreferences/model-evaluation/code_templates/2 files · 5 KB
references/model-evaluation/1 file · 5 KB
references/model-evaluation/references/9 files · 56 KBreferences/model-evaluation/scripts/3 files · 26 KB
references/model-selection/1 file · 7 KB
references/model-selection/references/benchmarks/8 files · 23 KBreferences/model-selection/references/3 files · 26 KB
references/model-selection/scripts/3 files · 15 KB
references/planning/1 file · 8 KB
references/planning/references/5 files · 10 KB
references/sdk-getting-started/1 file · 643 B
references/sdk-getting-started/references/2 files · 4 KB
references/use-case-specification/1 file · 2 KB
references/use-case-specification/references/2 files · 6 KB