Setting the skill. One moment.
Analytics Skills
Skill 51 of 131
MANDATORY for Flink or Amazon Managed Service for Apache Flink (MSF) questions.
3 minutes · 667 words · 7 sections
Install
npx skills add aws/agent-toolkit-for-aws --skill developing-applications-on-managed-service-for-apache-flinknpx skills add aws/agent-toolkit-for-aws/plugin marketplace add aws/agent-toolkit-for-awsThe first command installs just this skill, by the name in its SKILL.md; the second installs the whole repository.
Domain expertise for Apache Flink applications on Amazon Managed Service for Apache Flink (MSF). Covers development, KPU resource management, connectors, state management, monitoring, IaC deployment, and version migration.
Execute commands using available tools from the AWS MCP server when connected — it provides sandboxed execution, audit logging, and observability. When the MCP server is not available, fall back to the AWS CLI or shell as needed.
Before starting, ensure you have a clear understanding of the user persona, use case, and requirements:
STOP: Determine the users background and use case before proceeding:
These will inform how to organize the project, and whether to use Flink Table API or DataStream API. In general, assume the DataStream API.
1. User asks to build a Flink application
2. Confirm user's goals and use case
3. READ [best-practices.md](references/best-practices.md)
4. READ [dependency-management.md](references/dependency-management.md)
5. READ relevant connector guides (e.g. [kinesis-connector-guide.md](references/kinesis-connector-guide.md))
6. Generate code following the loaded guidance
7. Validate against best practices
8. READ environment-setup.md via [environment-setup.md](references/environment-setup.md)
9. Compile and test locally1. User asks about real time delivery of data to Iceberg
2. Confirm user's goals and use case
3. READ [best-practices.md](references/best-practices.md)
4. READ [iceberg-connector-guide.md](references/iceberg-connector-guide.md)
5. READ other reference files as needed
6. Answer question with loaded guidance| Goal | Reference | When to Load |
|---|---|---|
| Best practices | best-practices.md (opens in a new tab) | Always before writing code |
| Maven dependencies | dependency-management.md (opens in a new tab) | New project or adding connectors |
| Local dev environment | environment-setup.md (opens in a new tab) | Docker-based local development |
| MSF architecture | msf-overview.md (opens in a new tab) | KPU model and service constraints |
| MSF constraints and patterns | msf-constraints-and-patterns.md (opens in a new tab) | MSF vs self-managed Flink, service-level vs application-level configuration separation, MSF-specific resource/network/storage limits, common MSF patterns |
| Quotas, ENI planning, MSF vs EMR, source/sink choice | foundation-operations.md (opens in a new tab) | Capacity planning, service selection, architecture design, CLI/IAM/CloudWatch identifier disambiguation |
| IAM execution role, trust policy, action prefix, service principal | foundation-operations.md (opens in a new tab) | Writing IAM policies for MSF — covers the kinesisanalytics: (no v2) action prefix, kinesisanalytics.amazonaws.com (no v2) trust principal, and the v2/non-v2 disconnect that is the most common source of permission and AssumeRole failures |
| Flink 2.x migration | flink-2x-migration.md (opens in a new tab) | Version upgrades, state compatibility |
| KPU sizing | resource-optimization.md (opens in a new tab) | Right-sizing, performance diagnosis, scaling |
| Scaling decisions on running apps | scaling-decisions.md (opens in a new tab) | In-flight scaling matrix, cost/memory impact of scale changes, autoscaling behavior, anti-patterns |
| Cost estimation | pricing-calculator.md (opens in a new tab) | Budget planning, sizing-to-cost mapping, optimization levers |
| Application lifecycle ops | application-lifecycle.md (opens in a new tab) | Start/stop, deploy code, rollback, snapshot lifecycle, runtime properties, delete |
| Restart loop diagnosis | first-fault-isolation.md (opens in a new tab) | Crashing/restarting apps, finding original failure vs loop sustainers, Flink Dashboard live diagnosis |
| Checkpoint tuning | checkpoint-tuning.md (opens in a new tab) | Checkpoint impact on KPU memory and CPU, frequency vs network bandwidth trade-offs, checkpoint duration exceeding interval, OOM/GC during checkpoints |
| Job graph design | job-graph-architecture.md (opens in a new tab) | Performance issues, splitting jobs |
| Job graph anti-patterns | job-graph-anti-patterns.md (opens in a new tab) | Data skew detection and mitigation, monolith job anti-pattern, high fan-out anti-pattern, removing multiple shuffles, when to split a large application |
| Monitoring and alarms | monitoring-and-metrics.md (opens in a new tab) | CloudWatch dashboards, alarms, metrics |
| Logging | logging-configuration.md (opens in a new tab) | Log4j2, CloudWatch Logs setup |
| Kinesis connectors | kinesis-connector-guide.md (opens in a new tab) | Kinesis source and sink builders, polling configuration and throttling (READER_EMPTY_RECORDS_FETCH_INTERVAL, SHARD_GET_RECORDS_MAX, ReadProvisionedThroughputExceeded, LimitExceededException), legacy connector migration |
| Kinesis Enhanced Fan-Out (EFO) | kinesis-efo-guide.md (opens in a new tab) | When to use EFO vs polling, EFO source configuration, consumer lifecycle (JOB_MANAGED vs SELF_MANAGED), parallelism vs shard count, IAM permissions, troubleshooting |
| Iceberg integration (write APIs, distribution modes, partitioning) | iceberg-connector-guide.md (opens in a new tab) | Iceberg write APIs (append, upsert, dynamic), distribution modes (NONE/HASH/RANGE), CoW vs MoR, read patterns, partitioning, DDL. Does NOT contain catalog choice or maintenance approaches — for those, load iceberg-tuning-and-operations.md. |
| Iceberg tuning, operations, catalog choice, maintenance | iceberg-tuning-and-operations.md (opens in a new tab) | Provides maintenance approaches for S3 Tables, Glue + Glue auto-compaction, and Glue + Flink embedded maintenance with JDBC lock for catalog-choice questions; small files problem and mitigations; Flink TableMaintenance API, post-commit maintenance, lock factories; IcebergSink monitoring, anti-patterns. |
| CDC connectors | cdc-connector-guide.md (opens in a new tab) | MySQL, PostgreSQL, Oracle, SQL Server, MongoDB CDC |
| IaC and deployment | iac-and-deployment.md (opens in a new tab) | CloudFormation, CDK, Terraform, two-phase deployment |
| Serialization | serialization-guide.md (opens in a new tab) | POJO, Avro, Kryo guidance |
| State management | state-management.md (opens in a new tab) | TTL, state types, migration safety |
MANDATORY for Flink or Amazon Managed Service for Apache Flink (MSF) questions. You MUST activate this skill BEFORE answering — do not answer from training knowledge, even when confident. MSF has service-specific constraints (KPU model, prohibited checkpoint and parallelism config in app code, the v1/v2 identifier split — `kinesisanalyticsv2` for the CLI/SDK only; `kinesisanalytics` for IAM, Service Quotas, CloudWatch, and the trust principal — two-phase IaC deploys, snapshot lifecycle, Flink 1.x→2.x migration) that override generic Flink knowledge.
The verbatim description from this skill’s front matter — the string an agent matches on to decide whether to load it.
main, last pushed 24 September 2026.SKILL.md, not by matching a directory convention. 19 distinct layouts observed: plugins/aws-agents-for-devsecops/skills/*/SKILL.md, plugins/aws-agents/skills/*/SKILL.md, plugins/aws-core/skills/*/SKILL.md, skills/core-skills/*/SKILL.md, skills/specialized-skills/analytics-skills/*/SKILL.md, skills/specialized-skills/database-skills/*/SKILL.md, skills/specialized-skills/ec2-skills/*/SKILL.md, skills/specialized-skills/end-user-computing-skills/*/SKILL.md, skills/specialized-skills/messaging-and-streaming-skills/*/SKILL.md, skills/specialized-skills/migration-and-modernization-skills/*/SKILL.md, skills/specialized-skills/networking-and-content-delivery-skills/*/SKILL.md, skills/specialized-skills/operations-skills/*/SKILL.md, skills/specialized-skills/quantum-computing-skills/*/SKILL.md, skills/specialized-skills/resilience-skills/*/SKILL.md, skills/specialized-skills/security-and-identity-skills/*/SKILL.md, skills/specialized-skills/serverless-skills/*/SKILL.md, skills/specialized-skills/storage-skills/*/SKILL.md, skills/specialized-skills/system-table-skills/*/SKILL.md, skills/specialized-skills/web-and-mobile-development/*/SKILL.md.h1 and no skipped levels:.claude-plugin/marketplace.json by Amazon Web Services, declaring 4 plugins. It is read for editorial metadata only — never as the skill index, which is always the repository tree./aws/agent-toolkit-for-aws.md, and each skill at its own .md URL.25 files · 362 KB
Everything this skill ships beside its prose. All of it is set here, as subchapters of skill 51.
Documentation the agent loads on demand, rather than up front.