Content Experimentation Best Practices · sanity-io/agent-toolkit · Skills Docs

Skills

Skill 1 of 8

Content Experimentation Best Practices

Content experimentation and A/B testing guidance covering experiment design, hypotheses, metrics, sample size, statistical foundations, CMS-managed variants, and common analysis…

1 minute · 162 words · 7 sections

Install

$ npx skills add sanity-io/agent-toolkit --skill content-experimentation-best-practices

$ npx skills add sanity-io/agent-toolkit

» /plugin marketplace add sanity-io/agent-toolkit

The first command installs just this skill, by the name in its SKILL.md; the second installs the whole repository.

Principles and patterns for running effective content experiments to improve conversion rates, engagement, and user experience.

When to Apply

Reference these guidelines when:

Setting up A/B or multivariate testing infrastructure
Designing experiments for content changes
Analyzing and interpreting test results
Building CMS integrations for experimentation
Deciding what to test and how

Core Concepts

A/B Testing

Comparing two variants (A vs B) to determine which performs better.

Multivariate Testing

Testing multiple variables simultaneously to find optimal combinations.

Statistical Significance

The confidence level that results aren’t due to random chance.

Experimentation Culture

Making decisions based on data rather than opinions (HiPPO avoidance).

References

Start with the reference that matches the current problem, such as design, statistics, CMS integration, or pitfalls. See references/ for detailed guidance:

references/experiment-design.md — Hypothesis framework, metrics, sample size, and what to test
references/statistical-foundations.md — p-values, confidence intervals, power analysis, Bayesian methods
references/cms-integration.md — CMS-managed variants, field-level variants, external platforms
references/common-pitfalls.md — 17 common mistakes across statistics, design, execution, and interpretation

Appendix 1.1–1.4

4 files · 22 KB

Everything this skill ships beside its prose. All of it is set here, as subchapters of skill 1.

ReferencesMarkdown · 4 files

Documentation the agent loads on demand, rather than up front.

About this skill

Trigger

Content experimentation and A/B testing guidance covering experiment design, hypotheses, metrics, sample size, statistical foundations, CMS-managed variants, and common analysis pitfalls. Use this skill when planning experiments, setting up variants, choosing success metrics, interpreting statistical results, or building experimentation workflows in a CMS or frontend stack.

The verbatim description from this skill’s front matter — the string an agent matches on to decide whether to load it.

Colophon for this issue

Colophon

How this issue was made

Source

github.com/sanity-io/agent-toolkit, branch main, last pushed 16 September 2026.

Licence

MIT — the text of every skill is reproduced unmodified, frontmatter included, under the upstream licence.

Discovery

8 skills found by walking the repository tree for SKILL.md, not by matching a directory convention. One layout observed: skills/*/SKILL.md.

Issue colours

Resolved from a deterministic hash of the owner name. Two accent tones are generated per issue and each is proven against its own ground before it ships: a single accent that passes AA on both light and dark paper is arithmetically impossible.

Heading repairs

1 repair applied to this skill so the document has one h1 and no skipped levels:

Removed “Content Experimentation Best Practices”, a leading h1 that duplicated the skill title.

Spec compliance

Every skill satisfies the Agent Skills specification’s frontmatter rules.

Images

Images inside a skill come from the upstream repository. Where the author gave no alternative text we mark the image decorative rather than inventing a description — a plausible caption we made up is worse than none for the reader who depends on it.

Marketplace

A plugin manifest is published at .claude-plugin/marketplace.json by Sanity, declaring 1 plugin. It is read for editorial metadata only — never as the skill index, which is always the repository tree.

Signal

Install counts come from skills.sh. They measure downloads, not quality, and an unranked repository is not an unread one.

Agent surfaces

The whole issue is available as one markdown document at , and each skill at its own URL.

/sanity-io/agent-toolkit.md

.md