> **content-experimentation-best-practices** — skill 1 of 8 in [sanity-io/agent-toolkit](https://skillsdocs.com/sanity-io/agent-toolkit).
>
> Book (all skills, one file): https://skillsdocs.com/sanity-io/agent-toolkit.md
> Machine manifest: https://skillsdocs.com/sanity-io/agent-toolkit/.well-known/agent-skills/index.json
> Install the book: `npx skills add sanity-io/agent-toolkit`
> Upstream: https://github.com/sanity-io/agent-toolkit/blob/main/skills/content-experimentation-best-practices/SKILL.md @ `main`
> Raw bytes, no header: https://raw.githubusercontent.com/sanity-io/agent-toolkit/main/skills/content-experimentation-best-practices/SKILL.md
> Base for relative paths: https://raw.githubusercontent.com/sanity-io/agent-toolkit/main/skills/content-experimentation-best-practices/
> Licence: MIT — https://spdx.org/licenses/MIT.html
>
> Bundled files (4), referenced from this skill's directory:
>   - `references/cms-integration.md` — https://raw.githubusercontent.com/sanity-io/agent-toolkit/main/skills/content-experimentation-best-practices/references/cms-integration.md
>   - `references/common-pitfalls.md` — https://raw.githubusercontent.com/sanity-io/agent-toolkit/main/skills/content-experimentation-best-practices/references/common-pitfalls.md
>   - `references/experiment-design.md` — https://raw.githubusercontent.com/sanity-io/agent-toolkit/main/skills/content-experimentation-best-practices/references/experiment-design.md
>   - `references/statistical-foundations.md` — https://raw.githubusercontent.com/sanity-io/agent-toolkit/main/skills/content-experimentation-best-practices/references/statistical-foundations.md
>
> Content © its authors, served unmodified. Takedown: https://github.com/DreambaseAI/skillsdocs/issues/new?labels=takedown&title=Takedown+request

<!-- Verbatim upstream SKILL.md follows, YAML frontmatter included. -->

---
name: content-experimentation-best-practices
description: Content experimentation and A/B testing guidance covering experiment design, hypotheses, metrics, sample size, statistical foundations, CMS-managed variants, and common analysis pitfalls. Use this skill when planning experiments, setting up variants, choosing success metrics, interpreting statistical results, or building experimentation workflows in a CMS or frontend stack.
---

# Content Experimentation Best Practices

Principles and patterns for running effective content experiments to improve conversion rates, engagement, and user experience.

## When to Apply

Reference these guidelines when:
- Setting up A/B or multivariate testing infrastructure
- Designing experiments for content changes
- Analyzing and interpreting test results
- Building CMS integrations for experimentation
- Deciding what to test and how

## Core Concepts

### A/B Testing
Comparing two variants (A vs B) to determine which performs better.

### Multivariate Testing
Testing multiple variables simultaneously to find optimal combinations.

### Statistical Significance
The confidence level that results aren't due to random chance.

### Experimentation Culture
Making decisions based on data rather than opinions (HiPPO avoidance).

## References

Start with the reference that matches the current problem, such as design, statistics, CMS integration, or pitfalls. See `references/` for detailed guidance:
- `references/experiment-design.md` — Hypothesis framework, metrics, sample size, and what to test
- `references/statistical-foundations.md` — p-values, confidence intervals, power analysis, Bayesian methods
- `references/cms-integration.md` — CMS-managed variants, field-level variants, external platforms
- `references/common-pitfalls.md` — 17 common mistakes across statistics, design, execution, and interpretation