Setting the file. One moment.
Subchapter 23.15
references/iteration-evidence.mdMarkdown3 KBView on GitHub
Use this guide when improving a skill from positive examples, negative examples, review feedback, validation results, or observed agent behavior.
Store persistent improvement evidence under:
references/evidence/
├── findings-log.md
├── working-set.md
└── holdout-set.mdUse this directory only when examples should outlive the current task. For a one-off small fix, summarize the examples in SOURCES.md instead.
references/evidence/findings-log.md records interpreted findings:
references/evidence/working-set.md stores examples used while editing the skill.
references/evidence/holdout-set.md stores examples reserved for validation after edits. Do not tune directly against holdout examples unless the user explicitly moves them into the working set.
Use one record per example:
## EX-001: Short label
- Label: positive | negative
- Kind: true-positive | false-positive | false-negative | fix | regression | edge-case
- Origin: human-verified | mixed | synthetic
- Source: issue/PR/commit/log/user note/local validation pointer
- Status: working | holdout | resolved | deferred
- Expected behavior: concise statement
- Observed behavior: concise statement
- Skill delta: instruction, reference, description, or validation change
- Anonymization: what was removed or generalized
### Content
Summarized or redacted example content.Keep records concise. Preserve enough detail to reproduce the behavior, but redact secrets, customer data, private URLs, and unnecessary user content.
Positive findings are not just success stories. Use them to protect behaviors that must not regress.
Negative findings should identify the smallest failing decision:
Each negative finding should map to a concrete skill delta or an explicit deferred reason.
Promote evidence into the skill artifacts only when it changes future behavior:
SKILL.md.SOURCES.md.references/evidence/.Do not turn references/evidence/ into a changelog. The changelog belongs in SOURCES.md.