Setting the file. One moment.
Confidence Scoring · Guideline Generation · anthropics/knowledge-work-plugins · Skills Docs
How to assign and interpret confidence scores for generated brand guidelines.
Back to the top of the page 22
Validate Data
Tech Debt
62
Recruiting Pipeline
71
Vendor Check
125
Zoom Meeting SDK Web
88
Vendor Review
181
Create An Asset
Video Sdk/web
The guideline section is well-supported and actionable.
Criteria (must meet at least 3):
3+ corroborating sources
Explicit guidance found in at least one AUTHORITATIVE source
Consistent across document and conversation analysis
Specific, actionable instructions (not just vague principles)
No unresolved conflicts
Example: Voice attribute “Confident but not arrogant” appears in the official style guide, is demonstrated in email templates, and matches patterns in top performer calls.
The section is reasonable but could benefit from more data or team confirmation.
Criteria (must meet at least 2):
1-2 corroborating sources
Inferred from patterns rather than explicit instruction
Minor inconsistencies resolved via recency or authority
Actionable but some interpretation was required
May have one unresolved conflict
Example: Tone for social media inferred from email templates and one Slack thread, but no official social media guidelines exist.
The section is a best-effort recommendation. Team review strongly recommended.
Criteria (must meet at least 2):
Single source only
Primarily inferred from indirect evidence
Significant interpretation required
Unresolved conflicts between sources
Limited specificity
Example: Competitive positioning derived from a single sales call where a competitor was discussed, with no supporting documentation.
High : Attributes appear in official brand guide AND are demonstrated in templates or callsMedium : Attributes appear in one document type only, or are inferred from multiple conversationsLow : Attributes inferred from a single source or from indirect evidence
High : Value propositions documented in official materials AND used consistently in sales conversationsMedium : Documented but not observed in practice, OR observed but not documentedLow : Extracted from a single pitch deck or single call
High : Explicit tone guidance exists for the context AND matches observed behaviorMedium : Tone inferred from 3+ examples of content in that contextLow : Tone inferred from 1-2 examples, or extrapolated from similar contexts
High : Terms explicitly listed in a style guide or glossaryMedium : Terms consistently used in templates and calls (pattern-based)Low : Terms observed in a single document or inferred from brand personality
High : Pattern observed in 5+ calls across multiple speakersMedium : Pattern observed in 3-4 calls or from a single top performerLow : Pattern observed in 1-2 calls only When guidelines are generated primarily from conversational sources (no AUTHORITATIVE documents available):
Voice Attributes derived from 5+ transcripts = Medium (not Low)
Messaging Framework from consistent patterns across 5+ calls = Medium
Language Patterns weight increases from 10% to 20% in aggregate calculation (subtract 10% from Voice Attributes)
Note this in the guideline metadata: “Guidelines generated primarily from conversational sources — team review recommended to formalize.”
Calculate overall guideline confidence as the weighted average of section scores:
Convert scores: High = 1.0, Medium = 0.6, Low = 0.3
Voice Attributes: High (1.0 x 0.30 = 0.30)
Messaging: Medium (0.6 x 0.25 = 0.15)
Tone: Medium (0.6 x 0.20 = 0.12)
Terminology: High (1.0 x 0.15 = 0.15)
Language: Low (0.3 x 0.10 = 0.03)
Overall: 0.75 = Medium-High confidence Aggregate score thresholds:
0.85–1.0 = High
0.60–0.84 = Medium
Below 0.60 = Low
Present confidence alongside each section header:
For Medium and Low confidence sections, include a brief note explaining why confidence is limited and what would raise it.
Low confidence sections should generate corresponding open questions:
Low confidence + conflict = High Priority open questionLow confidence + gap = Medium Priority open questionMedium confidence + minor inconsistency = Low Priority open question Every open question includes a recommendation that, if confirmed, would raise the section’s confidence score.
references/confidence-scoring.md