Setting the file. One moment.
Skill 07 · Auditing Experiments Flags
Subchapter 7.3
references/flag-checks.mdMarkdown5 KBView on GitHub
Run these checks against each flag fetched via feature-flag-get-definition or feature-flag-get-all.
Detects active boolean flags that are effectively permanent and can be removed from code.
Look at: active, filters.multivariate (should be absent or null for boolean flags), filters.groups, last_called_at
Findings:
Fully rolled out boolean flag: Flag is active: true, has no multivariate config, and at least one release condition (filters.groups entry) with rollout_percentage: 100 and no properties (empty array or missing). This flag always evaluates to true.
Possibly unused: Flag has last_called_at that is more than 30 days ago, regardless of rollout configuration.
Detects flags that were created but never activated.
Look at: active, created_at, activity logs (if available)
Findings:
Stale draft flag: active is false, flag is more than 30 days old (based on created_at), and activity logs confirm it was never activated.
Note: The “never activated” check requires activity logs. If activity logs are unavailable, skip this sub-check and only report based on the flag being inactive and old:
Detects flags whose linked experiments are all done.
Look at: experiment_set (list of linked experiment IDs), and for each experiment, check its end_date and archived status via experiment-get.
Findings:
active: true, has entries in experiment_set, and ALL linked experiments have end_date set (completed) or archived: true.
Checks multivariate flag rollout percentages for correctness.
Look at: filters.multivariate.variants (array of {key, rollout_percentage, ...}), experiment_set
Findings:
Variant sum != 100%: The sum of all rollout_percentage values across filters.multivariate.variants does not equal 100.
Dead variant (0% rollout): A variant has rollout_percentage: 0 on a flag that is NOT linked to an experiment (empty experiment_set).
Dead condition (0% rollout): A release condition in filters.groups has rollout_percentage: 0.
Manual rollout on experiment flag: A flag with entries in experiment_set has release conditions where rollout_percentage is not the expected even split. This suggests someone manually adjusted the rollout outside the experiment.
Checks for flags with unstable or high-churn configurations. These checks require activity logs. If unavailable, skip and note it.
Look at: Activity log entries for the flag, created_at
Findings:
Toggle instability: The flag has been toggled on/off (active → inactive or vice versa) more than 3 times based on activity logs.
High config churn: The flag has more than 20 activity log entries AND the average rate exceeds 0.5 changes per day (calculated from first to last activity log entry).