1{2 "skill_name": "performing-multi-agent-code-review",3 "eval_type": "catch-rate",4 "purpose": "Merged pull requests in public Bitwarden repositories that introduced a bug a later pull request fixed. The buggy lines are added lines in the original diff. expected_location uses new-side line numbers in the file at head_sha. bug_summary is given to the grader to confirm a hit describes the same defect. See ./README.md for the run flow and scoring.",5 "evals": [6 {7 "id": 1,8 "repo": "bitwarden/clients",9 "pr_url"
13 "bug_summary": "decryptAes256 replaced Utils.fromArrayToUtf8 with Utils.fromArrayToByteString (Latin-1 byte mapping), so any multi-byte UTF-8 character in LastPass-imported Vault Data is corrupted.",
23 "bug_summary": "Bulk restore fetches data-ownership-enforced org user IDs before restoring; the users are still Revoked (an exempt status), so the Contains check at line 244 is never true and no default collection is created on bulk restore.",
33 "bug_summary": "The salt changed from STANDARD.encode(salt) (a String) to B64::from(...), whose as_bytes() returns the decoded raw bytes, so the export key is derived from a different salt than other clients use and password-protected exports cannot be decrypted by them.",
43 "bug_summary": "syncNewPolicies returns early when both policiesNew and policies are empty, so newPolicyService state is never cleared and stale policies keep being enforced after all policies are removed.",
53 "bug_summary": "The explicit startup check (timeout action == .logout and timeout != .never) was replaced with checkSessionTimeouts, which relies on hasPassedSessionTimeout; that never fires for .onAppRestart, so 'log out on app restart' accounts are only locked.",