Reviewing Tests

Unit, integration, and end-to-end tests should have distinct purposes:

• Avoid redundant overlap: Do not assert the same business outcomes across all levels.
• Share utilities: Reuse test data generation factories and helper objects across test suites.
• Isolate unit tests: Use test doubles to replace external dependencies (such as databases and APIs) to keep tests fast and deterministic.

Minimize the use of mocks. AI tools and developers often over-rely on mocks, which leads to fragile tests that pass even when integration points are broken. Use the most appropriate testing double:

Test smells

Flag test smells (poor testing practices) during code reviews. Unaddressed smells cause test suites to become flaky, slow, and hard to maintain.

SDK contracts, web UI, and dynamic waits

Apply specific standards when reviewing tests for libraries, SDKs, or web interfaces:

• SDK tests: Verify public attributes, return types, method chaining, and default values. When introducing breaking changes, write tests that assert deprecation warnings are emitted and verify legacy behavior.
• UI selectors: Use accessibility-aware selectors (like roles or labels) rather than absolute page layouts or CSS class structures.
• Waits: Use asynchronous, dynamic conditional waiting (polling until state is met) instead of hardcoded sleep intervals.

Generative AI and deterministic verification

AI-generated tests can create a coverage illusion by asserting current code behavior—including existing bugs—rather than the actual business requirements.

Use a structured workflow for AI-assisted tests:

1. Behavioral specification: Draft empty test blocks defining expectations before generating code.
2. AI generation: Use AI to write boilerplate setup, mocks, and execution logic.
3. Human review: Evaluate the assertions against business requirements. Ask if the tests would pass if a bug were introduced.
4. Mutation testing: Run mutation tools (such as StrykerJS, mutmut, or pitest) to verify that tests catch changes to production code.

To avoid manual mocking and flaky behavior, consider deterministic verification tools (like BitDive or Skyramp) that capture and replay database and network calls. Keep a centralized, repository-managed registry of regression test cases to train and update automated verification agents.

Review matrix

Use the following checklist and questions to evaluate test files:

Open-ended questions

Ask yourself these questions during code review:

• “What is the reasoning behind this specific test architecture or mock setup?”
• “Are there performance, database query count, or latency impacts associated with this setup?”
• “How does this testing strategy align with our team’s conventions and standards?”
• “What changes would improve the readability and maintainability of this test structure?”
• “What testing strategy do you recommend for high-risk safety requirements?”
• “As a skeptical engineer, where will the tests pass when they shouldn’t?”