TestGenAI Report
AI-written tests can lie. TestGenAI makes every test prove itself.
Every acceptance criterion becomes either a Playwright assertion derived by a named rule, or an explicit test.fixme(). Planted bugs show whether the passing tests would actually fail on broken code.
Components
17
with user stories
Criteria
224
acceptance criteria
Asserted
201
90% · rest pending
Passing
201
0 failed
Bugs caught
41/47
planted, one at a time
Mutants killed
198/259
37 unproven · 6 suspect
+ Bob's tests
234/259
same mutants, either suite
Green is not the same as useful
Same components, same planted bugs, one at a time. A bug counts as caught only if at least one passing test fails on the broken build.
| Suite | Planted bugs caught | |
|---|---|---|
v1 generator tests/e2e-baseline | 3 / 28 | |
Current generator tests/e2e | 41 / 47 |
Held-out rounds
IBM Bob wrote each component and its story without reading the generator, with the rules frozen. These are the numbers before any fix.
| Round | Component | Correct | Pending | Wrong | What we learned |
|---|---|---|---|---|---|
| 1 | Toggle | 3 / 10 | 7 | 0 | Rules untouched before Bob's run. Gaps: text that flips with state, aria-checked, named classes. Now 10/10. |
| 2 | Rating | 4 / 12 | 7 | 1 | Frozen at heldout-freeze. Bob caught a test asserting the opposite of its criterion; fixed for every rule by a polarity safety net. Now 10/12. |
| 3 | Alert | 1 / 11 | 7 | 3 | Frozen at heldout-freeze-2; Bob used TestGenAI through MCP. 3 tests targeted an element only rendered under a prop. Fixed generically. Now 11/11. |
| 4 | Counter | 0 / 10 | 6 | 4 | Frozen before Bob's run. All 4 asserted tests passed but were wrong: a fixture chosen for 'is disabled' never reached the maximum. Bob flagged all 4. Fixed generically (preconditions are never guessed): now 0 wrong, the rest honestly pending. `testgen verify` on the unedited suite flags exactly the lying tests. |
| 5 | CharTextarea | 0 / 12 | 8 | 4 | Frozen before Bob's run. 3 wrong + 1 partial (clicking a textarea instead of typing; a precondition 'after clearing' ignored). Bob flagged all 4. Fixed generically: now 1 correct, 0 wrong, 11 pending. |
| 6 | SearchBox | 1 / 12 | 6 | 5 | Frozen at heldout-freeze-5. As recorded through Bob: 1 correct, 6 pending, 5 flawed, and all 5 FAILED rather than passing (typing done as a click, Enter never pressed, the input clicked instead of the Search button, two fixtures in which the clear button never renders). Found afterwards: Bob's MCP server had been running since before the freeze and served older rules; regenerated with the frozen rules the result is 2 correct, 6 pending, 4 flawed (all failing). The MCP server now reloads the rules on every call and reports the commit they come from. Fixed generically (text fields without a length limit, elements revealed by typing, Enter key, callback arguments from the text state, exact quoted labels): now 11/12, 1 honestly pending. |
Prove it: testgen verify
No hand-planted bugs here. verify derives mutants from each component's source (callback not called, condition inverted, guard removed, off by one…), re-runs the suite against each one, and reports tests that never fail (unproven) and tests whose title names something a surviving mutant deleted (🚩 suspect). A surviving mutant can be equivalent, so a low score is a prompt to look, not a verdict.
| Suite | Mutants killed | + Bob's tests | Unproven tests | Suspects |
|---|---|---|---|---|
Accordiontests/e2e/Accordion.spec.ts | 6 / 9 | 9 / 9 | 3 | 0 |
Alerttests/e2e/Alert.spec.ts | 9 / 9 | – | 1 | 0 |
Buttontests/e2e/Button.spec.ts | 5 / 5 | – | 0 | 0 |
CharTextareatests/e2e/CharTextarea.spec.ts | 13 / 20 | 14 / 20 | 1 | 🚩 1 |
🚩 “When the text length reaches the maximum, the counter receives a visual indicator class that it is at the limit” says limit, yet still passes with atLimit removed (class dropped, line 56) | ||||
Countertests/r4/Counter.spec.ts | 3 / 24 | – | 0 | 🚩 2 |
🚩 “The increment button is disabled when the value reaches the maximum” says maximum, yet still passes with max removed (guard removed, line 41)🚩 “The decrement button is disabled when the value reaches the minimum” says minimum, yet still passes with min removed (guard removed, line 36) | ||||
Countertests/e2e/Counter.spec.ts | 11 / 20 | 12 / 20 | 1 | 🚩 1 |
🚩 “The counter never displays a value below the minimum or above the maximum” says minimum, yet still passes with min removed (guard removed, line 36) | ||||
DataTabletests/e2e/DataTable.spec.ts | 17 / 20 | 20 / 20 | 2 | 🚩 1 |
🚩 “Displays columns: select checkbox, name, email, status, created date, actions” says name, yet still passes with name removed (rendered value dropped, line 115) | ||||
Dropdowntests/e2e/Dropdown.spec.ts | 14 / 18 | 17 / 18 | 2 | 🚩 1 |
🚩 “Search input filters options by label” says label, yet still passes with label removed (rendered value dropped, line 91) | ||||
FileUploadtests/e2e/FileUpload.spec.ts | 16 / 20 | 20 / 20 | 2 | 0 |
LoginFormtests/e2e/LoginForm.spec.ts | 15 / 16 | 16 / 16 | 1 | 0 |
Modaltests/e2e/Modal.spec.ts | 13 / 13 | – | 2 | 0 |
Paginationtests/e2e/Pagination.spec.ts | 12 / 20 | 18 / 20 | 5 | 0 |
Ratingtests/e2e/Rating.spec.ts | 11 / 20 | 15 / 20 | 2 | 🚩 1 |
🚩 “The star buttons have descriptive aria-label attributes (e.g. "3 stars")” says label, yet still passes with label removed (rendered value dropped, line 56) | ||||
SearchBoxtests/e2e/SearchBox.spec.ts | 19 / 20 | 20 / 20 | 2 | 0 |
Steppertests/e2e/Stepper.spec.ts | 14 / 20 | 18 / 20 | 5 | 0 |
Tabstests/e2e/Tabs.spec.ts | 8 / 10 | 10 / 10 | 8 | 🚩 1 |
🚩 “Tab buttons have proper ARIA roles (tablist, tab)” says tablist, yet still passes with tabId removed (callback argument wrong, line 27) | ||||
Toggletests/e2e/Toggle.spec.ts | 6 / 8 | 7 / 8 | 0 | 0 |
Tooltiptests/e2e/Tooltip.spec.ts | 9 / 11 | 11 / 11 | 0 | 0 |
IBM Bob fills the gaps
TestGenAI left Pagination's layout criteria as honest test.fixme(). These bugs were planted before Bob wrote tests for those criteria. Generated: 3/6. Bob: 4/6. Together: 6/6.
| Planted bug in Pagination | Generated tests | Bob's tests |
|---|---|---|
| Next skips a page | caught | missed |
| Previous enabled on the first page | caught | missed |
| leading ellipsis never shown when pages are skipped | missed | caught |
| last page button missing when the window ends early | missed | caught |
| window shows 7 page numbers instead of 5 | missed | caught |
| window starts at the current page instead of centring on it | caught | caught |
Components
Accordion 16/18 asserted · 16 passed · 3/3 bugs caught
| Acceptance criterion | Rule | Result |
|---|---|---|
| Renders accordion with multiple items from props | collection | ✓ passed |
| Displays each item with a title and toggle button | collection | ✓ passed |
| Shows content when item is expanded | disclosure | ✓ passed |
| Hides content when item is collapsed | disclosure | ✓ passed |
| Clicking button toggles item expanded state | disclosure | ✓ passed |
| Displays expand/collapse icon that rotates | pending | – pending |
| Single item mode: expanding one item collapses others | disclosure | ✓ passed |
| Multiple item mode (allowMultiple=true): multiple items can be expanded | disclosure | ✓ passed |
| Calls onItemToggle callback when item is toggled | callback | ✓ passed |
| Supports keyboard navigation (Enter/Space to toggle) | keyboard-toggle | ✓ passed |
| Assigns unique data-testid to each item | unique-testids | ✓ passed |
| Uses aria-expanded attribute for accessibility | aria | ✓ passed |
| Uses aria-controls for semantic associations | aria | ✓ passed |
| Returns to collapsed state by default | disclosure | ✓ passed |
| Handles empty items array gracefully | degenerate-data | ✓ passed |
| All content is hidden when not expanded | disclosure | ✓ passed |
| Button text displays item title clearly | collection | ✓ passed |
| Visual icon indicates expansion state | pending | – pending |
Planted bugs
- caught allowMultiple ignored (always single mode)by “Multiple item mode (allowMultiple=true): multiple items can be expanded”
- caught onItemToggle never calledby “Calls onItemToggle callback when item is toggled”
- caught aria-expanded stuck at falseby “Clicking button toggles item expanded state”
Alert 11/11 asserted · 11 passed · 6/6 bugs caught
| Acceptance criterion | Rule | Result |
|---|---|---|
| The alert displays the message text passed to it | renders-prop | ✓ passed |
| When a title is provided, the title appears above the message | order | ✓ passed |
| When no title is provided, the title area is not rendered | absent-when-prop | ✓ passed |
| The alert reflects the correct severity: info, success, warning, and error variants are all supported | class-from-prop | ✓ passed |
| A dismissible alert shows a dismiss button | present | ✓ passed |
| A non-dismissible alert does not show a dismiss button | absent-when-prop | ✓ passed |
| Clicking the dismiss button removes the alert from the page | removes-self | ✓ passed |
| Clicking the dismiss button calls the onDismiss callback | callback | ✓ passed |
| After dismissal the alert is no longer visible | removes-self | ✓ passed |
| The dismiss button has an accessible label so screen-reader users know its purpose | aria | ✓ passed |
| The alert container has the ARIA role "alert" so assistive technology announces it automatically | aria | ✓ passed |
Planted bugs
- caught dismiss does not remove the alertby “Clicking the dismiss button removes the alert from the page”
- caught onDismiss never calledby “Clicking the dismiss button calls the onDismiss callback”
- caught dismiss button shown even when not dismissibleby “A non-dismissible alert does not show a dismiss button”
- caught empty title element rendered when no titleby “When no title is provided, the title area is not rendered”
- caught severity class hard-coded to infoby “The alert reflects the correct severity: info, success, warning, and error variants are all supported”
- caught dismiss button loses its accessible labelby “The dismiss button has an accessible label so screen-reader users know its purpose”
Button 6/7 asserted · 6 passed · 2/2 bugs caught
| Acceptance criterion | Rule | Result |
|---|---|---|
| Button renders with provided label text | renders-prop | ✓ passed |
| Button is clickable and triggers onClick handler | callback | ✓ passed |
| Button displays disabled state when disabled prop is true | is-disabled | ✓ passed |
| Button supports primary and secondary variants | class-from-prop | ✓ passed |
| Button supports small, medium, and large sizes | class-from-prop | ✓ passed |
| Button prevents click handler when disabled | prevent-when-disabled | ✓ passed |
| Button has proper accessibility attributes | pending | – pending |
Planted bugs
- caught disabled prop not forwardedby “Button displays disabled state when disabled prop is true”
- caught variant class hard-codedby “Button supports primary and secondary variants”
CharTextarea 11/12 asserted · 11 passed
| Acceptance criterion | Rule | Result |
|---|---|---|
| The textarea renders with the provided placeholder text when the value is empty | text-placeholder | ✓ passed |
| The character counter displays the remaining characters out of the maximum in the format "remaining/max" | text-counter | ✓ passed |
| Typing in the textarea updates the displayed value and decreases the remaining character count | text-typing | ✓ passed |
| When the text length reaches the maximum, no additional characters can be entered | text-limit | ✓ passed |
| When the text length reaches the maximum, the counter receives a visual indicator class that it is at the limit | text-limit-class | ✓ passed |
| The Clear button is disabled when the textarea is empty | text-clear-disabled | ✓ passed |
| Clicking the Clear button removes all text and resets the character counter to the full maximum | text-clear | ✓ passed |
| After clearing, the Clear button becomes disabled again | text-clear-after | ✓ passed |
| When the disabled prop is true, the textarea cannot be edited | text-readonly-when-disabled | ✓ passed |
| When the disabled prop is true, the Clear button is disabled even when there is existing text | pending | – pending |
| Typing fires the onChange callback with the current value on every keystroke | callback | ✓ passed |
| Clicking Clear fires the onChange callback with an empty string | text-clear-callback | ✓ passed |
Counter 10/10 asserted · 10 passed
| Acceptance criterion | Rule | Result |
|---|---|---|
| The counter displays its initial value when first rendered | numeric-initial | ✓ passed |
| Clicking the increment button increases the displayed value by one step | numeric-step | ✓ passed |
| Clicking the decrement button decreases the displayed value by one step | numeric-step | ✓ passed |
| The increment button is disabled when the value reaches the maximum | numeric-boundary | ✓ passed |
| The decrement button is disabled when the value reaches the minimum | numeric-boundary | ✓ passed |
| Clicking the Reset button returns the value to the initial value | numeric-reset | ✓ passed |
| Incrementing calls the onChange callback with the new value | numeric-callback | ✓ passed |
| Decrementing calls the onChange callback with the new value | numeric-callback | ✓ passed |
| When the counter is disabled, none of the buttons can be clicked and no value changes occur | numeric-frozen | ✓ passed |
| The counter never displays a value below the minimum or above the maximum | numeric-clamp | ✓ passed |
DataTable 13/15 asserted · 13 passed · 4/4 bugs caught
| Acceptance criterion | Rule | Result |
|---|---|---|
| Renders table with data rows from props | collection | ✓ passed |
| Displays columns: select checkbox, name, email, status, created date, actions | columns | ✓ passed |
| Allows clicking row to trigger onRowClick callback with row data | callback | ✓ passed |
| Allows sorting data by name, email, or status prop | pending | – pending |
| Allows filtering by status prop (active, inactive, pending, all) | filter-by-prop | ✓ passed |
| Supports multi-select with individual row checkboxes | multi-select | ✓ passed |
| Select all checkbox selects/deselects all visible rows | select-all | ✓ passed |
| Delete button on each row triggers onDelete callback with row id | callback | ✓ passed |
| Delete Selected button bulk deletes all selected rows | bulk-action | ✓ passed |
| Delete Selected button is disabled when no rows are selected | is-disabled | ✓ passed |
| Shows "No data available" message when data array is empty | quoted-text | ✓ passed |
| Handles empty data array gracefully without errors | degenerate-data | ✓ passed |
| Clicking on a row selects it visually without triggering onRowClick during checkbox interaction | without-triggering | ✓ passed |
| Status column displays status with appropriate styling | class-from-item | ✓ passed |
| Date formatting displays in locale-appropriate format | pending | – pending |
Planted bugs
- caught bulk delete removes every row, not just selectedby “Delete Selected button bulk deletes all selected rows”
- caught row checkbox click also triggers onRowClickby “Clicking on a row selects it visually without triggering onRowClick during checkbox interaction”
- caught status filter ignoredby “Allows filtering by status prop (active, inactive, pending, all)”
- caught onDelete receives the row index instead of its idby “Delete button on each row triggers onDelete callback with row id”
Dropdown 16/19 asserted · 16 passed · 3/3 bugs caught
| Acceptance criterion | Rule | Result |
|---|---|---|
| Renders dropdown button with selected value or placeholder | selected-value | ✓ passed |
| Clicking button opens dropdown menu | disclosure | ✓ passed |
| Menu displays all available options from props | collection | ✓ passed |
| Clicking option selects it and closes menu | close-action | ✓ passed |
| Selected option is highlighted visually | conditional-class | ✓ passed |
| Search input filters options by label | search-filter | ✓ passed |
| Shows "No options found" when no matches | search-filter | ✓ passed |
| Handles disabled state (button disabled, menu closed) | is-disabled | ✓ passed |
| Calls onSelect callback with selected option | callback | ✓ passed |
| Closes menu when clicking outside dropdown | close-action | ✓ passed |
| Closes menu when option is selected | close-action | ✓ passed |
| Displays selected option value in button | selected-value | ✓ passed |
| Arrow icon rotates when menu is open | pending | – pending |
| Supports aria-expanded and aria-haspopup for accessibility | aria | ✓ passed |
| Each option has unique data-testid | unique-testids | ✓ passed |
| Option click event works correctly | pending | – pending |
| Search input updates filter in real-time | search-filter | ✓ passed |
| Empty options array handled gracefully | degenerate-data | ✓ passed |
| Multiple rapid selections work correctly | pending | – pending |
Planted bugs
- caught menu stays open after selecting an optionby “Clicking option selects it and closes menu”
- caught search box does not filterby “Search input filters options by label”
- caught click outside no longer closes the menuby “Closes menu when clicking outside dropdown”
FileUpload 15/15 asserted · 15 passed · 3/3 bugs caught
| Acceptance criterion | Rule | Result |
|---|---|---|
| Drop zone accepts dragged files | file-upload | ✓ passed |
| Drop zone shows visual feedback when dragging over | file-upload | ✓ passed |
| Clicking drop zone opens file picker | file-upload | ✓ passed |
| File input accepts files and triggers onUpload | file-upload | ✓ passed |
| Selected files display in a list | file-upload | ✓ passed |
| Each file shows name and remove button | file-upload | ✓ passed |
| Remove button deletes file from selection | file-upload | ✓ passed |
| Multiple file support controlled by multiple prop | file-upload | ✓ passed |
| Validates against maxSize prop | file-upload | ✓ passed |
| Shows error message when file too large | file-upload | ✓ passed |
| Shows error message when multiple files not allowed | file-upload | ✓ passed |
| onError callback fires when validation fails | file-upload | ✓ passed |
| Accept attribute filters file types | file-upload | ✓ passed |
| Error message clears when valid files selected | file-upload | ✓ passed |
| Handles empty file selection gracefully | file-upload | ✓ passed |
Planted bugs
- caught maxSize off by one (rejects a file of exactly maxSize)by “Validates against maxSize prop”
- caught remove button does not remove the fileby “Remove button deletes file from selection”
- caught error not cleared after a valid selectionby “Error message clears when valid files selected”
LoginForm 8/8 asserted · 8 passed · 3/3 bugs caught
| Acceptance criterion | Rule | Result |
|---|---|---|
| Form renders with email and password inputs | present | ✓ passed |
| Form validates email format | form | ✓ passed |
| Form requires both email and password | form | ✓ passed |
| Form displays error messages | form | ✓ passed |
| Form is disabled during submission | form | ✓ passed |
| Form calls onSubmit with correct credentials | form | ✓ passed |
| Form clears errors on new attempt | form | ✓ passed |
| Submit button shows loading state | form | ✓ passed |
Planted bugs
- caught required-field validation removedby “Form requires both email and password”
- caught onSubmit arguments swappedby “Form calls onSubmit with correct credentials”
- caught inputs stay enabled while loadingby “Form is disabled during submission”
Modal 13/14 asserted · 13 passed · 3/3 bugs caught
| Acceptance criterion | Rule | Result |
|---|---|---|
| Modal is hidden when isOpen prop is false | render-when-prop | ✓ passed |
| Modal is visible when isOpen prop is true | render-when-prop | ✓ passed |
| Displays modal title from title prop | renders-prop | ✓ passed |
| Displays modal content from children prop | renders-prop | ✓ passed |
| Close button (×) closes modal by calling onClose callback | close-action | ✓ passed |
| Clicking overlay background closes modal | close-action | ✓ passed |
| Cancel button closes modal without confirming | close-action | ✓ passed |
| Confirm button calls onConfirm callback when present | callback | ✓ passed |
| Confirm button text customizable via confirmText prop | renders-prop | ✓ passed |
| Cancel button text customizable via cancelText prop | renders-prop | ✓ passed |
| Danger modal variant shows red confirm button when isDangerous is true | conditional-class | ✓ passed |
| Clicking inside modal content doesn't close modal | close-action | ✓ passed |
| Modal handles both confirmation and info-only modes | pending | – pending |
| Accessibility: Close button has aria-label | aria | ✓ passed |
Planted bugs
- caught overlay click no longer closesby “Clicking overlay background closes modal”
- caught clicks inside the dialog bubble to the overlay and close itby “Clicking inside modal content doesn't close modal”
- caught cancel button confirms instead of closingby “Cancel button closes modal without confirming”
Pagination 11/15 asserted · 11 passed · 3/6 bugs caught
| Acceptance criterion | Rule | Result |
|---|---|---|
| Displays current page and total pages | renders-prop | ✓ passed |
| Previous button is disabled when on first page | is-disabled | ✓ passed |
| Next button is disabled when on last page | is-disabled | ✓ passed |
| Previous button navigates to previous page when enabled | navigation | ✓ passed |
| Next button navigates to next page when enabled | navigation | ✓ passed |
| Shows page number buttons for navigation | present | ✓ passed |
| Current page button is highlighted/active | conditional-class | ✓ passed |
| Clicking page number navigates to that page | navigation | ✓ passed |
| Shows ellipsis (...) when pages are skipped | pending | – pending |
| Always shows first and last page numbers | pending | – pending |
| Shows maximum of 5 page numbers plus ellipsis | pending | – pending |
| Page numbers centered around current page | pending | – pending |
| Displays "Page X of Y" text | quoted-text | ✓ passed |
| Accessible: Previous/Next buttons have aria-labels | aria | ✓ passed |
| Accessible: Current page has aria-current="page" | aria | ✓ passed |
Planted bugs
- caught Next skips a pageby “Next button navigates to next page when enabled”
- caught Previous enabled on the first pageby “Previous button is disabled when on first page”
- missed leading ellipsis never shown when pages are skipped
- missed last page button missing when the window ends early
- missed window shows 7 page numbers instead of 5
- caught window starts at the current page instead of centring on itby “Previous button navigates to previous page when enabled”
Rating 10/12 asserted · 10 passed · 3/5 bugs caught
| Acceptance criterion | Rule | Result |
|---|---|---|
| Renders the correct number of star buttons (default 5) | item-count | ✓ passed |
| Displays "Not rated" when no value has been selected | state-text | ✓ passed |
| Displays the selected value as "<n> / <max>" after a star is clicked | template-text | ✓ passed |
| Clicking a star selects that rating and calls the onChange callback with the numeric value | callback | ✓ passed |
| The clicked star and all stars before it receive the filled style (rating-star-filled class) | cumulative-class | ✓ passed |
| Stars after the selected star are rendered as unfilled | cumulative-class | ✓ passed |
| Hovering over a star highlights that star and all stars before it without changing the committed value | hover-preview | ✓ passed |
| Moving the mouse away from the stars reverts the highlight to the committed value | hover-preview | ✓ passed |
| A disabled Rating cannot be clicked and does not fire onChange | prevent-when-disabled | ✓ passed |
| A read-only Rating cannot be clicked and does not fire onChange | pending | – pending |
| The star buttons have descriptive aria-label attributes (e.g. "3 stars") | aria | ✓ passed |
| The component renders a visible label text identifying the rating group | pending | – pending |
Planted bugs
- caught fill is off by one (clicked star not filled)by “The clicked star and all stars before it receive the filled style (rating-star-filled class)”
- caught hover highlight not cleared on mouse leaveby “Moving the mouse away from the stars reverts the highlight to the committed value”
- caught empty-state text changedby “Displays "Not rated" when no value has been selected”
- missed onChange receives a 0-based index instead of the star value
- missed read-only is ignored on click
SearchBox 11/12 asserted · 11 passed
| Acceptance criterion | Rule | Result |
|---|---|---|
| The search input renders with the provided placeholder text when no value is entered | text-placeholder | ✓ passed |
| Typing into the search input updates the displayed value and calls the onChange callback with the new text | callback | ✓ passed |
| The clear button is not visible when the input is empty | text-gated-visibility | ✓ passed |
| The clear button appears once the user has typed at least one character | text-gated-visibility | ✓ passed |
| Clicking the clear button empties the input and calls the onClear callback | text-clear | ✓ passed |
| Clicking the clear button also calls the onChange callback with an empty string | text-clear-callback | ✓ passed |
| Pressing Enter while the input is focused calls the onSearch callback with the current input value | key-callback | ✓ passed |
| Clicking the Search button calls the onSearch callback with the current input value | callback | ✓ passed |
| When the disabled prop is true, the input cannot be typed in and the Search button is disabled | text-readonly-when-disabled | ✓ passed |
| When the disabled prop is true, the clear button is disabled when it is visible | pending | – pending |
| The search input has an accessible label "Search" | aria | ✓ passed |
| The clear button has an accessible label "Clear search" | aria | ✓ passed |
Stepper 16/19 asserted · 16 passed · 2/2 bugs caught
| Acceptance criterion | Rule | Result |
|---|---|---|
| Renders all steps from props with labels | collection | ✓ passed |
| Displays current step as active/highlighted | conditional-class | ✓ passed |
| Previous steps marked as completed with checkmark | navigation | ✓ passed |
| Clicking completed step navigates back to it | navigation | ✓ passed |
| Clicking current step is disabled | navigation | ✓ passed |
| Clicking future steps is disabled (except next step) | navigation | ✓ passed |
| Next button advances to next step | navigation | ✓ passed |
| Previous button goes back to previous step | navigation | ✓ passed |
| First step has Previous button disabled | is-disabled | ✓ passed |
| Last step has Next button disabled | is-disabled | ✓ passed |
| Step indicator shows current position (e.g. "Step 2 of 5") | quoted-text | ✓ passed |
| Connector lines show progress between steps | pending | – pending |
| Connector lines highlight for completed steps | pending | – pending |
| Calls onStepChange callback when step changes | callback | ✓ passed |
| Supports disableNavigation prop to prevent clicking | navigation | ✓ passed |
| Each step has unique data-testid | unique-testids | ✓ passed |
| Handles single step gracefully | degenerate-data | ✓ passed |
| Handles empty steps array gracefully | degenerate-data | ✓ passed |
| Step status prop reflects in styling | pending | – pending |
Planted bugs
- caught Previous not disabled on the first stepby “First step has Previous button disabled”
- caught current step also marked completedby “Previous steps marked as completed with checkmark”
Tabs 13/15 asserted · 13 passed · 0/1 bugs caught
| Acceptance criterion | Rule | Result |
|---|---|---|
| Displays all tabs from tabs prop | collection | ✓ passed |
| First tab is active by default | navigation | ✓ passed |
| Can set default active tab via defaultTabId prop | navigation | ✓ passed |
| Clicking tab button activates that tab | navigation | ✓ passed |
| Active tab button shows active styling | conditional-class | ✓ passed |
| Displays content of active tab | navigation | ✓ passed |
| Active tab button has aria-selected="true" | aria | ✓ passed |
| Inactive tab buttons have aria-selected="false" | aria | ✓ passed |
| Disabled tabs cannot be clicked | navigation | ✓ passed |
| Disabled tabs show disabled styling | conditional-class | ✓ passed |
| Tab buttons have proper ARIA roles (tablist, tab) | aria | ✓ passed |
| Tab panel has proper ARIA controls | aria | ✓ passed |
| Tab content updates when tab changes | pending | – pending |
| onTabChange callback fires when tab is clicked | callback | ✓ passed |
| Each tab can have different content types | pending | – pending |
Planted bugs
- missed disabled tabs can be activated
Toggle 10/10 asserted · 10 passed · 4/4 bugs caught
| Acceptance criterion | Rule | Result |
|---|---|---|
| Toggle renders with the provided label text visible | renders-prop | ✓ passed |
| Toggle renders with `data-testid="toggle-state"` showing "Off" by default | state-text | ✓ passed |
| Toggle renders with `data-testid="toggle-state"` showing "On" when the `checked` prop is true | state-text | ✓ passed |
| Clicking the toggle switch flips the state from "Off" to "On" | state-text | ✓ passed |
| Clicking the toggle switch again flips the state back from "On" to "Off" | state-text | ✓ passed |
| Clicking the toggle calls the `onChange` callback with the new boolean value | callback | ✓ passed |
| The switch button has `aria-checked` set to `false` when off and `true` when on | aria | ✓ passed |
| The switch button has the class `toggle-on` when checked and no such class when unchecked | named-class | ✓ passed |
| The toggle switch is disabled and cannot be clicked when the `disabled` prop is true | is-disabled | ✓ passed |
| A disabled toggle does not call `onChange` when interacted with | prevent-when-disabled | ✓ passed |
Planted bugs
- caught onChange never calledby “Clicking the toggle calls the `onChange` callback with the new boolean value”
- caught state text inverted (shows On when off)by “Toggle renders with `data-testid="toggle-state"` showing "Off" by default”
- caught aria-checked stuck at falseby “The switch button has `aria-checked` set to `false` when off and `true` when on”
- caught toggle-on class never appliedby “The switch button has the class `toggle-on` when checked and no such class when unchecked”
Tooltip 11/12 asserted · 11 passed · 2/2 bugs caught
| Acceptance criterion | Rule | Result |
|---|---|---|
| Tooltip is hidden by default | disclosure | ✓ passed |
| Tooltip appears on mouse enter after delay | disclosure | ✓ passed |
| Tooltip disappears on mouse leave | disclosure | ✓ passed |
| Tooltip displays content from content prop | renders-prop | ✓ passed |
| Tooltip position can be set to top, bottom, left, or right | class-from-prop | ✓ passed |
| Tooltip position defaults to top | class-from-prop | ✓ passed |
| Delay can be customized via delay prop | disclosure | ✓ passed |
| Delay defaults to 200ms | pending | – pending |
| Tooltip has role="tooltip" for accessibility | aria | ✓ passed |
| Multiple hover events handled correctly | disclosure | ✓ passed |
| Clicking trigger doesn't hide tooltip | disclosure | ✓ passed |
| Tooltip position styling applied correctly | class-from-prop | ✓ passed |
Planted bugs
- caught delay prop ignored (always 200ms)by “Tooltip appears on mouse enter after delay”
- caught tooltip does not hide on mouse leaveby “Tooltip disappears on mouse leave”