Skip to content

feat(create-bestax): Add Visual Regression Testing and Synchronized Versioning - #95

Merged
allxsmith merged 12 commits into
mainfrom
feat/94-visual-regression-testing
Oct 17, 2025
Merged

allxsmith merged 12 commits into
mainfrom
feat/94-visual-regression-testing

Conversation

@allxsmith

Copy link
Copy Markdown
Owner

Description

This PR implements comprehensive visual regression testing for create-bestax and introduces synchronized versioning between bulma-ui and create-bestax packages.

  • create-bestax (create-bestax)
  • bulma-ui (@allxsmith/bestax-bulma)

Related Issue(s)

Closes #94

Type of Change

  • New feature
  • Bug fix
  • Refactor
  • Documentation
  • Performance
  • Build tooling
  • Other (please describe):

Changes

Visual Regression Testing

  • Added Playwright E2E testing infrastructure
  • Created 17 parallel test scenarios covering all template variations
    • Templates: vite, vite-ts
    • Bulma flavors: complete, prefixed, no-helpers, no-helpers-prefixed, no-dark-mode
    • Icon libraries: none, fontawesome, mdi, ionicons, material-icons, material-symbols
  • Implemented 10 comprehensive tests per scenario:
    • Hero section rendering
    • Info cards layout
    • Notification toggle functionality
    • Counter increment/reset
    • Responsive layouts (desktop, tablet, mobile)
  • Added daily GitHub Actions workflow (2 AM UTC)
  • Configured visual regression with 0.2 threshold and 100px tolerance
  • Improved test coverage from 97.08% to 99.02%

Synchronized Versioning

  • Both bulma-ui and create-bestax now maintain identical version numbers
  • Updated release configs to analyze commits from both scopes
  • Synced baseline version to 2.3.3
  • Added VERSIONING.md documentation
  • When either package has changes, both bump to the same version

Checklist

  • My code follows the project style guidelines
  • I have performed a self-review of my code
  • I have added tests that prove my fix is effective or that my feature works
  • I have added/updated documentation as needed
  • I have updated Storybook stories as needed (bulma-ui)
  • My changes require a change to the documentation
  • All new and existing tests passed
  • Any relevant dependencies are updated

Screenshots / Demos

Playwright test infrastructure with 17 parallel scenarios testing all create-bestax template variations.

Additional Context

Benefits

  • Quality Assurance: Automated detection of visual regressions
  • Comprehensive Coverage: All 17 template variations tested daily
  • Fast Feedback: Parallel execution completes in ~5-7 minutes
  • Version Clarity: Users always know which versions work together
  • Simplified Dependency Management: create-bestax templates use matching bulma-ui versions

Files Added

  • .github/workflows/visual-regression.yml
  • create-bestax/e2e/ directory with tests, config, and utilities
  • VERSIONING.md documentation
  • Baseline screenshot files

…ersioning

- Add 17 parallel test scenarios for all template variations
- Implement E2E tests with screenshot comparison
- Add daily GitHub Actions workflow
- Configure visual regression with 0.2 threshold and 100px tolerance
- Test responsive layouts across desktop, tablet, and mobile
- Improve test coverage from 97.08% to 99.02%
- Implement synchronized versioning between bulma-ui and create-bestax
- Both packages now maintain identical version numbers
- Sync baseline version to 2.3.3

Closes #94
@allxsmith
allxsmith force-pushed the feat/94-visual-regression-testing branch from ea8851b to b937f84 Compare October 16, 2025 22:56
Comment thread .github/workflows/publish-devto.yml Fixed
Comment thread .github/workflows/publish-devto.yml Fixed
Comment thread .github/workflows/test-devto-action.yml Fixed
Comment thread .github/workflows/test-devto-action.yml Fixed
@allxsmith
allxsmith force-pushed the feat/94-visual-regression-testing branch from be5f9f5 to 8ead9ac Compare October 16, 2025 23:16
allxsmith and others added 8 commits October 16, 2025 20:08
- Keep logo and title on same line across all screen sizes (removed flex-direction: column on mobile)
- Standardize button spacing by reducing GitHub buttons gap from 1.5rem to 1rem for consistency
- Fix React Hooks rules-of-hooks violations in 5 components:
  * File.tsx: Use prefixedClassNames for conditional alignment and icons
  * Control.tsx: Use prefixedClassNames for conditional icon rendering
  * Card.tsx: Use prefixedClassNames in renderHeader, renderFooter, and CardHeader
  * Message.tsx: Use prefixedClassNames for conditional message sections
  * Modal.tsx: Use prefixedClassNames for legacy API rendering

- Update ESLint configuration:
  * Add test-apps/** to ignore patterns to prevent linting scaffolded apps
  * Add create-bestax/test-results/** to ignore patterns

- Fix create-bestax lint script:
  * Change from 'eslint .' to 'eslint src e2e/tests e2e/utils --ext .ts,.tsx'
  * Prevents linting of test-results and other generated files

- Add create-bestax/.gitignore to exclude test artifacts

All changes ensure React Hooks are called unconditionally at component top level
by using the non-hook prefixedClassNames(classPrefix, ...) function for
conditional rendering, while calling useConfig() once at the top level.

Resolves ESLint hanging issue where it was attempting to lint thousands of
files in test-apps directory.
Create custom GitHub Action that uses GitHub API to create verified commits.
This replaces git CLI commits with API-based commits that are automatically
signed by GitHub's web-flow GPG key, satisfying branch protection requirements.

Uses only GitHub's official actions/github-script action - no third-party
dependencies.
…ction

The glob module is not available in actions/github-script environment.
Replace with native find command for file pattern matching.
Extract base directory from glob pattern before running find command.
This allows find to search subdirectories like app.spec.ts-snapshots/.
@allxsmith
allxsmith force-pushed the feat/94-visual-regression-testing branch from 8d3d159 to a626d20 Compare October 17, 2025 00:11
@allxsmith
allxsmith merged commit 7dae8ff into main Oct 17, 2025
3 checks passed
@allxsmith
allxsmith deleted the feat/94-visual-regression-testing branch October 17, 2025 00:25
@github-actions

Copy link
Copy Markdown
Contributor

🎉 This PR is included in version 2.4.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@github-actions

Copy link
Copy Markdown
Contributor

🎉 This PR is included in version 1.2.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@github-actions

Copy link
Copy Markdown
Contributor

🎉 This PR is included in version 1.0.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

allxsmith added a commit that referenced this pull request Aug 28, 2026
Two gaps Copilot found in the field sanitizer, both real.

The credential "defang" was worse than doing nothing. Entity-encoding the
underscore in a credential-shaped match left `ghs_...` as `ghs_...`, which
GitHub renders and copies back as the original token - and because it ran
before findCredentialLeak, it also hid the value from the exact-value
comparison against the job's own secrets. So a live GITHUB_TOKEN in a title
would have been published, defeating the check added to catch exactly that.
Sanitizing a secret was the wrong instinct: the caller REFUSES to publish a
body carrying one, and that only works while the value survives intact.
Removed; both the exact-value and shape checks see it again.

Escaping `<`/`>` did not cover Markdown structure. CommonMark decodes
character references inside an explicit link destination, so
`[click](https&#58;//evil.example)` still rendered a live link, and `![](...)`
a tracking beacon under bestaxbot - the cross-reference silence step 7 claims
was not delivered. Link and image delimiters are now escaped, backslashes
FIRST so that model text of `\[` cannot become `\\[` and reopen the
delimiter, and GFM's bare `www.` autolink (a raw scan that decodes no
entities) gets its own break. An ordinary `[Refactor] ...` title still reads
correctly.

Also, two comments that no longer described their mechanism:

- findMarkerComment's JSDoc still said it searches back for the latest marker
  comment with a parseable target. It stops at the latest marker comment
  either way and returns null for a retraction, which is the whole point of
  the change - a reader following the old wording would reintroduce the
  fall-through.
- The triage checkout said the job "never runs project code" while the render
  step executes scripts/render-triage-comment.mjs beside both tokens. The
  boundary that actually holds is narrower and now stated: the code is pinned
  to the default branch, so no PR author can substitute their own.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Add Visual Regression Testing for create-bestax

2 participants