Set up end-to-end testing

[tcompa]

This is not the highest priority, but it will be nice to have it a later stage.

Obvious benefits:

• It will be easier to catch errors due to breaking updates of fractal-server.
• It will be easier to maintain fractal-web, since small interface changes would be "guaranteed" to not break the underlying logic.

[tcompa]

In my view (and partly after a discussion with @mfranzon), the list below would be a good starting point. Note: I did not check the technical feasibility of all ideas, but this is at least a starting point.

1. We should not address unit testing of single components, for the moment.
2. We should start with a handful of integration tests that cover the typical use cases, e.g.
   • The user can log-in.
   • The user can log-in, collect tasks, create a project, modify its datasets, create another dataset, create/import a workflow, modify an argument of a task, submit the workflow for execution.
3. The test suite should also be responsible for configuring/starting/stopping fractal-server instances.
4. The tests should run as a github action, and also locally.
5. The tests should be run with different browsers.
6. The first goal would be to understand and setup the testing framework, and not to aim to large coverage. Even just the first test ("the user can log-in") running both locally and on github (and taking care of point 3) would be great, and then we can build on it in the future.

Playwright looks like the standard tool to use, so let's check whether we can set it up as in 6. Some relevant links, related to the points above:

• https://playwright.dev/docs/ci-intro (ref point 4)
• https://playwright.dev/docs/ci (ref point 4)
• https://playwright.dev/docs/test-webserver#multiple-web-servers (ref point 3)
• https://playwright.dev/docs/browsers (ref point 5)

[tcompa]

We recently hit this error, related to a non-secure context:

• crypto.randomUUID not available outside secure context #231

How would we catch it in a playwright test? To be reviewed (in the future)

[tcompa]

Here are some more thoughts on (playwright) testing, to streamline future discussions.

The first milestone IMHO is:

1. To have a basic test defined,
2. To have it run both locally and as a GitHub action,
3. To add a browser matrix in the GitHub action.

The basic test may look like:

• We start up a new fractal-server instance, with its (clean) sqlite db
• We start up a fractal-web instance
• A user browses to the log-in page, types their credentials, clicks the login button.
• The user is logged in.
• The user browses to the projects page, types the name of a new project, clicks the create-project button.
• The project is created, and is visible in the project list.
• The user clicks the “delete project” button.
• The project is deleted, and is not present in the project list.

[tcompa]

A few points that came up today while discussion with @zonia3000:

1. Basic github action config: https://playwright.dev/docs/ci-intro#setting-up-github-actions
2. The playwright install step also fetches multiple browsers, and it may be too heavy for GitHub action. As soon as needed, let's make sure we include some cache mechanism (ref [Question]: How to cache on github actions? microsoft/playwright#7249)
3. The ideal coverage report should merge three sources:
   • vitest unit tests (already included in a work-in-progress version)
   • the client side of playwright end-to-end tests (already included in a work-in-progress version)
   • the server side of playwright end-to-end tests -> this is currently ignored. Just for the record, would an update like this work? I'm not suggesting we go in that direction (as it requires a change in the code base which is only done in view of testing), but it may be part of the ongoing exploration.