Add smoke-test workflow

[uuf6429]

Purpose

The objective is to provide an extendable workflow for testing basic compatibility* of different drivers in isolation concurrently.

* latest drivers and oldest supported dependencies

Third-party driver developers can also add a test themselves (in ci-test-unofficial.yml), but will be treated a little bit differently, to avoid failures reflecting badly on our CI results (since GitHub doesn't allow us to ignore failures).

Current Driver Tests

• ✔️ BrowserKit (http client)
• ✔️ BrowserKit (http kernel)
• ✔️ Selenium2
• 🚧 Chrome
  • many failures without a seemingly good reason (their CI passes fine though)
  • (it's worth noting that their custom tests use data uris instead of a webserver)
• ✔️ Classic
• ♻️ Sahi / Zombie / Goutte / Electron - deprecated/abandoned
• ❓ Panther

[uuf6429]

I just pushed a major update; it should work much better now and should be relatively easy to follow.

There's one major annoyance (on GitHub's side): they do not evaluate expressions in job.name, (that line above), when the job is skipped.

This causes some ugly UI:

But the impact is just that, so I think we can live with it.

• https://github.com/orgs/community/discussions/13261
• job.name expression not evaluated when job skipped actions/runner#1215

[stof]

A way to solve that would be to put those jobs in a separate workflow instead (that can be dispatched manually) instead of putting them in the CI workflow but skipping them anytime it is not dispatched manually. This would not cause duplication, as the common logic is already in the reusable workflow anyway.

[uuf6429]

Good idea, I'll try implementing it so.

[uuf6429]

@stof I tried it out on my personal branch and it looked like it works really nicely. Can't see any downsides so far, in fact (except that the current "CI" entry will become obsolete, but I think that's really minor).

It can be removed if we manually remove the past workflow logs, it's a bit cumbersome but I can do that if you want (just let me know). Or we can leave it there for now (maybe github even removes it eventually, when it's old enough).

[aik099]

Looks almost ready. At least all builds pass.

[uuf6429]

Looks almost ready. At least all builds pass.

@aik099 sadly I still have issues with the chrome driver. Interestingly, I managed to replicate the same errors locally yesterday. I'll spend a day or two to fix it and if I can't, I'll ask the maintainer and merge this PR.

[uuf6429]

@acoulton after many attempts at getting the chrome driver tests to run, I finally give up.

The testcase is still there if you feel like giving it a look. I'll probably remove it in a week or two if there is no interest in getting it to work.

[acoulton]

@uuf6429 I wasn't aware you were working on this till you pinged me here. I could probably have pointed you in the right direction and saved you some time.

For the record, I am not a maintainer on chrome-mink-driver: we use it heavily for work (and have done for a long time) so I've built up a good bit of experience of some of the challenges of getting Chrome running happily with mink, and I contribute to the driver when I can.

I can try to take a look at this once I've finished the work I'd already started to migrate chrome-mink-driver's own tests to docker compose, which I think will make life easier all round. It may take me longer than the next week or two, but that doesn't mean there's "no interest in getting it to work", it just means it's voluntary open source work and it depends what else comes up.

However the link you provided to the testcase in your comment appears to just go to the full diff of this PR, and I can't see a testcase there?

[uuf6429]

@acoulton it's here now: .github/workflows/ci-test-unofficial.yml

Feel free to ask if anything is unclear. There's also no real time constraints, of course. What I meant is that I don't want to be the one to add something half-baked that is never used.