[Feature] Run codeception tests in a dockerized environment

[LucasDemea]

Is your feature request related to a problem? Please describe.
It would be nice if we could test our plugin right away without having to rely on a preinstalled wordpress site.

Describe the solution you'd like
I'd like to use the official wordpress docker image to run the tests with a single command that would that care of building the container, launch selenium, and run the tests. It would also allow a better integration in our CI pipelines.

I'm almost ready with setting up this solution in one of my projects, and would be happy to submit a PR with a generalized solution, if you think that would be an improvement to WPBP.

[Mte90]

Yes of course!
I don't use docker for development (VVV) and for CI I have github/gitlab/codeship that run the codecept command.
So go ahead I will add also an option to the https://github.com/WPBP/generator for it :-)

[LucasDemea]

Shall we discuss about the requirement then ?
How do you actually use your VVV environment ? Is there any reason you need this instead of a fresh wp install (as WPLoader does) for running your acceptance & functional tests ?
IMHO, in the use case of testing a plugin, we don't care about having a preconfgured site installed, we could just go with a fresh wp install at each run.
What are your thoughts about this ?

[Mte90]

I have no thoughts about it as I don't use docker at all and I have no bias for tests.
I use VVV for development with vagrant so I run everything inside it (with another db) and for other stuff CI.
So I think that the best way is do a dockerfile with everything and test it.

As you see also the CI of the boilerplate doesn't run the tests https://github.com/WPBP/WordPress-Plugin-Boilerplate-Powered/blob/master/.github/workflows/integrate.yml so it is complete from this side.

[MikeiLL]

Been using wp-env, which is dockerized and a pretty sweet package.

[erikyo]

@Mte90 I prepared a POC that can solve this issue, can install the codeception tests and run them inside wp-env... anyhow seems something is wrong because it returns there are 0 tests executed... am i doing something wrong?

https://github.com/erikyo/WordPress-Plugin-Boilerplate-Powered/blob/32da4b59a2896330da9a47917e08e177583f9c38/plugin-name/package.json#L34-L35

[Mte90]

I think that isn't detecting the codeception settings.
The CI is working so probably something is missing https://github.com/WPBP/WordPress-Plugin-Boilerplate-Powered/blob/master/.github/workflows/integrate.yml

[erikyo]

yeah i see... looks like some env variables were switched during the ci workflow

WordPress-Plugin-Boilerplate-Powered/.github/workflows/integrate.yml

Lines 215 to 216 in ccd199f

	rm tests/_envs/.env
	mv tests/_envs/.env.testing tests/_envs/.env

and the original one doesn't fit the wp-env ports / domain etc.

[Mte90]

If you run codecept with -vvv you get the full debug log with information like if loaded the codeception configuration file and so on.

[JanKruft]

@erikyo i guess if you run the bootstrap command the yml is created in root instead of plugin folder. Also on the run command. This is why no tests are found.
If you want to have codecept running in plugin folder use.
"wp-env:test": "npm run wp-env run tests-wordpress -- --env-cwd=wp-content/plugins/my-plugin-name vendor/bin/codecept run".
But im stucking at the issue that wordpress-test container throws Db: could not find mysql driver while creating PDO connection.
wp-env dosnt allow me to create a dockerfile and manipulating the docker-compose.yml created by wp-env is a bad DX.

[Mte90]

As I can see on https://developer.wordpress.org/block-editor/reference-guides/packages/packages-env/#customizing-the-wp-tests-config-php-file there is the file with the parameters to use the docker env to access the DB for tests so probably needs to be updated inside the plugin folder to fix the issue.

[JanKruft]

@Mte90 the envs are not the Problem. the missing mysql pdo inside the container the is currently my issue. May i need to swicht back to selfmade docker setup instead of @wordpress/env

[Mte90]

Did you opened a ticket to the wp-env project for that?