Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add feature tests via Behat #6070

Merged
merged 43 commits into from
Apr 27, 2021
Merged

Add feature tests via Behat #6070

merged 43 commits into from
Apr 27, 2021

Conversation

schlessera
Copy link
Collaborator

@schlessera schlessera commented Apr 14, 2021

Summary

This PR adds feature testing via Behat to the plugin. It reuses the testing infrastructure and Behat definitions from WP-CLI (see https://github.com/wp-cli/wp-cli-tests).

Before the tests are run for the first time, the following command has to be executed:

composer prepare-tests

This should create the needed table(s) in the database for the tests to be run within.

Database credentials can be provided via environment variables if the database is not found on localhost:3306:

  • WP_CLI_TEST_DBHOST is the host to use and can include a port, i.e 127.0.0.1:33060 (defaults to localhost)
  • WP_CLI_TEST_DBROOTUSER is the user that has permission to administer databases and users (defaults to root).
  • WP_CLI_TEST_DBROOTPASS is the password to use for the above user (defaults to an empty password).
  • WP_CLI_TEST_DBUSER is the user that the tests run under (defaults to wp_cli_test).
  • WP_CLI_TEST_DBPASS is the password to use for the above user (defaults to password1).

Then, to run the tests, the following command can be used:

vendor/bin/behat

Note: The code coverage is currently not being executed on GHA because there is still an issue with the interpretation of the code coverage data within codecov (probably a path discrepancy). This should be fixed and re-enabled in a separate PR.

Fixes #3076

Checklist

  • My code is tested and passes existing tests.
  • My code follows the Engineering Guidelines (updates are often made to the guidelines, check it out periodically).

@schlessera schlessera added Infrastructure Changes impacting testing infrastructure or build tooling Testing Issues related with Unit, E2E, Smoke, and other testing requirements/needs labels Apr 14, 2021
@codecov
Copy link

codecov bot commented Apr 14, 2021

Codecov Report

Merging #6070 (17d8501) into develop (8d1425e) will decrease coverage by 3.83%.
The diff coverage is 0.00%.

❗ Current head 17d8501 differs from pull request most recent head 07a844f. Consider uploading reports for the commit 07a844f to get more accurate results
Impacted file tree graph

@@              Coverage Diff              @@
##             develop    #6070      +/-   ##
=============================================
- Coverage      75.03%   71.20%   -3.84%     
+ Complexity      5829     5827       -2     
=============================================
  Files            232      233       +1     
  Lines          17618    18566     +948     
=============================================
  Hits           13219    13219              
- Misses          4399     5347     +948     
Flag Coverage Δ Complexity Δ
feature 0.00% <0.00%> (?) 0.00 <0.00> (?)
javascript 79.84% <ø> (ø) 0.00 <ø> (ø)
php 70.83% <0.00%> (-3.99%) 5827.00 <0.00> (-2.00)
unit 74.78% <0.00%> (?) 5827.00 <0.00> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ Complexity Δ
includes/bootstrap.php 0.00% <0.00%> (ø) 0.00 <0.00> (?)
src/Cli/TransformerCommand.php 0.00% <ø> (ø) 29.00 <0.00> (-2.00)
src/Admin/PluginActivationNotice.php 59.09% <0.00%> (-40.91%) 7.00% <0.00%> (ø%)
...cludes/embeds/class-amp-playlist-embed-handler.php 77.06% <0.00%> (-18.40%) 35.00% <0.00%> (ø%)
src/AmpWpPlugin.php 85.18% <0.00%> (-14.82%) 9.00% <0.00%> (ø%)
src/DevTools/CallbackReflection.php 85.00% <0.00%> (-12.15%) 17.00% <0.00%> (ø%)
...s/sanitizers/class-amp-accessibility-sanitizer.php 70.58% <0.00%> (-9.42%) 6.00% <0.00%> (ø%)
src/Admin/PluginRowMeta.php 90.90% <0.00%> (-9.10%) 4.00% <0.00%> (ø%)
src/DevTools/LikelyCulpritDetector.php 90.90% <0.00%> (-9.10%) 10.00% <0.00%> (ø%)
includes/sanitizers/class-amp-img-sanitizer.php 88.77% <0.00%> (-8.88%) 87.00% <0.00%> (ø%)
... and 91 more

@westonruter westonruter added this to the v2.1 milestone Apr 15, 2021
@schlessera schlessera marked this pull request as ready for review April 25, 2021 09:55
@github-actions
Copy link
Contributor

github-actions bot commented Apr 25, 2021

Plugin builds for 07a844f are ready 🛎️!

@westonruter
Copy link
Member

I tried running composer prepare-tests but I got:

> install-package-tests
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
Detected MySQL at version .
/app/public/content/plugins/amp/vendor/bin/install-package-tests: 75: [: Illegal number:
+ mysql -e CREATE DATABASE IF NOT EXISTS `wp_cli_test`; -uroot
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
Script install-package-tests handling the prepare-tests event returned with error code 1

@westonruter
Copy link
Member

I guess that's because my environment variables are empty.

Copy link
Member

@westonruter westonruter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests are working in CI context, so that's the most important thing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Infrastructure Changes impacting testing infrastructure or build tooling Testing Issues related with Unit, E2E, Smoke, and other testing requirements/needs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add functional tests for CLI commands
3 participants