Add build tool to check ruleset.xml for obsolete exclude patterns and optionally fix the file #40278

richard67 · 2023-04-01T10:44:59Z

Pull Request for Issue # .

Summary of Changes

This pull request (PR) adds a new, small tool (PHP CLI script) for checking any ruleset XML file for phpcs, by default the "ruleset.xml" file in the Joomla root, if it contains exclude patterns for files or folders which don't exist anymore because some PR has removed or moved that file or folder, and optionally fix the file.

I have used this tool to create PR's #40276 and #40277 .

I have used file() to read the file and not simplexml_load_file because I want to output the line number of the XML file to make it easier to manually review and fix the file.

The tool is limited to handle exclude patterns like we currently use them in the "ruleset.xml" file.

E.g. for folders: https://github.com/joomla/joomla-cms/blob/4.3-dev/ruleset.xml#L6-L17

Or files: https://github.com/joomla/joomla-cms/blob/4.3-dev/ruleset.xml#L32

I.e. anything which ends with "/" or "/*" is a folder, and anything else is a file, and any dots are escaped.

If in future more sophisticated expressions will be used in exclude patterns, the tool has to be adapted to that.

As the tool is a maintainer tool in the "build" folder, it doesn't go into any installation or update package, and so any feature freeze policy for the 4.3-dev branch due to the RC phase doesn't apply here.

For the same reason it does not need any documentation on docs.joomla.org .

It could make sense to add some documentation to manual.joomla.org if the tool becomes part of the regular release workflow.

But maybe that's not really necessary because it is so small that it's quite self explaining, and it has a description in the top comment block and a --help option to show the help at the command line.

Testing Instructions

Copy the one file added by this PR to a branch of your choice into the build folder.

Then run the tool at the command line, e.g. with php ./build/check_ruleset_xml.php when in the Joomla root folder.

Show the help: php ./build/check_ruleset_xml.php --help.
Check the "ruleset.xml" file in the Joomla root folder without fixing the file: php ./build/check_ruleset_xml.php.
Check the "ruleset.xml" file in the Joomla root folder and fix the file: php ./build/check_ruleset_xml.php --fix.
Check the "ruleset.xml" file in the "build/psr12" folder: php ./build/check_ruleset_xml.php --file ./build/psr12/ruleset.xml.
Free test: Feel free to add some exclude patterns for files and folders which don't exist to some ruleset XML file, then check that file as shown above in steps 2 or 3, depending on which file you have modified or created.

Actual result BEFORE applying this Pull Request

There is no tool to check if exclude patterns in a ruleset XML file for phpcs are still relevant or obsolete.

Expected result AFTER applying this Pull Request

There is a PHP CLI script "check_ruleset_xml.php" in the "build" folder.

Show the help:

php ./build/check_ruleset_xml.php --help

Usage: php ./build/check_ruleset_xml.php [options]

[options]:
--file <path>:	Path to the PHPCS ruleset XML file to be checked, defaults to '/home/richard/lamp/public_html/joomla-cms-4.3-dev/ruleset.xml'.
--fix:		Fix the XML file if any obsolete exclude patterns were found.
--help:		Show this help output.

(Of course it shows a different path as default on your environment)

Check the "ruleset.xml" file in the Joomla root folder without fixing the file.

On a clean, current 4.3-dev branch with PR #40276 merged:

php ./build/check_ruleset_xml.php
Checking file '/home/richard/lamp/public_html/joomla-cms-4.3-dev/ruleset.xml' ...
... done.
No obsolete lines found.

On a clean, current 4.3-dev branch using the ruleset.xml file from before PR #40276 was merged:

Checking file '/home/richard/lamp/public_html/joomla-cms-4.3-dev/ruleset.xml' ...
Line no. 123: File "plugins/editors/tinymce/tinymce.php" doesn't exist.
Line no. 124: File "plugins/system/cache/cache.php" doesn't exist.
Line no. 239: File "plugins/content/emailcloak/emailcloak.php" doesn't exist.
Line no. 243: File "plugins/editors/none/none.php" doesn't exist.
... done.

You can download the ruleset.xml file from before PR #40276 was merged here: https://test5.richard-fath.de/ruleset.xml

Check the "ruleset.xml" file in the Joomla root folder and fix the file.

On a clean, current 4.3-dev branch using the ruleset.xml file from before PR #40276 was merged:

php ./build/check_ruleset_xml.php --fix
Checking file '/home/richard/lamp/public_html/joomla-cms-4.3-dev/ruleset.xml' ...
Line no. 123: File "plugins/editors/tinymce/tinymce.php" doesn't exist.
Line no. 124: File "plugins/system/cache/cache.php" doesn't exist.
Line no. 239: File "plugins/content/emailcloak/emailcloak.php" doesn't exist.
Line no. 243: File "plugins/editors/none/none.php" doesn't exist.
... done.
Updating file '/home/richard/lamp/public_html/joomla-cms-4.3-dev/ruleset.xml' ...
... done.

Check the "ruleset.xml" file in the "build/psr12" folder. On a clean, current 4.3-dev branch:

php ./build/check_ruleset_xml.php --file ./build/psr12/ruleset.xml
Checking file './build/psr12/ruleset.xml' ...
Line no. 120: File "plugins/editors/tinymce/tinymce.php" doesn't exist.
Line no. 121: File "plugins/system/cache/cache.php" doesn't exist.
Line no. 236: File "plugins/content/emailcloak/emailcloak.php" doesn't exist.
Line no. 240: File "plugins/editors/none/none.php" doesn't exist.
... done.

Link to documentations

Please select:

No documentation changes for docs.joomla.org needed
No documentation changes for manual.joomla.org needed
If someone points me to the right place at joomla/internal-documentation, i.e. in which file to which section I should add something, I can make a PR for that there if maintainers require that.

brianteeman · 2023-04-01T11:24:12Z

LGTM

laoneo · 2023-05-10T11:02:52Z

Should this tool not be executed somewhere automatically in the development process. Not sure when, but if we have to execute it manually for every minor release it will get forgotten along the release build process.

richard67 · 2023-05-10T11:56:43Z

Should this tool not be executed somewhere automatically in the development process. Not sure when, but if we have to execute it manually for every minor release it will get forgotten along the release build process.

@laoneo The tool shall advise the user (maintainer or release lead). The user still has to review the result and manually make the correction. The tool doesn't fix the file. I could add a "--fix" option for the latter. But where to call or use or document the tool is not my business since it's the release manager who have to use it. If you want to block this PR with these discussions I can close it. Right now we have nothing, no tool. With this PR we have some, that's an improvement.

richard67 · 2023-05-12T15:42:12Z

I've added an option to fix the file and have updated testing instructions. Please test. Thanks in advance.

richard67 · 2023-05-18T10:50:10Z

Should this tool not be executed somewhere automatically in the development process. Not sure when, but if we have to execute it manually for every minor release it will get forgotten along the release build process.

@laoneo The tool shall advise the user (maintainer or release lead). The user still has to review the result and manually make the correction. The tool doesn't fix the file. I could add a "--fix" option for the latter. But where to call or use or document the tool is not my business since it's the release manager who have to use it. If you want to block this PR with these discussions I can close it. Right now we have nothing, no tool. With this PR we have some, that's an improvement.

Update: I've added a --fix option so the tool can be used in an automated action to create a pull request, similar to the translation updates. But I think the automated action should be created with a follow-up pull request, not with this one here.

Regarding documentation: If someone points me to the right place at joomla/internal-documentation, i.e. in which file to which section I should add something, I can make a PR for that there.

laoneo · 2023-09-26T05:48:15Z

Thanks!

* Added api test for user field group (#41850) * Added api test for user field group (#41851) * system test com_config webservices (#41765) * system test com_config * id * divideandrollback * grr * rollback --------- Co-authored-by: Allon Moritz <[email protected]> * [4.3] WebauthnField missing translation (#41723) * redo of #41520 (#41856) * [4.4] add system tests for admin users (group/level) (#41910) * Use $document variable (#41917) * redirect (#41912) * Cypress test for com_privacy admin (#41834) * Cypress test for com_privacy admin * Update Consent.cy.js * [4.4] 500 error when 404 and debug on (#41893) * fix * missed new params * [4] Mismatched tag ids/tag names in articles webservice/api (#41397) * Mismatched tag ids/tag names in articles webservice/api * redone * new function * use the new helper function * cs --------- Co-authored-by: Martin Carl Kopp <[email protected]> * [4.4] Joomla Update Component check package when using Upload & Update (#41757) * Add checks of uploaded file to com_joomlaupdate * Use global namespace * Simplify language strings * Remove the major version check * Fix doc block of new method * Language string improvements - thanks Brian Co-authored-by: Brian Teeman <[email protected]> * Only check if zip PHP extension is loaded * Check manifest XML instead of Version.php * English improvements * Remove empty line from language file Co-authored-by: George Wilson <[email protected]> * Add alternative method if no zip PHP extension * Do it as elswehere * Remove unsupported compression method * Make sure file is opened as binary file * Remove special version suffix for pull request patched packages * Move version check to own method * Read chunks of max. 1MiB from central directory * Fix method name and calls * Fix code from tests with a CLI script * Add code comments * Small code simplification * Fix empy manifest XML file handling * Handle empty manifest file and start of file * Fix method description comment * Close file when throwing exceptions while open * Fix typo in code comment * Fix code comment. Co-authored-by: Quy <[email protected]> * Language string consistency --------- Co-authored-by: Brian Teeman <[email protected]> Co-authored-by: George Wilson <[email protected]> Co-authored-by: Allon Moritz <[email protected]> Co-authored-by: Quy <[email protected]> * Update template.es6.js (#41909) Optional chaining fix * Add build tool to check ruleset.xml for obsolete exclude patterns and optionally fix the file (#40278) * Add tool to check ruleset.xml for obsolete exclude patterns * Fix PHPCS * Add option to fix the XML file * Rename to check_ruleset_xml.php * Better help text for default value * Fix comment, too * [4][webservice] user patch (#41688) * [4] webservice com_user patch PATCH user alters password * system test --------- Co-authored-by: Allon Moritz <[email protected]> * Update tests/System/integration/administrator/components/com_users/Groups.cy.js Co-authored-by: heelc29 <[email protected]> * Update tests/System/integration/administrator/components/com_users/Levels.cy.js Co-authored-by: heelc29 <[email protected]> * Use maximebf/debugbar v1.19.0 (#41931) * Rebuild composer lock file * Explanation why we mix UI with API tests * Rebuild composer lock file because of webauth upgrade * Fix code style --------- Co-authored-by: rajputanuj31 <[email protected]> Co-authored-by: Nicola Galgano <[email protected]> Co-authored-by: heelc29 <[email protected]> Co-authored-by: Richard Fath <[email protected]> Co-authored-by: andyforrest <[email protected]> Co-authored-by: Denitz <[email protected]> Co-authored-by: Martin Carl Kopp <[email protected]> Co-authored-by: Brian Teeman <[email protected]> Co-authored-by: George Wilson <[email protected]> Co-authored-by: Quy <[email protected]> Co-authored-by: Olivier Buisard <[email protected]>

Add tool to check ruleset.xml for obsolete exclude patterns

cecae75

joomla-cms-bot added the PR-4.3-dev label Apr 1, 2023

This was referenced Apr 1, 2023

[4.3] Remove obsolete exclude patterns from ruleset.xml for phpcs #40276

Merged

[4.4] Remove obsolete exclude pattern from ruleset.xml for phpcs #40277

Merged

Fix PHPCS

39eb211

Hackwar added the bug label Apr 8, 2023

Merge branch '4.3-dev' into 4.3-dev-add-tool-for-ruleset-xml-check

b53b014

Add option to fix the XML file

7d6b44e

richard67 changed the title ~~Add build tool to check ruleset.xml for obsolete exclude patterns~~ Add build tool to check ruleset.xml for obsolete exclude patterns and optionally fix the file May 12, 2023

Merge branch '4.3-dev' into 4.3-dev-add-tool-for-ruleset-xml-check

cb142dc

richard67 added 4 commits May 18, 2023 12:21

Rename to check_ruleset_xml.php

282c8bb

Merge branch '4.3-dev' into 4.3-dev-add-tool-for-ruleset-xml-check

a29fba9

Better help text for default value

837722c

Fix comment, too

4b03a59

richard67 added 2 commits May 24, 2023 18:52

Merge branch '4.3-dev' into 4.3-dev-add-tool-for-ruleset-xml-check

875989e

Merge branch '4.3-dev' into 4.3-dev-add-tool-for-ruleset-xml-check

913b71d

richard67 changed the base branch from 4.3-dev to 4.4-dev September 20, 2023 12:25

Merge branch '4.4-dev' into 4.3-dev-add-tool-for-ruleset-xml-check

f480352

joomla-cms-bot added the PR-4.4-dev label Sep 20, 2023

richard67 removed the PR-4.3-dev label Sep 20, 2023

laoneo added this to the Joomla! 4.4.0 milestone Sep 26, 2023

laoneo merged commit b9390c5 into joomla:4.4-dev Sep 26, 2023

richard67 deleted the 4.3-dev-add-tool-for-ruleset-xml-check branch September 26, 2023 06:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Add build tool to check ruleset.xml for obsolete exclude patterns and optionally fix the file #40278

Add build tool to check ruleset.xml for obsolete exclude patterns and optionally fix the file #40278

Uh oh!

richard67 commented Apr 1, 2023 •

edited

Loading

Uh oh!

brianteeman commented Apr 1, 2023

Uh oh!

laoneo commented May 10, 2023

Uh oh!

richard67 commented May 10, 2023

Uh oh!

richard67 commented May 12, 2023

Uh oh!

richard67 commented May 18, 2023 •

edited

Loading

Uh oh!

laoneo commented Sep 26, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Add build tool to check ruleset.xml for obsolete exclude patterns and optionally fix the file #40278

Add build tool to check ruleset.xml for obsolete exclude patterns and optionally fix the file #40278

Uh oh!

Conversation

richard67 commented Apr 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of Changes

Testing Instructions

Actual result BEFORE applying this Pull Request

Expected result AFTER applying this Pull Request

Link to documentations

Uh oh!

brianteeman commented Apr 1, 2023

Uh oh!

laoneo commented May 10, 2023

Uh oh!

richard67 commented May 10, 2023

Uh oh!

richard67 commented May 12, 2023

Uh oh!

richard67 commented May 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

laoneo commented Sep 26, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

richard67 commented Apr 1, 2023 •

edited

Loading

richard67 commented May 18, 2023 •

edited

Loading