[4.4] Add database configuration option for "sql_big_selects" session variable on MySQL or MariaDB databases - alternative

[richard67]

Pull Request for Issue #41156 .

See also issues #17580 and #39479 , discussion #42198 and support forum thread https://forum.joomla.org/viewtopic.php?f=811&t=995872 .

Alternative approach to PR #42557 .

Summary of Changes

This pull request (PR) adds a new option to the "Database" section of Global Configuration which allows to set the "sql_big_selects" session variable to on or off or leave it untouched, which is the default.

In opposite to PR #42557 , this one here doesn't require an update of the database dependency from the framework, so it can be used in case if the framework PR joomla-framework/database#292 gets rejected for some reason.

So when people run into the 1104 The SELECT would examine more than MAX_JOIN_SIZE rows ... error with their database server, they can fix that by switching the new option to "Yes".

When the default value of the option is used, which is also the case after a CMS update to a new version which includes this PR here, then nothing is done, so there is no unnecessary SQL statement issues with each connection in that case.

The issue has been reported more frequently in recent times, so I think we should fix it as a bug fix in 4.4-dev and merge it up later.

I have made the additional feature PR #42559 for the 5.1-dev branch to show the values of the "sql_big_selects" and "max_join_size" variables in the System Information on MySQL or MariaDB databases.

Testing Instructions

This PR is only relevant for MySQL or MariaDB databases, not for PostgreSQL.

For some test steps which change the global "sql_big_select" variable it needs to have administrator privileges for the database. E.g. on a local MySQL you have to connect with user "root" when using e.g. phpMyAdmin. That means that these test steps can't be executed in a shared hosting environment.

1. Add the following line of code to the "index.php" file of your site template below the use statements in the PHP code at the top, e.g. if Cassiopeia is used file "templates/cassiopeia/index.php" at line 20, to show the current value of the "sql_big_select" session variable:

echo 'TEST: ' . Factory::getDbo()->setQuery('SELECT @@sql_big_selects;')->loadResult() . '<br>';

2. Go to the site (frontend) and note the current value of the "sql_big_select" session variable shown at the top after the text "TEST: ".
3. Check the global and session variables "sql_big_selects" in a client like e.g. phpMyAdmin:

SELECT @@GLOBAL.sql_big_selects, @@SESSION.sql_big_selects;

4. Check that the values from step 3 are the same as the value from step 2.
5. Apply the changes from this PR.
   You can find packages for new installation and update and a custom update URL here: https://artifacts.joomla.org/drone/joomla/joomla-cms/4.4-dev/42558/downloads/72502 .
6. In the administrator, go to Global Configuration, tab "server", section "Database" and check the options at the end of that section.
   Result: There is a new list select option "Allow large queries", and the first value "No change (server controlled)" is selected, which is the default,
7. On site, check again the current value of the "sql_big_selects" session variable as described in step 2. If the page is still open, reload the page to get the current value.
   Result: The value of the "sql_big_selects" session variable has not changed, it is still the same as in step 2.
8. In the administrator in Global Configuration, change the value of the new "Allow large queries" option so that the value of the "sql_big_selects" session variable should change. I.e. if the value recorded in step 2 was "0" select "Yes" and if the value from step 2 was "1" select "No", and save the changes.
9. On site, check again the current value of the "sql_big_selects" session variable as described in step 2. If the page is still open, reload the page to get the current value.
   Result: The value of the "sql_big_selects" session variable has changed to the desired value (0 for "No" and 1 for "Yes").
10. This step requires administrator privileges for the database, e.g. user "root".
    In a client like e.g. phpMyAdmin, change the value of the "sql_big_selects" global variable so that the session variable should change. E.g. if the current value of the session variable check in step 9 was 0, change the global variable to 1, and vice versa.
    Example for changing to 1:

SET GLOBAL sql_big_selects=1;

11. On site, check again the current value of the "sql_big_selects" session variable as described in step 2. If the page is still open, reload the page to get the current value.
    Result: The value of the "sql_big_selects" session variable has not changed, it is still the same as selected in step 9.
12. In the administrator in Global Configuration, change the value of the new "Allow large queries" option back to the default "No change (server controlled)".
13. On site, check again the current value of the "sql_big_selects" session variable as described in step 2. If the page is still open, reload the page to get the current value.
    Result: Now the value of the "sql_big_selects" session variable has changed compared to steps 9 and 11. It is equal to the value set in step10 for the global variable.
14. In the administrator in Global Configuration, change the value of the new "Allow large queries" option so that the value of the "sql_big_selects" session variable should change. I.e. if the value recorded in step 13 was "0" select "Yes" and if the value from step 13 was "1" select "No", and save the changes.
15. If you want, continue to play around with the global variable as described in step 10 and the different values of the new Global Configuration option.
    Result: When the new option has its default value "No change (server controlled)", the "sql_big_selects" session variable has the value as set by the server with the global variable.
    If the new option has value "No", the "sql_big_selects" session variable is set to 0.
    If the new option has value "Yes", the "sql_big_selects" session variable is set to 1.
16. Clean up the changes to the global variable from step 10:
    Either just restart the database sever, or set the global variable back to the value which it had in step 3.

Actual result BEFORE applying this Pull Request

There is no way to change the value of the "sql_big_select" session variable.

Expected result AFTER applying this Pull Request

A new list select option "Allow large queries" is available at the end of the "Database" section in the "Server" tab of Global Configuration ($dbsqlbigselects in configuration.php).

The option is only shown when the selected database type is "MySQLi" or "MySQL (PDO)". When "PostgreSQL (PDO)" is selected, the new option is hidden.

The value of that option is the default value "No change (server controlled)" after a new installation or an update.

When the new option has that default value, the value of the "sql_big_selects" session variable is not changed by the CMS, and no additional SQL statement is performed when connecting to the database.

When the new option has value "No", the value of the "sql_big_selects" session variable is set to 0 when connecting to the database.

When the new option has value "Yes", the value of the "sql_big_selects" session variable is set to 1 when connecting to the database. This will prevent the SQL error 1104 The SELECT would examine more than MAX_JOIN_SIZE rows ....

Link to documentations

Please select:

• Documentation link for docs.joomla.org: https://docs.joomla.org/Help4.x:Site_Global_Configuration https://docs.joomla.org/Help4.x:Site_Global_Configuration_Server

• No documentation changes for docs.joomla.org needed

• Pull Request link for manual.joomla.org:

• No documentation changes for manual.joomla.org needed

A description of the new "Allow large queries" parameter should be added to the end of sub-section "database" of the "Server" section on https://docs.joomla.org/Help4.x:Site_Global_Configuration / https://help.joomla.org/proxy?keyref=Help44:Site_Global_Configuration&lang=en .

Suggested text:

• Allow large queries
  -- "No change (server controlled)" (default)
  -- "No"
  -- "Yes"
  Use "Yes" if you get the SQL error "1104 The SELECT would examine more than MAX_JOIN_SIZE rows" on your site.

The screenshot on https://docs.joomla.org/Help4.x:Site_Global_Configuration_Server needs to be updated so it shows the new option.

[richard67]

Meanwhile I think that the fix provided here will not really be necessary anymore when PR #42576 will be merged.

Closing in favour of #42576 .