[4.0] Error in Joomla update. Invalid Ajax data. Fix #21160

[ReLater]

Pull Request for Issue #21160

This file was provided by @mbabker (https://gist.github.com/mbabker/5e171b1df99f71a30290125e0292b1d9).

I only created this pr.

Summary of Changes

mbabker:

We're going to have to get tricky with this file. The intent is to overload the core classes, so as they're now namespaced, the overloading has to use the correct namespacing as well. I think this is right (there are no IDE reported errors and no reported syntax errors linting the file), but I never write files using multiple namespaces, so I have no idea if this is going to do what we want it to.

Testing Instructions

I've tested it like this. The dilettantish way:

• See [4.0] AJAX error with com_joomlaupdate Upload&Install #21211 and [4.0] Error in Joomla update. Invalid Ajax data. #21160

• Install a nightly build 4.0 or alpha.

• Download the nightly full package and also the update package 4.0.

• Go to Joomla Update > Tabulator Upload&Update.

• Try to update with one of the downloaded packages.
  You'll get an AJAX Error.

• Extract the downloaded packages and exchange administrator/components/com_joomlaupdate/restore_finalisation.php with the file of this pr.

• Zip the packages.

• Repeat the steps above.

• No AJAX error. Update successful.

[ghost]

At "Upload&Update" got:

But i guess its about zipping as there are different Sizes at modified (17,9MB) and original (13,3MB) zip:

System information

• Nightly Build 4.0.0-alpha5-dev
• Multilingual Sample Data (French, German DE, Persian)
• Template: Cassiopeia
• macOS Sierra, 10.13.6
• Firefox 61 (64-bit)

CloudAccess.net

• PHP 7.1.15
• MySQLi 5.7.18-cll-lve

[laoneo]

Can this not being removed from core, @nikosdion?

[nikosdion]

I am not sure what you are asking.

Removing the constant completely would be a massive security issue. This script must only be allowed to run under very specific circumstances. Removing this check would allow anyone to run it anytime, causing damage to the site.

Renaming the constant is possible, with the caveat that it's not a rename, it's an addition to Akeeba Restore (restore.php). As you can see, this constant is defined in the Akeeba Restore preamble and it's used by my backup software's integrated restoration to make sure post-restoration finalization won't be executable over the web for the same security reasons. We could add another line
define('_JOOMLA_UPDATE', 1);
and change the line in restore_finalisation.php to
defined('_JOOMLA_UPDATE', 1) or die();
Renaming it won't happen because it'd break my software.

I think that's the best solution. After all, restore.php is a third party library included with Joomla, not part of Joomla! itself. This is what the erstwhile Production Leadership Team had declared and that's the context of our conversation.

I could go one step further and make a special distribution of Akeeba Restore just for the Joomla! project. Please note that this would be a custom distribution, not code with shared OSM copyright, i.e. it will still not be part of Joomla! itself. The custom distribution could do away with all the features the Joomla! project does not need such as support for JPA and JPS archives and all post-processing engines except Hybrid (which magically figures out when to use direct file writes and FTP).

This latter option is extra work for me since it requires refactoring my code. I don't have time to do this right now but if you'd give me a rough estimate of a 4.0 beta freeze I could try. Since the custom distribution would be backwards compatible in the com_joomlaupdate context, even if I don't make it by the beta freeze we can still replace the regular restore.php with the custom one anytime in the 4.0 release cycle.

In fact, we could first add the extra define and then move to the custom distribution. Makes sense for you?

[laoneo]

Whatever works best. I would add the _JOOMLA_UPDATE joomla code and deprecate the _AKEEBA_RESTORATION check. I guess we can't remove it in the closer future as it will break the old Akeeba backup installations?

[nikosdion]

If you really don't want that constant in the file the only course of action is the second option, making a special distribution of restore.php for Joomla!. As I explained this will take a while.

In the meantime you can continue using the old constant. I would also recommend adding more comment text at the top of the file to explain when this file is used and what it does.

[brianteeman]

For my own education and only if you have time can you explain why removing the akeeba restoration constant will allow unwelcome access - we still have the JEXEC or die?

[wilsonge]

This file is run directly from restore.php as a cleanup step. As a result we're not in the context of a Joomla entry file but the restore.php context - hence we haven't actually defined JEXEC for this single file (you can see we define it on the line below)

[brianteeman]

thank you

[nikosdion]

Since other people will be reading this, there is a reason why we run this outside the Joomla! context.

When we are going between Joomla! release families (e.g. 3.8 to 3.9) we can reasonably expect that some files / folders are removed or moved around. However, extracting a ZIP file will only create files, not delete the ones which are removed.

This is not just a nuissance (dead code left behind) but a real functionality issue. Due to the way the class autoloader works, old code files might be prioritized over the new ones. This will create a disparity in what the rest of Joomla! expects to be loaded and what is actually loaded. This would lead to dead sites.

Moreover, sometimes we need to apply some database changes (either schema or data conversions) to ensure that the upgraded site will work correctly.

Finally, on sites which use a code cache (such as OPcache) we need to reset the code cache after Joomla! finishes updating. Otherwise PHP may end up loading a mix of old files from the cache and new files from disk, leading to broken sites. Unfortunately many servers either have a code cache enabled without asking the user or brand it as "site optimization" without sufficient explanation of the technical pitfalls every time you upgrade code on your site. This can be resolved programmatically in most cases.

These actions need to run after Joomla is upgraded but before you access it. Therefore we cannot run in the Joomla! CMS application context, nor can we (usually) do so safely in the context of a custom web application based on the Joomla! framework because of the deleted / moved files issue. The only way to handle it is by doing this outside of Joomla!, selectively loading the bits and pieces which are guaranteed to be working. Hence a separate script. The need to load bits of Joomla! is why we also define _JEXEC in that script; to allow those Joomla bits to load without dying.

The question is, how do you load this script over the web without accessing Joomla! and without posing a security threat (i.e. you can't have a directly web accessible file). The solution is given by Akeeba Restore itself. The last step it does after the extraction is over is calling the restore_finalisation.php script. Since communication with Akeeba Restore is secured and limited to the update process it does not pose a security risk.

The way the finalisation script is protected against direct access is similar to what Joomla! does: by checking whether a constant is defined. This thread is about renaming that constant.

[brianteeman]

Thank you