[4.0] Defer load Javascript files as default

[laoneo]

Summary of Changes

As speed matters, Javascript files should be loaded as default in defer mode. Most scripts do wait for the DOM to be rendered anyway so there is no point to block the head and tell the browser to parse the JS file just to wait for the DOM to be loaded fully. Better to do the two tasks in parallel.

More documentation can be found on MDN, but for them who are to lazy to open the link, here is the defer attribute doc text:

This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed, but before firing DOMContentLoaded.
Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.
To achieve a similar effect for dynamically inserted scripts use async=false instead. Scripts with the defer attribute will execute in the order in which they appear in the document.

As jQuery acts often for browser abstraction, it has to be loaded in sync mode otherwise it will be a hard BC break as inline jQuery code will not work anymore then. This is often the case on old extensions.

Testing Instructions

Browse around in Joomla.

Expected result

All is working.

Actual result

All is working, just a bit slower as it should in 2018.

Documentation Changes Required

Needs to be documented.

[Fedik]

That a good thing, except 2 thing 😉

You should not care about jquery specific code, but care about inline code.
So you can force defer for every script,
but if (!empty($document->_script['text/javascript'])) then you should not force defer

[laoneo]

Because of BC I think it would be not too bad to have jQuery in sync mode.
Inline scripts is not an argument to remove defer. You can have inline scripts and defer mode loaded scripts and vice versa. The target is to remove the inline scripts anyway for better CSP compatibility (yes I know you can add the nonce).

[brianteeman]

This has the potential to cause a lot of problems with extensions

[laoneo]

In 2018, we have to do it. But with jQuery still in sync we can reduce the damage to an acceptable minimum.

[brianteeman]

We don't have to do anything

[laoneo]

That's one of the things we have to, to stay competitive. But thanks for your input, I'v added the "Documentation required" flag already to indicate this needs to be documented in the migration guide.

[mbabker]

but if (!empty($document->_script['text/javascript'])) then you should not force defer

It's not about just that. You also have to look for inline content entered by users in WYSIWYG editors and other inline spots. You can't just break the platform to "force" an arbitrary standard.

[Fedik]

Because of BC I think it would be not too bad to have jQuery in sync mode.

No, it does not make much sense to care about only jQuery. You can defer jquery as any other script, until there not exist inline <script>.

I can have another non jQuery script foobar.js and add inline code that depend from this script,
you defer foobar.js and in result my inline code crashes.
So you see, it does not matter, which scrip sync/or async. We should force defer only if there no inline script.

I wanted to make similar PR. But you already did 😉

Inline scripts is not an argument to remove defer.

it huge argument to remove defer, because it only reason why we cannot use defer

[Fedik]

You also have to look for inline content entered by users in WYSIWYG editors and other inline spots.

I not really care about <script> in editor 😄
mostly it some GA trap, which will work fine

the same about inline code in the layouts

[laoneo]

because it only reason why we cannot use defer

I do not get that. If the inline script wants to do something on the DOM, then it must wait anyway till it is loaded. Can you post here a use case which supports your argument? As @mbabker already said, there are cases where you have inline code added not through the API and these will not being caught by your logic.

[Fedik]

If the inline script wants to do something on the DOM, then it must wait anyway till it is loaded

it not always true, you not always have to wait,
the script can do some random calculation, myLib.doStuff();.

hm hm, well, maybe you are right, it also not very important, people who do such things can move their code in to the file, or wait on window load

where you have inline code added not through the API

they should add the code through the API, or as you said, wait everything loaded

this will fail:
<script>callFooBarScript();</script>
this will work:
<script>window.addEventListener('load', function(){callFooBarScript();})</script>

[laoneo]

Perhaps "must" was the wrong word, better would be should wait till the DOM is loaded.

[Fedik]

okay, I think you convinced me,
I have read everything again, and I think the PR is good,
if need b/c for jQuery then it can stay in sync mode

[laoneo]

I still see a lot of the following code when dealing with customers web sites:

• jQuery( document ).ready(function() {//do stuff});
• <a href="#" onclick="jQuery(this).datepicker('save');jQuery('form').submit();">Text</a>

So it would be wise to load jQuery in sync.

[Fedik]

yeah, this will fail jQuery( document ).ready(function() {//do stuff});

but this will continue to work, because it require jquery only on click event:
<a href="#" onclick="jQuery(this).datepicker('save');jQuery('form').submit();">Text</a>

[laoneo]

Yes, you are right, doing actually too many things together.

[dgrammatiko]

@laoneo you example

<a href="#" onclick="jQuery(this).datepicker('save');jQuery('form').submit();">Text</a>
is totally fine even if jQuery is not defined at all (until you press that button, eg invoke the onclick event).
https://codepen.io/dgrammatiko/pen/BqjaaR

In other words this is not the problem. The real problem are inline scripts and as @Fedik mentioned if you defer something and you also have inline script that depends on the deferred script then you effectively cancel the defer (at least this is what happens in FF, Chrome and Safari, for Edge and IE I have no clue how that will turn out :D).

For what is worth according to bundlephobia jQuery costs 0.6 seconds ( https://bundlephobia.com/result?p=jquery@3.3.1 ) for transferring the code (add parsing and executing and on a median 3g phone you get over 1 second) Add Bootstrap (this is only the js: https://bundlephobia.com/result?p=bootstrap@4.1.3 ) and then you have at least another second (with the css). Mobile is the main consumer (over 65%) of the web pages. Just pick your dependencies carefully or the first time visitors will always be unhappy when they navigate to a Joomla 4 created page.

FWIW you need to disallow inline scripts in the body tag and then you're golden! Simple solution

[laoneo]

During the lifecycle of Joomla 4 we have to live with the fact that inline scripts will exist. If the mindset has changed till Joomla 5 will be out is another story which nobody can predict right now. For me is important that we work correctly out of the box. With the move to bootstrap 4 (or agnostic) an extension dev has to redo the view files anyway which includes the JS files as well.

[dgrammatiko]

Well, yes and no, depends on how you evaluate things right now. My personal thoughts on the js for 4, as I already had expressed them to George is that the product should be delivering ES6 files to the browsers as the default and have a fallback for ES5. Right now J4 is delivering the lowest denominator (ES5) to all browsers (apart the custom elements which are correctly deliver whatever the browser supports. Now why is such a thing important, you may ask. Here are some points worth mentioning:

• IE11 will reach EOL in 2020 (although the official statement is stating 2025 but that won't happen)
• the lowest denominator strategy although it worked fine (and still does) comes with the cost on performance and since Joomla users will end up using the product to compete for a better ranking on the search engines, this is quite important and affect majorly the ROI of the produced sites!
• if you manage to release J4 in 2019 then your es5 code will be fine for 1 year (or less) and then for the rest of the life of J4 will be outdated
• if you just try to keep existing devs/users happy without expanding the user/dev base your plans for J5 might be totally irrelevant, you need to attract and expand the user base and the dev's coding against the product.

For me is important that we work correctly out of the box.

Well Joomla will work "correctly" out of the box if you declare that script tags are not allowed in the body tag of any component/module/plugin layout (by the way it's a huge anti pattern to use script tags randomly in the body tag). IE check https://frontendchecklist.io and try to tick what Joomla is doing "correctly" out of the box. So I think the whole point is not about correctness but more about enforcing best practices.

Anyways my 2c

[laoneo]

The core can't and should not control what the end user or extension dev is doing in terms of JS output. The core should out of the box follow best practices within our requirements (which are including IE11). If I'm not wrong, with the CSP component we have now a tool to even report the end user about the inline script tags as they are not compliant without the nonce (which I doubt most of the inline code doesn't have). All we can do is to lead and recommend, the result is not in our hands.

[mbabker]

Well Joomla will work "correctly" out of the box if you declare that script tags are not allowed in the body tag of any component/module/plugin layout (by the way it's a huge anti pattern to use script tags randomly in the body tag).

And what are you going to tell those users who copy/paste HTML and JavaScript snippets into a mod_custom instance, learn how to write a module to "properly" register a script? Make Sourcerer a de facto requirement to run a Joomla site so arbitrary PHP can be run anywhere? You are dealing with a platform where the only way to register scripts is not through a PHP API to "properly" put them somewhere (properly being very subjective). We can do what we can to follow best practices for the core architecture, but how users input data through the interface must continue to be considered, unless you are seriously saying we hardcode a rule somewhere that arbitrarily rejects/strips a <script> tag from any user input source (which I'm sure most on JSST would actually be OK with for one reason or another) because it lets them "use script tags randomly in the body tag".

[Fedik]

@laoneo please check the fix Digital-Peak#13

I have tested it and it seems looks good.
But it not possible to check everything so I think it would be good if we get it merged earlier,
to see all possible issues.

[Fedik]

And what are you going to tell those users who copy/paste HTML and JavaScript snippets into a mod_custom instance, learn how to write a module to "properly" register a script?

I just thought, maybe we should not care much about it.
If they get a problem they always can use:

document.addEventListener('DOMContentLoaded' function(){ ...my fancy code here...})

That should become popular on the forum, if it will be really an issue. In theory.

[mbabker]

Maybe for most of us as developers we don't need to care about it because we're the types of people that would rather others be forced to do things the way we want the software to work and work with our workflows. But the reality of it is that end users, who aren't developers, integrators, or any other "technical" oriented thinkers, are going to see finding how to wrap their JavaScript snippets that they just copy/paste from somewhere on the internet into their articles or modules or any <textarea> (including those decorated by a WYSIWYG editor) into a DOMContentLoaded event listener (and not even knowing they have to do that until they start trying to figure out why their scripts are broken) are just going to see that as a barrier for use.

So yes, it is something that we need to be aware of and how it impacts how the end user interacts with the platform. I'm not saying the change is wrong from a technical perspective, I am saying the change needs to be thought about as more than "is the code correct?".

[laoneo]

An alternative can be to deprecate the defer flag in 4. As I just did in #23116.

[Fedik]

it does not required, because css-vars-ponyfill.min.js has defer:false 😉
or, then need to remove defer:false for css-vars-ponyfill.min.js (look in settings.json)

[laoneo]

If I do check the source code, then it doesn't have the defer flag set at all.

[Fedik]

have you run npm install ?
or node build.js --copy-assets to be quick