[4.0] Defer all scripts by default, second try. So much speed, wow!

[Fedik]

Summary of Changes

The patch make all JavaScripts deferred by default, even inline scripts (which now has a type "module" by default).

This is another try (first was #22460 by @laoneo)
Since we dropped IE support the things become much easier.

And when we said "module" then should come @dgrammatiko and say "module!" 😉

Testing Instructions

Apply patch, and make sure all works as before.
Enable debug mode and check all still works as before.

Please test in all (supported) browser you have.

Documentation Changes Required

I think it may need to add a note to https://docs.joomla.org/Potential_backward_compatibility_issues_in_Joomla_4
This introduce a little B.C. changes for people who have used inline <script> or <script type="text/javascript">.
And can be easily fixed by changing to <script type="module">.

Example old code: <script>jQuery(document).ready(function(){ .... })</script>
The new version: <script type="module">jQuery(document).ready(function(){ .... })</script>

However there no B.C. for people who have used $doc->addScriptDeclaration($code);

[dgrammatiko]

@Fedik you do realize that this debugbar is not compatible with the csp (eg the rest of the assets)?
I will suggest that the document nonce (whatever the function name ) becomes a trait to the HTMLDocument so it can be used here as well...

[Fedik]

it another issue for another PR, I just made it work, I not tried to fix everything here 😉

[dgrammatiko]

Of course, I’m just mentioning so maybe someone can eventually fix it

[mbabker]

Removed by @mbabker

[mbabker]

Removed by @mbabker

[Fedik]

And the impact this causes to end users who paste script snippets into WYSIWYG editors is?

As I wrote, use <script type="module"> instead of <script>.
just another copy/paste, additionally to existing snippets what can be easier? 😉

How many people will have issues pasting script snippets into their sites and waste time debugging because they don't understand the need for this attribute?

Well, if it will be really a big problem (that I doubt), then it will be trending in Google.
I meant an easy searchable solution.

By forcing this behavior out-of-the-box, you are continuing the current trend of making Joomla 4 more difficult to use for non-developers.

Allowing paste <script> in to WYSIWYG also not an easy task, from this point of view,
Who have enough strength to do it also will be able to change <script> to <script type="module">.
I think :)

[brianteeman]

Have to agree with Michael. I didn't even know what type=module was. As to the prospect of checking 4000 articles to see if they have inline scripts. Yuk. No thanks.

[mbabker]

Removed by @mbabker

[dgrammatiko]

type=module is the current de facto for js, people will have to educate/adapt/upgrade/update (use whatever verb you like here) themselves. We're not in 2008 anymore, things have changed
My 2c

[Fedik]

As to the prospect of checking 4000 articles to see if they have inline scripts.

If all that your script have is google analytics or other analytics, or social stuf etc, then you can sleep well 😉

... not seen one library, tutorial, or StackOverflow question in the last 5 years that points to this type="module" stuff

Me to. Time to start learning, to fill that gap.

Before pushing your personal preferences on hundreds of thousands of websites, please take a minute to understand the practical impact of that decision on people of all skill levels

I made an offer, no more no less 😉
From all possibilities it is a best that I can offer. It allow to have more or less optimized site out of the box.
And again, use type="module" not required, but potential changes.

Also it easy to revert. We can wait a couple Alpha releases, and look to a feedbacks, if it really that bad, then just revert.

[brianteeman]

Alpha releases are almost never tested with live site data. In fact hardly anyone tests them at all.

[Fedik]

Alpha releases are almost never tested with live site data

I bet there a people who use Alpha in live, there always someone 😄
Well, it easy to revert at any time.

[mbabker]

Removed by @mbabker

[joeforjoomla]

I totally agree with @mbabker and @brianteeman
If someone wants to speed up its website, he's free to install a third-party plugin such as JCH Optimize.

But the core can't require a technical people to be managed. I've developed dozens of extensions since 2005 and i'm one that still ignore what type="module' is.

Joomla 4 should be the reborn of Joomla, if we still continue to make errors and make it more and more complex then we should not cry if the usage percentage of Wordpress increases and the Joomla one decreases.

[brianteeman]

There is a massive difference between supporting a new feature and requiring it.
There is a massive difference between carefully adopting new features without going all in and converting everything - even when it clearly isnt ready yet and/or there is no one involved with the skill/time/commitment/knowledge to support it

[dgrammatiko]

@mbabker I think you're wrong here, let me explain:
The CMS has a definitive way to add inline scripts. If the CMS was carrying for its users all those years would have introduced a field, plugin whatever that extends the article form where a bozo user could paste some js or css, because both fall in the same category. Now that never happened and AFAIK never even proposed. So we come to this point where the CMS has accumulated 15 years of bad practices and some people just proposing to fix those: the response is "you're breaking it for the naive users". Sure but then again why the freak didn't the CMS fix those huge missuses years before.
Anyways, as @Fedik mentioned this PR is: take it or leave it.
FWIW the problem you're so much concerned for is very easily fixable so your view her is pessimistic for no real reson...

[mbabker]

Removed by @mbabker

[dgrammatiko]

@mbabker dude it's not about pure or whatever. It's the fact that Joomla is slower than the slowest sloth, I have posted the web archives result for Joomla and it's on par with WIX (the worst thing someone can publish on the web). The reason for this poor performance is that the CMS is unopinionated on the way that JS and CSS are delivered. This PR improves things by a huge factor and it comes with the penalty that nobody did anything on that aspect for 15 years.

BTW I'm not trying to force anything frankly because I don't do PRs anymore so how the heck an observer is trying to force anything?

even when it clearly isnt ready yet and/or there is no one involved with the skill/time/commitment/knowledge to support it

This comes from ECMA262. You know the 1/3rd of what makes the web (the other 2 are HTML and CSS in case you're not aware)

[brianteeman]

And quite frankly I'm sick and tired of developers crapping all over the work of other developers or non-technical users because they deliver things that are not technically pure but actually get the job done.

we're used to it :( and the patronising comments of others

[mbabker]

Removed by @mbabker

[dgrammatiko]

Joomla supports the HTML standard and the API should have the flexibility

But what you're complaining here is for people that DIDN'T use the API, eg inline script not through the CMS' API

I'm confused...

[mbabker]

Removed by @mbabker

[dgrammatiko]

I'm pointing out that people do it because they don't have the technical skill to write the PHP code required to use the API and pasting stuff in a WYSIWYG is exponentially easier.

That was OUR (devs/contributors) problem that we never agreed to provide a realistic solution all these years. I'm not blaming the users don't get me wrong

Others are just pasting script snippets inline in articles, custom HTML modules, template configurations, or directly in a layout or template file without use of the API. For those people, this default use of defer causes problems.

Ok, let's make a plugin that covers this scenario. I mean if this is all it takes I'll provide that missing piece...

claim Joomla's too difficult to use, and go to Wix or WordPress.

Glad you've said that. A few years ago when I've started that grunt thing to integrate 3rdpd assets (js, fonts, css, you know those things) a lot of people were scared that the codebase will be alienated and devs will never come around those changes. Fast forward 3-4 years and WP is also doing the same but then again Joomla moved already one step forward (no grunt or gulp). I will admit that many things proposed/implemented in J4 are a brand new world for many people but I'll stay on my point of view: "the codebase needs to last as long as possible introducing the least b/c breaks"

[mbabker]

Removed by @mbabker

[dgrammatiko]

That end users aren't faced with an over-engineered interface to complete a "simple" task.

Users will continue pasting their inline scripts as before, I'm not suggesting adding fields for these tasks, although with custom fields this is extremely easy to do, 1 field could handle both css and js.
The plugin that I'm suggesting (disabled by default) will parse the component and modules of each page, extract any inline css and js and introduce them through the API. Nothing fancy here.

It doesn't play nice with your ideal scenario of there being no random inline scripts in a page so you can defer or lazyload resources (and it's the same problem with defaulting scripts to the end of the body)

There is a huge difference between deferred and lazy loaded scripts. Obviously, I prefer the second but realistically such a change will be too much for the majority of the users and also the devs. Unfortunately, Joomla's devs are poorly educated on JS and CSS (and probably HTML markup). Bootstrap contributed hugely to this...

[mbabker]

Removed by @mbabker

[Bakual]

I also agree with Brian, Michael and Joe.
The CMS by default has to be as robust as possible., even if that means it is not "best practice".
But it has to allow those best practice as well on a template level.

So by default scripts are loaded into the header, but the template can decide to load it whever it wants (best practice would be by using a template parameter). So the site admin can decide if it will work when the scripts are deferred or in the body or wherever.
The CMS itself will never know that and can't make that decision.

[dgrammatiko]

The CMS itself will never know that and can't make that decision.

Sure, because Joomla is so unopinionated in let's say the router, the db, the MVC, the plugins, etc even in the frontend as it's stack to the monolith era of Bootstrap, so why the heck the delivery of the assets should be opinionated?

Am I reading this right @Bakual ?

Also, you're missing the forest for that old tree: such changes will have a huge impact on performance and therefore the UX. And you know better UX means better chances for people to sell their work on Joomla...

[mbabker]

Go use a CMS that is as opinionated as you are as it relates to frontend delivery if you don't like how Joomla works. Joomla doesn't aim to be a "this is the only way to do it and if you don't do it this way you're a bloody idiot" platform.

Anywho, I'm done here, I can't debate with someone who calls people handicapped because they use tools that someone else 💩 all over.

[brianteeman]

@dgrammatiko what respect I had for you has just been flushed down the toilet by your comment about the handicapped. Goodbye

[infograf768]

Joomla doesn't aim to be a "this is the only way to do it and if you don't do it this way you're a bloody idiot" platform.

Even though I think this statement is false (see among others the forced use of bootstrap which consequence was that a lot of users had to forget about making their own simple templates when not specialists), I do agree that such a new feature should at least include a new plugin to cover our a...s.

As for comments stating anyone is "patronizing" someone else ( #25365 (comment) ), I think they should look into their own past statements of the style: "We are no more in 2005..."

@Fedik Can you add such a plugin to this PR?

[Fedik]

Any influence of the patch to Joe Average it just an assumptions.
Only way to confirm or to refute is to test it (I already have suggested a way).

I did not offer anything exotic, like an example Magento 2 have <script type="magento-init">.
All in the border of a specifications.

I do not think that the plugin really need. The issue may happen or may not happen.
I meant the regular <script> not forbidden, and it still will work. The problem may happen if that script have a code that depend from another script from a header. I do not think that really that often case, but again it an assumption.
Also the plugin involve editing an user content, that not very good idea.

I think in general it is a good feature.
The "script blocking" one of the first issue that User hit when tries to fix Speed Test, to improve a SEO.

I leave the pull request opened, j4 maintainers has to decide, is it worth to have it or no.
For me no big difference.
Later I will move DebugBar to separated pull, as it have some more issue that need to be fixed.

[mbabker]

The problem may happen if that script have a code that depend from another script from a header. I do not think that really that often case, but again it an assumption.

Quit assuming things then. Enabling this behavior by default means people have to bend over backwards to paste script snippets dependent on jQuery or Bootstrap into their WYSIWYG editors, and if that doesn't work they will either claim Joomla is broken or worsen their site performance by loading a second version of those libraries in their WYSIWYG by pasting a CDN URL (without defer) in.

It is easier for an administrator of a website to enable these types of optimizations on a case-by-case basis than it is for someone to reverse over-eager optimizations because someone does not understand how end users use the freakin' software. Stop staring at the Google PageSpeed and Lighthouse outputs and trying to find ways to make Joomla machine friendly at the expense of being user friendly.

Only way to confirm or to refute is to test it (I already have suggested a way).

The ONLY way it is acceptable to put an "experimental" patch into a release package for end user testing is if it is behind a feature flag. Otherwise, something that is not going to be included in a release package should NOT land in the code repository, period.

[Bakual]

What would be wrong with adding this feature as an opt-in parameter to the template? Disabled by default.
This way it would be a good example for other templates devs how this can be achieved. And easy for the user to switch on and off depending on their needs, it could even be toggled for some pages by using different template styles.

[mbabker]

What would be wrong with adding this feature as an opt-in parameter to the template? Disabled by default.

It's still an all or nothing approach. Realistically, you need to be able to manage all of the assets independently to decide what can be deferred, what can be lazy loaded using some other optimization trick, and what must be loaded immediately in page load (i.e. you might need jQuery to immediately load to support some snippet inserted by users, but the script we use on the joomla.org homepage for all the countups or the ad block notification could be deferred or lazily loaded after the initial page load (and yes, the countup is one of those cases where a JavaScript <script> tag is inlined in a custom HTML module because there's really no point in going back and writing an entirely custom module for that)).

Since there is no central asset registry in Joomla that is aware of every possible asset (and no, the web asset API doesn't count because everything that isn't core is still lazy loaded), trying to build an interface with this level of control isn't the most sane thing to try and attempt, not to mention it would really be overwhelming for the end user and complex for the developer to design properly (because realistically you'd need to raise a flag if you try to defer something that is depended on by something else which isn't deferred)

[joeforjoomla]

Realistically, it should be up to developers to know when and what to load in defer mode. I use defer mode for my scripts since years, but thinking that everything within Joomla should be opt-in to be loaded in defer mode unconditionally sounds really insane.

I would rather try to document that Joomla supports defer mode for scripts loading so that developers know that they can do this with their scripts when possible.

[Bakual]

It's still an all or nothing approach.

Yep, but one that the (educated) user may be able to decide. And if it breaks, it's easy enough to revert.
I could see that being a nice feature on a template level. But certainly not as a default and not as a site-wide setting like proposed in this PR. Templatesettings can be different based on menuitems, so it's quite a mighty feature.

[Fedik]

The template option can be a bit tricky to implement.

well, nevermind, I have a bit another idea