Separate third party inline scripts

[PaperStrike]

PR Checklist

• The commit message follows guidelines for NexT.
• Tests for the changes was made (for bug fixes / features).
  • analytics
    • baidu-analytics
    • google-analytics
    • growingio
  • chat
    • chatra
  • comments
    • changyan
    • disqus
    • disqusjs
    • gitalk
    • isso
    • livere
    • utterances
  • math
    • katex
    • mathjax
  • statistics
    • firestore
    • lean-analytics
  • tags
    • mermaid
    • pdf
  • nprogress
  • quicklink
  • rating

The main problem is there are so many third party scrips to test.

PR Type

• Refactoring (no functional changes, no api changes).

What is the current behavior?

Continue working on #226! Another pull request for more dedicated tests!

Issue resolved: #220

What is the new behavior?

• Link to demo site with this changes: sliphua.buggy.work

Links below, if not specific, redirect to the related code line(s) / file.

• Third party inline scripts moved to source/js/third-party/[*/]*.js.
• Third party configurations can be accessed by CONFIG.[third-party-name].* and are updated on the first evaluation after a page load.
• Export configurations to front end by using next_data. This new helper uses <script type="application/json"> to store data, which is documented in MDN, discussed in Stack Overflow, and doesn't violate any CSP rule.
• (Current Status: Reverted for readability) Replace IIFE by ES6 block statements.
• (Current Status: Replaced) Use NexT.utils.getScriptPromise to load scrips instead of inserting them to DOM directly without callback control. And it potentially replaces NexT.utils.getScript in all use cases.
• (Current Status: Replaced) New NexT.utils.loadCommentsPromise potentially replaces NexT.utils.loadComments.
• (Current Status: All Adjusted) Scripts injected to bodyEnd, along with math scripts and quicklink, will not be reloaded by Pjax anymore. Corresponding third party code have been edited for this change in their own files. Corresponding Pjax code, line 19 in source/js/pjax.js, will be edited in future commits.

IMPORTANT & Requires Discussions:

• (Current Status: Put Off, samples reverted) Whether or not it is better to move nunjucks code in scripts/filters/comment/*.js to corresponding position inside layout/_third-party/comments/*.njk. Potentially, scripts in scripts/filters/comment/*.js can be combined into one if this is done. Besides, to achieve this, a layout/_third-party/comments/common.njk has been created to replace scripts/filters/comment/common.js.
  Not finished yet, possible samples have been made in
  • layout/_third-party/comments/disqus.njk with scripts/filters/comment/disqus.js,
  • layout/_third-party/comments/changyan.njk with scripts/filters/comment/changyan.js.
• (Current Status: minify.js Updated) Should we keep using minify.js to clean third party scripts? Or add third party scripts to route only when needed? If the latter is better, how should we achieve it?
• (Current Status: See Separate third party inline scripts #241 (comment) for detail) In quicklink.js, quicklink.ignores is used and may include functions by design. But functions can not be parsed from strings safely in JS runtime. Allow using eval or Function here, disallow functions in ignores, or ...?

[PaperStrike]

Google Analytics and WidgetPack Rating on Pjax sites seem to have been broken ever since Pjax is introduced, long before this PR.

• For Google Analytics, 015dd00 tries to fix the problem by firing page_view event in pjax:success events, which is officially documented and recognized as History Change event in Google's Tag Assistant. So I consider it suitable for our Pjax sites.
• For Widget Pack, e0df44d switches to call WPac.init in pjax:success events, using Using on AJAX sites | Star Rating Widget API | Widget Pack as a reference.

An unexpected thing is that Widget Pack's official script uses Function constructor, passes script code in URL(s), and injects inline stylesheets, which leads to CSP rules: unsafe-eval and unsafe-inline in script-src, and unsafe-inline in style-src.
I can't solve this.

[njzjz]

Or add third party scripts to route only when needed? If the latter is better, how should we achieve it?

I have an idea:

hexo.extend.filter.register('after_generate', () => {
  // minify js files
  hexo.route.list().filter(path => (all_js.includes(path) && !used_js.includes(path))).forEach(path => {
    hexo.route.remove(path);
  });
});

used_js can be added in next_js.

[PaperStrike]

used_js can be added in next_js.

I tried before but failed. It seems like next_js runs at the render stage, after the generate stage, which means used_js will not be ready at all at that time. And hexo.route.add mysteriously not functioning when called in next_js. 🤔

I might be wrong anyway.

[PaperStrike]

Status Update: Function and Regex in config file's quicklink.ignores will be deprecated once merged.

Theme config file has been updated:

hexo-theme-next/_config.yml

Lines 554 to 557 in 7a5b348

	# For more flexibility you can add some patterns (RegExp, Function, or Array) to ignores.
	# See: https://github.com/GoogleChromeLabs/quicklink#custom-ignore-patterns
	# This option is deprecated. Use `CONFIG.quicklink.ignores` in your custom scripts instead.
	ignores:

If used, error message will be shown as:

hexo-theme-next/source/js/third-party/quicklink.js

Lines 15 to 16 in 7a5b348

	// eslint-disable-next-line no-console
	console.error('Setting regex and function in config file is deprecated. Try setting `CONFIG.quicklink.ignores` in any script code.');

Users using this config should turn to configure CONFIG.quicklink.ignores in any custom script code / file.

[PaperStrike]

The rest comment systems are hard for me to test, I suggest merging now and getting some feedback.

[PaperStrike]

To NexT plugin authors,

Sorry that I forgot the plugins which are outside this repo and use NexT.utils.getScript, NexT.utils.loadComments, or bodyEnd inject point. I merged this after only a simple search.

As you may have seen in the PR description,

• NexT.utils.getScript and NexT.utils.loadComments are refactored.
• bodyEnd will not be refreshed on Pjax reloads anymore.

NexT.utils.getScript and NexT.utils.loadComments now returns a Promise, so your callback functions can be associated by using its then method. Like:

// Before:
NexT.utils.getScript('example.js', onLoad);

// After:
NexT.utils.getScript('example.js').then(onLoad);
NexT.utils.getScript('example.js').then(onLoad, onError); // better

The condition parameter of NexT.utils.getScript is repositioned and the function now accepts more options. If you need to have your <script> more customized, try it now:

// Before:
NexT.utils.getScript('example.js', onLoad, window.example);

// After:
NexT.utils.getScript('example.js', { condition: window.example }).then(onLoad, onError);

// Before:
const script = document.createElement('script');
script.async = true;
script.dataset.foo = 'bar';
script.onload = onLoad;
script.url = 'example.js';
document.getElementById('example').appendChild(script);

// After:
NexT.utils.getScript('example.js', {
  attributes: {
    async: true,
    dataset: { foo: 'bar' }
  },
  parentNode: document.getElementById('example')
}).then(onLoad, onError);

To have more precise and elegant control, bodyEnd, along with scripts inside, will not be reloaded by Pjax now. To keep your plugin Pjax-compatible,

• make sure the elements in your plugin are consistent in every page,

and when needed,

• try using the data-pjax attribute to mark a whole <script> to be reloaded automatically
• or listen to pjax:success (fired when a Pjax request succeeds) or the new page:loaded (fired when DOMContentLoaded or a Pjax request succeeds) events to reload some functions. (always preferred)

{# Before: #}
{%- if theme.example.enable %}
  <script src="example.js"></script>
  {%- if is_home() %}
    <link rel="stylesheet" href="example.css">
    <script src="example.home.js"></script>
    <script>
      window.example.home.load({{ theme.example.id }});
    </script>
  {%- endif %}
{%- endif %}


{# After: #}
{%- if theme.example.enable %}
  <link rel="stylesheet" href="example.css">
  <script{{ pjax }} src="example.js"></script>
  {{ next_data('example', theme.example) }}
  <script>
    document.addEventListener('page:loaded', () => {
      if (!CONFIG.page.isHome) return;
      NexT.utils.getScript('example.home.js', { condition: window.example.home })
        .then(() => {
          window.example.home.load(CONFIG.example.id);
        });
    });
  </script>
{%- endif %}

The new next_data helper is used to export data to front-end scripts. The first parameter is the name of the exported data, the data will then be accessible as CONFIG[name] in the front-end. The rest parameter(s), if more than one, will be combined into one single Object, and then be exported as the data.

Besides, as the initial focus of this PR, it's recommended to seperate the inline scripts into single file(s).

If you want to keep supporting older versions of NexT, see #259 (comment), or use your previous code directly when next_data is not defined.

{%- if next_data %}
  {# For NexT>=8.4.0 #}
{%- else %}
  {# For NexT<=8.3.0 #}
{%- endif %}

Kind regards,
PaperStrike