HTML Filter doesn't work on service worker requests

[gunir]

Prerequisites

• I verified that this is not a filter list issue. Report any issues with filter lists or broken website functionality in the uAssets issue tracker.
• This is not a support issue or a question. For support, questions, or help, visit /r/uBlockOrigin.
• I performed a cursory search of the issue tracker to avoid opening a duplicate issue.
• The issue is not present after disabling uBO in the browser.
• I checked the documentation to understand that the issue I am reporting is not normal behavior.

I tried to reproduce the issue when...

• uBO is the only extension.
• uBO uses default lists and settings.
• using a new, unmodified browser profile.

Description

HTML Filter is not working as expected, I used a fresh profile, fresh uBlock install, check my reproduce steps.

A specific URL where the issue occurs.

https://voz.vn/

Steps to Reproduce

• Add this voz.vn##^.p-body-sidebar to My Filter
• Go to https://voz.vn/
• Check the right sidebar
• F5 again
• The sidebar appear again even if you keep pressing F5:
  
• Now press Ctrl+F5, it disappears again:
  

I've recorded a video, please check it: https://streamable.com/ek1var

Expected behavior

The right sidebar should always get cleaned.

The right sidebar contains many iframes, that's why it's better to remove with HTML Filter ##^ than hiding with Element Hiding ##

I've recorded a video, please check it: https://streamable.com/ek1var

Actual behavior

The right sidebar only disappear if I Ctrl+F5, which is pointless because it's a hard refresh, but it shows up again if I use F5.

I've recorded a video, please check it: https://streamable.com/ek1var

uBO version

1.52.0

Browser name and version

Firefox ESR 115

Operating System and version

Windows 10

[gwarser]

Try:

||voz.vn$csp=worker-src 'none'

and unregister service worker from about:serviceworkers.

[gunir]

Try:

||voz.vn$csp=worker-src 'none'

and unregister service worker from about:serviceworkers.

Thank you, it worked!

Test video: https://streamable.com/o3ly9l

Should we have a workaround for this issue ?

Header Editor also has this HTML Filtering capability but it doesn't have this issue, it can still modify above webpage even with that serviceworker running.

[stephenhawk8054]

@gwarser I think we need this https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/webRequest/filterResponseData#permissions ?

From Firefox 95, to use this API to intercept requests related to the loading of service worker scripts, the "webRequestFilterResponse.serviceWorkerScript" permission is also required.

[uBlock-user]

@gwarser I think we need this https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/webRequest/filterResponseData#permissions ?

From Firefox 95, to use this API to intercept requests related to the loading of service worker scripts, the "webRequestFilterResponse.serviceWorkerScript" permission is also required.

@gorhill

[gwarser]

@stephenhawk8054 uBO filters only main HTML document, does not touch any script or service worker.

[uBlock-user]

What stephenhawk is suggesting is an enhancement if we want to intercept and block web-workers without having to resort to "CSP" filters.

[gwarser]

Adding these permissions [to manifest] does not help. Something more need to be done to filter these requests.

[uBlock-user]

Permissions open the way for the enhancement, Obviously, specific code needs to be added to make it happen, hence an enhancement.

[gunir]

Adding these permissions [to manifest] does not help. Something more need to be done to filter these requests.

I have a working code for Header Editor addon, I don't know why but it DOES work without manually changing CSP "worker-src", this enhancement will make using/writting HTML Filter more bearable.

This is my code:

Name: VozClean
Modify response body
Regular Expression
^.*?voz.vn/$
Custom function: return val.replace(/<div class="p-body-sidebar">[\s\S]*?<\/footer>/, '<!-- CLEANED -->');

Screenshot:

[gunir]

Should we mark this as enhancement and reopen ? It's no longer an issue but improvement is nice.

[krystian3w]

A little looks as AdGuard replace, then filter will works only after publish new add-on (if all others ideas fails and uBo add support).

https://adguard.com/kb/general/ad-filtering/create-own-filters/#replace-modifier

I checked on the latest free AdGuard add-on 4.2: it can handle it for Firefox:

||voz.vn^$replace=/<div class="p-body-sidebar">[\s\S]*?<\/footer>/<!-- CLEANED -->/

Unfortunately, AdGuard and their HTML Filtering cannot remove the element without damaging the entire DOM of the website.

---

I would suggest checking if the add-on "Header Editor" is actually working with action "VozClean" before browser parse HTML.

[gunir]

Unfortunately, AdGuard and their HTML Filtering cannot remove the element without damaging the entire DOM of the website.

This is very bad imo, but they will fix for sure, they're professional after all.

But well, talking about uBO's HTML Filtering, I don't mind removing worker src manually but it's such a hassle for end-users, HTML Filtering is very beneficial, its actually improve page loading speed by skipping network connections of removed contents entirely.

In this tested, you can see removing vs hiding that sidebar saved a lot of bandwidth:
Removing (116 total requests):

Hiding (141 total requests):

HTML Filtering is a bit advanced, but I believe it worth our cares to enhance it further to make it more user-friendly.

My 2 cents.

[gunir]

I would suggest checking if the add-on "Header Editor" is actually working with action "VozClean" before browser parse HTML.

It does, if you want proof, take this filter: #2833 (comment)

Look at this result pic and check in view-source:

Source: https://voz-vn.translate.goog/t/tong-hop-nhung-addon-chat-cho-firefox-pc-mobile.682181/page-315?_x_tr_sl=vi&_x_tr_tl=en&_x_tr_hl=en&_x_tr_pto=wapp#post-27877026

[krystian3w]

It looks uBo replace gorhill/uBlock@7c3e060c01 have same problem. In AdGuard replace worked fine 2 months ago without kill service worker.

[gorhill]

I think this is fixable by giving precedence to content-type when evaluating whether to html-filter a response.

[gunir]

I think this is fixable by giving precedence to content-type when evaluating whether to html-filter a response.

Beautiful and thank you, the recent replace filter loves this fix!