Filterlist updating is slow

[troysjanda]

Prerequisites

• I verified that this is not a filter issue (MUST be reported at filter issue tracker)
• This is not a support issue or a question
• I performed a cursory search of the issue tracker to avoid opening a duplicate issue
• The issue is not present after wholly disabling uBlock Origin ("uBO") in the browser
• I checked the documentation to understand that the issue I report is not a normal behavior

I tried to reproduce the issue when...

• uBO is the only extension
• uBO with default lists/settings
• using a new, unmodified browser profile

Description

when in the Ubo settings panel on the filters list tab and selecting purge and update, the process slows greatly after the builtin section. Not really sure when this started but seems dramatically slower.

• uBlock Origin: 1.44.5b0
• Firefox: 105
• filterset (summary):
• network: 184643
• cosmetic: 85489
• scriptlet: 21106
• html: 780
• listset (total-discarded, last-updated):
• removed:

• plowe-0: null
  

• added:

• https://raw.githubusercontent.com/troysjanda/MyBlockLists/master/UBOXPI: 482-1, 9m
  

• https://raw.githubusercontent.com/troysjanda/MyBlockLists/master/yahoocomplete.txt: 512-0, 9m
  

• https://www.i-dont-care-about-cookies.eu/abp/: 25366-4, 9m
  

• adguard-spyware-url: 763-3, 10m
  

• block-lan: 44-0, 10m
  

• curben-phishing: 97633-4, 9m
  

• curben-pup: 191-0, 9m
  

• fanboy-social: 23358-9, 9m
  

• ublock-annoyances: 4845-9, 9m
  

• default:

• user-filters: 1-0, never
  

• ublock-filters: 33562-133, 10m
  

• ublock-badware: 4204-0, 10m
  

• ublock-privacy: 268-0, 10m
  

• ublock-abuse: 76-0, 10m
  

• ublock-unbreak: 1870-3, 10m
  

• ublock-quick-fixes: 245-0, 10m
  

• easylist: 63219-250, 10m
  

• easyprivacy: 30615-1195, 9m
  

• urlhaus-1: 6456-0, 9m
  

• filterset (user): [array of 1 redacted]
• trustedset:
• added: [array of 40 redacted]
• hostRuleset:
• added: [array of 3 redacted]
• modifiedUserSettings:
• advancedUserEnabled: true
• showIconBadge: false
• webrtcIPAddressHidden: true
• modifiedHiddenSettings:
• autoCommentFilterTemplate: none
• extensionUpdateForceReload: true
• filterAuthorMode: true
• filterOnHeaders: true
• updateAssetBypassBrowserCache: true
• userResourcesLocation: [redacted]
• supportStats:
• allReadyAfter: 219 ms (selfie)
• maxAssetCacheWait: 237 ms

A specific URL where the issue occurs

inturnal page

Steps to Reproduce

1. open the settings panel
2. select the filter list tab
3. hit purge all caches
4. hit update now
5. after the builtin section seems to dramatically slow

Expected behavior

purge and update

Actual behavior

purge is fine but updating after the builtin section is slow

uBlock Origin version

1.44.5b0

Browser name and version

Firefox 105

Operating System and version

Windows 10 19045.2075 22H2

[gorhill]

EasyList server is having issue, including timing out sometimes. Remote servers issues are out of control of uBO and not a uBO issue.

[mapx-]

stephhawk

Can anyone access https://easylist.to/easylist/easylist.txt ? I cannot update EL and EP in uBO

monzta 

the domain is cached/protected via cloudflare. i'm not sure if that is the issue but when i login 
to the dashboard i see a huge amount of traffic coming from india. and a lot of threats do get blocked 
from there. the system status of cloudflare seems fine too. any advice what to do? 

[stephenhawk8054]

Not sure if the current EasyList's server situation causes this: uBlockOrigin/uAssets#14968

From the troubleshooting information, the user's EL list is trimmed down to 17 filters

default: 
    user-filters: 5-1, never
    easylist: 17-1, 3d.6h.52m
    easyprivacy: 30595-1147, 3d.6h.58m
    plowe-0: 3673-529, 5h.52m
    ublock-abuse: 76-0, 15h.21m
    ublock-badware: 4200-2, 3d.6h.56m
    ublock-filters: 32975-30, 3d.6h.55m
    ublock-privacy: 246-1, 15h.20m
    ublock-quick-fixes: 241-6, 5h.51m
    ublock-unbreak: 1855-29, 3d.6h.54m
    urlhaus-1: 7070-0, 5h.50m

Maybe we can switch to https://secure.fanboy.co.nz/easylist.txt (and https://secure.fanboy.co.nz/easyprivacy.txt) as the first update list temporarily until easylist.to server works normal again?

[dportvine]

From the troubleshooting information, the user's EL list is trimmed down to 17 filters

Maybe it's a bug?

uBlock Origin: 1.44.4
Chromium: 105
filterset (summary): 
  network: 100844
  cosmetic: 126563
  scriptlet: 24039
  html: 0
listset (total-discarded, last updated): 
  removed: 
    plowe-0: null
    urlhaus-1: null
  added: 
    https://easylist-downloads.adblockplus.org/bitblock.txt: 31411-27, 5m
    https://easylist-downloads.adblockplus.org/cntblock.txt: 1510-24, 5m
    fanboy-cookiemonster: 41090-287, 5m
    ublock-annoyances: 4827-52, 5m
  default: 
    user-filters: 14-0, never
    RUS-0: 42158-146, 6m
    easylist: 62950-86, 5m
    easyprivacy: 30630-1570, 5m
    ublock-abuse: 76-1, 5m
    ublock-badware: 4200-103, 5m
    ublock-filters: 32865-171, 5m
    ublock-privacy: 246-8, 5m
    ublock-quick-fixes: 233-9, 5m
    ublock-unbreak: 1859-45, 5m
filterset (user): [array of 13 redacted]
modifiedUserSettings: 
  advancedUserEnabled: true
  contextMenuEnabled: false
  showIconBadge: false
modifiedHiddenSettings: 
  filterAuthorMode: true
  popupFontSize: 13px
  popupPanelHeightMode: 1
supportStats: 
  allReadyAfter: 803 ms (selfie)
  maxAssetCacheWait: 647 ms

[stephenhawk8054]

Maybe it's a bug?

I'm not sure. Looks like not everyone has truncated lists issue. Some others have reported missing filters in EL list (one didn't have reddit.com##.promotedlink and one reported having 26k filters instead of 60k):

https://www.reddit.com/r/uBlockOrigin/comments/xgnupg/is_anyone_else_suddenly_getting_reddit_promoted/iot4b1p/?context

[Felipefpl]

The download of fanboy annoyances' list is slow too.

[uBlock-user]

https://adguard.com/en/blog/easylist-filter-problem-help.html