Set 1 sec/chapter as default download rate

[dteviot]

From @gamebeaker

as an idea maybe select a value for Manual Throttle|Delay per chapter in the "default" configuration meaning if someone installs the extension for the first time. The reason is that a lot of sites have a cloudflare protection or something similar. I also know a few pages that if your IP gets on a blocklist they don't send a 4xx code but an empty document. And to get unblocked you have to wait.
If you want to implement it I would set 1 sec/chapter as default.

[Kiradien]

Just want to tack an addendum on here, it might be worth setting the rate parameter as a parser variable; different sites are 'good' at different speeds, CloudFlare pretty much demands 3-5 seconds depending on the domain.

[dteviot]

@Kiradien
I'm going to assume you mean "save value on a per site basis". If I'm wrong please correct.

[Kiradien]

"Save Value on a per site basis" would be a potential starting point for it, so long as the default for each site was configurable via its respective Parser.js. It's probably the 'best' solution, but also the most complicated.

The 'easiest' alternative is to have an extra object variable in Parser.js. amounting to [Minimum Delay] = 0, and calling Math.max(manualDelayPerChapterValue, this.[Minimum Delay]) before util.sleep.
Then we'd just set this.[Minimum Delay] = 3000-5000 inside the parser for the troublesome cloudflare sites, or longer on other domains that'd require/request it.
Under this option, I'd assume the goal of a 'checkbox' to ignore minimum delay would be desired. Icky front-end stuffs.

The main goal of this issue is to reduce the root cause of many-a-ticket, but the 1-second-delay is a stopgap without extensibility. There are dozens of ways to resolve it, aiming for by-domain configuration just seems best to me - whether that setting is code-only or not.

[Kiradien]

Wow, I'm getting back to this message late, I blame travelling.
Kiradien@a28c612 - an example of what I was referring to. We just set that object variable in each parser where speed is a known issue.

We can also, if needed, set the global default minimum of Parser.js to 1000 for 1 second, if needed, but it does seem like a more tailored option might be better.

It's tested and works for ScribbleHub and Fanfiction.net in the above changeset, but those are just the ones I remember using Cloudflare.

The question is: Do we want to go with this approach?

[dteviot]

Test versions for Firefox and Chrome have been uploaded to https://drive.google.com/drive/folders/1B_X2WcsaI_eg9yA-5bHJb8VeTZGKExl8?usp=sharing. Pick the one suitable for you, follow the "How to install from Source (for people who are not developers)" instructions at https://github.com/dteviot/WebToEpub/tree/ExperimentalTabMode#user-content-how-to-install-from-source-for-people-who-are-not-developers and let me know how it goes.

[dteviot]

Updated version (0.0.0.167) has been submitted to Firefox and Chrome stores.
Firefox version is available now.
Chrome might be available in a few hours to 21 days.