Some images from Exhentai will never load because of H@H

[ghost]

Every now and then, one of their H@H clients will shit itself and stop serving images which means it will never load. Luckily there's a way to load images from the main server which you can do by clicking on "Click here if the image fails loading". It'd be great if gallery-dl could do it too. Have it switch over to the main server when it can't download an image from H@H and then switch back to H@H for other images and repeat the process whenever it fails to grab something from H@H.

https://exhentai.org/s/afac70af68/570103-31 < image won't load because one of their H@H clients is crapping out as always
https://exhentai.org/s/afac70af68/570103-31?nl=33983-444667 < image will load but it'll take a huge chunk from your image limit because it's a direct link to the main server

[downloader.http][warning] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond')) (1/inf)

[wankio]

my account have about 4-5mil point (by seeding with H@H, upload gallery) and fully upgraded option, so i can reset limit each time i'm reached it. i think you should raising your account limit instead

[ghost]

And what does that have to do with gallery-dl failing to download images from H@H?

[wankio]

dont know but most of gallery i'm downloaded with gallery-dl have no problemt at all, when i'm reached my images limit, i'm just click reset limit in my account on g.e-hentai.org, it cost 5000point if i'm remember and my imgs limit is 37k(or higher)

[ghost]

dont know

Of course you don't know what you're talking about. This issue has nothing to do with the site's image limit. Are you trolling or bad at reading?

[mikf]

Images not loading because of a H@H failure thankfully never happened for me ...
I'm guessing in that case a download will never succeed even after 10 or more retries?

The "Click here if the image fails loading" method doesn't really work with gallery-dl's current infrastructure. It supports fallback URLs, but they need to be available beforehand, which would mean clicking the link in all cases and paying the image limit.

[ghost]

With retries set to infinite it keeps trying to load the image from the normal URL until a different H@H client takes over which can take up to 24~ hours or days. We used to be able to go to account settings and disable H@H and only load images from the main server, but that option was removed in 2019 when Tenboro moved the main server from the Netherlands to Moldova.

I'm not a programmer but how does gallery-dl find the image URL from a gallery? Is there really no way to program it to take the URL from "Click here if the image fails loading" and use it as a backup URL when the normal URL is failing?

Edit: Actually the option to disable H@H was removed in 2020 and not in 2019. https://forums.e-hentai.org/index.php?showtopic=236113&hl=image+loading

Either way, this has been a problem for a half decade now. H@H is too unreliable and I don't think we'll get the option to disable it again.
https://forums.e-hentai.org/index.php?showtopic=220122&hl=image+loading
https://forums.e-hentai.org/index.php?showtopic=216757&hl=H@H+cluster

[mikf]

Is there really no way to program it to take the URL from "Click here if the image fails loading" and use it as a backup URL when the normal URL is failing?

There is, but with how things currently are, it would have to click the link and pay the image limit cost before starting the regular download even though it wouldn't be needed, i.e. it has to fetch any fallback URLs before the real download starts.

I'll probably implement something like a LazyURL to defer clicking on "Click here if the image fails loading" until absolutely necessary. This would also help with #886, where it currently would request the archive download link regardless if it was downloaded before or not.

[Fukitsu]

Any progress on the LazyURL? I've been getting lots of errors with exhentai lately, no address associated with host name, maximum retries exceeded, SSL key too small, etc. I have to change IP multiple times to get it working

[PascualBerchem]

I recently noticed that there are some H@H nodes which will always fail (and should also fail in a browser?!?), since their TLS setup is outdated (I can only guess how that happens, maybe totally out-of-date Java version - since H@H is implemented in Java). They fail with:

[SSL: DH_KEY_TOO_SMALL] dh key too small (_ssl.c:997)

Since this is only an issue with DH ciphersuites, a workaround which doesn't reduce security is quite easy: Just remove DH suites from the handshake. Here is a test script:

#!/usr/bin/env python3
import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.ssl_ import create_urllib3_context

CIPHERS = 'DEFAULT:!DH'

class NoDHSecurityAdapter(HTTPAdapter):
    """
    A TransportAdapter that re-enables 3DES support in Requests.
    """
    def init_poolmanager(self, *args, **kwargs):
        context = create_urllib3_context(ciphers=CIPHERS)
        kwargs['ssl_context'] = context
        return super().init_poolmanager(*args, **kwargs)

    def proxy_manager_for(self, *args, **kwargs):
        context = create_urllib3_context(ciphers=CIPHERS)
        kwargs['ssl_context'] = context
        return super().proxy_manager_for(*args, **kwargs)

s = requests.Session()
s.mount('https://', NoDHSecurityAdapter())
r = s.get('https://mprievm.komhhyguvgqr.hath.network:8443/')

[mikf]

@PascualBerchem Using the ciphers option should also work: -o ciphers="DEFAULT:!DH"