[feature] Add support curl-impersonate to obfuscate basic HTTP requests to mimic regular browser behavior as much as possible

[wrobelda]

Version and OS
0.44 on Linux.

Is your feature request related to a problem? Please describe.
I am getting 403 on some of the websites recently, somehow they must detect Playwright is in use instead regular Chrome instance, since I can access the same sites on my regular browser just fine over the same network.

curl-impersonate makes effort to come off as identical as known UI browsers, e.g. through replicating a set of encoding handshakes, thus avoiding getting fingerprinted. Support for curl-impersonate can be easily added as a stand-in for existing libcurl, wherever the latter is in use. I have previously added support for it to rss-bridge and it was fairly straightforward, since they use PHP which itself relies on libcurl internally for HTTP handling. That having said, I see that changedetection uses Requests library for its basic HTTP calls, and as such it would require additional support for curl to be added for that to work.

[dgtlmoon]

I am getting 403 on some of the websites recently, somehow they must detect Playwright is in use instead regular Chrome instance, since I can access the same sites on my regular browser just fine over the same network.

hmm but playwright is chrome... it's not todo with the encoding, they are detecting based on if the machine supports GPU, type of screen etc etc

[dgtlmoon]

Support for curl-impersonate can be easily added as a stand-in for existing libcurl,

They dont support an 'official' python library either :/

another thing is, I'de like to see some proof where this works and using playwright does not