Disable wayback machine for ".onion" addresses

[kjozwiak]

Test plan

See brave/brave-core#5306

Description

As per several community requests, we should look into disabling the Wayback Machine for .onion domains whenever they hit any of the following error codes:

static base::flat_set<int> responses = {
      net::HTTP_NOT_FOUND,              // 404
      net::HTTP_REQUEST_TIMEOUT,        // 408
      net::HTTP_GONE,                   // 410
      451,                              // Unavailable For Legal Reasons
      net::HTTP_INTERNAL_SERVER_ERROR,  // 500
      net::HTTP_BAD_GATEWAY,            // 502,
      net::HTTP_SERVICE_UNAVAILABLE,    // 503,
      net::HTTP_GATEWAY_TIMEOUT,        // 504,
      509,                              // Bandwidth Limit Exceeded
      520,                              // Web Server Returned an Unknown Error
      521,                              // Web Server Is Down
      523,                              // Origin Is Unreachable
      524,                              // A Timeout Occurred
      525,                              // SSL Handshake Failed
      526                               // Invalid SSL Certificate
  };

The Internet Archive doesn't archive/index .onion sites.

Community request:

• https://www.reddit.com/r/brave_browser/comments/g3y1iw/wayback_machine_in_private_mode_with_tor/

Steps to Reproduce

We'll need to find some STR as I don't see anything via the community thread. Most likely need to find an .onion site that has links within it that 404 or run into any of the above error codes.

Actual result:

The Wayback Machine is currently being displayed when users run into any of the above error codes while on .onion sites.

Expected result:

The Wayback Machine should be disabled for .onion domains.

Reproduces how often:

Still needs a case/STR.

Brave version (brave://version info)

Brave | 1.9.32 Chromium: 81.0.4044.113 (Official Build) nightly (64-bit)
-- | --
Revision | cf9d66371ea608e227eed56ccba3abc2701bd23d-refs/branch-heads/4044@{#936}
OS | macOS Version 10.15.3 (Build 19D76)

Version/Channel Information:

• Can you reproduce this issue with the current release? Yes
• Can you reproduce this issue with the beta channel? Yes
• Can you reproduce this issue with the dev channel? Yes
• Can you reproduce this issue with the nightly channel? Yes

Other Additional Information:

• Does the issue resolve itself when disabling Brave Shields? N/A
• Does the issue resolve itself when disabling Brave Rewards? N/A
• Is the issue reproducible on the latest version of Chrome? N/A

Miscellaneous Information:

CCing @rebron @bsclifton @diracdeltas @tomlowenthal @Miyayes

[DanielKim1]

Found a way to reproduce:

1. Click New Private Window with Tor (option + command + N)
2. Go to http://www.nytimes3xbfgragh.onion/404 (The 404 page for the New York Times)

[btlechowski]

Verification passed on

Brave	1.10.71 Chromium: 81.0.4044.138 (Official Build) dev (64-bit)
Revision	8c6c7ba89cc9453625af54f11fd83179e23450fa-refs/branch-heads/4044@{#999}
OS	Ubuntu 18.04 LTS

Reproduced on 1.9.x using http://www.nytimes3xbfgragh.onion/404

Verified test plan from brave/brave-core#5306
http://www.nytimes3xbfgragh.onion/404

https://open.nytimes.com/https-open-nytimes-com-the-new-york-times-as-a-tor-onion-service-e0d0b67b7482

Verification passed on

Brave | 1.10.79 Chromium: 81.0.4044.138 (Official Build) dev (64-bit)
-- | --
Revision | 8c6c7ba89cc9453625af54f11fd83179e23450fa-refs/branch-heads/4044@{#999}
OS | Windows 10 OS Version 1803 (Build 17134.1006)

Reproduced in 1.9.7 using in TOR tabs http://www.nytimes3xbfgragh.onion/404

Verified test plan from brave/brave-core#5306

using http://www.nytimes3xbfgragh.onion/404 in TOR tabs

Reproduced the original issue on macOS 10.15.5 x64 using the following build:

Brave | 1.9.76 Chromium: 81.0.4044.138 (Official Build) (64-bit)
-- | --
Revision | 8c6c7ba89cc9453625af54f11fd83179e23450fa-refs/branch-heads/4044@{#999}
OS | macOS Version 10.15.5 (Build 19F101)

Verification PASSED on macOS 10.15.5 x64 using the following build:

Brave | 1.10.81 Chromium: 81.0.4044.138 (Official Build) beta (64-bit)
-- | --
Revision | 8c6c7ba89cc9453625af54f11fd83179e23450fa-refs/branch-heads/4044@{#999}
OS | macOS Version 10.15.5 (Build 19F101)