Inconsistency: Proxy Bypass list

[erosman]

Inconsistency in application of Bypass list to PAC scripts

• In Firefox passthrough applies to individual proxy settings as well as autoConfigUrl
• In Chrome bypassList applies to individual proxy settings but NOT pacScript

The global Bypass list that applies to both individual proxies and PAC script is more functional and useable.
N.B. Other browsers have not been checked.

rules has a bypassList property.

const config = {
  mode: "fixed_servers",
  rules: {
    proxyForHttp: {
      scheme: "socks5",
      host: "1.2.3.4"
    },
    bypassList: ["foobar.com"]
  }
};

pacScript doesn't have a bypassList property. Adding it will solve the issue.

const config = {
  mode: "pac_script",
  pacScript: {
    data: "function FindProxyForURL(url, host) {\n" +
          "  if (host == 'foobar.com')\n" +
          "    return 'PROXY blackhole:80';\n" +
          "  return 'DIRECT';\n" +
          "}",
    bypassList: ["foobar.com"]  // <- add bypassList
  }
};

proxy.json

{
"id": "ProxyRules",
  "type": "object",
  "description": "An object encapsulating the set of proxy rules for all protocols. Use either 'singleProxy' or (a subset of) 'proxyForHttp', 'proxyForHttps', 'proxyForFtp' and 'fallbackProxy'.",
  "properties": {
    "singleProxy": {"$ref": "ProxyServer", "optional": true, "description": "The proxy server to be used for all per-URL requests (that is http, https, and ftp)."},
    "proxyForHttp": {"$ref": "ProxyServer", "optional": true, "description": "The proxy server to be used for HTTP requests."},
    "proxyForHttps": {"$ref": "ProxyServer", "optional": true, "description": "The proxy server to be used for HTTPS requests."},
    "proxyForFtp": {"$ref": "ProxyServer", "optional": true, "description": "The proxy server to be used for FTP requests."},
    "fallbackProxy": {"$ref": "ProxyServer", "optional": true, "description": "The proxy server to be used for everthing else or if any of the specific proxyFor... is not specified."},
    "bypassList": {"type": "array", "items": {"type": "string"}, "optional": true, "description": "List of servers to connect to without a proxy server."}
  }
},
{
  "id": "PacScript",
  "type": "object",
  "description": "An object holding proxy auto-config information. Exactly one of the fields should be non-empty.",
  "properties": {
    "url": {"type": "string", "optional": true, "description": "URL of the PAC file to be used."},
    "data": {"type": "string", "optional": true, "description": "A PAC script."},
    "mandatory": {"type": "boolean", "optional": true, "description": "If true, an invalid PAC script will prevent the network stack from falling back to direct connections. Defaults to false."}
  }
},

[erosman]

See also Proxy override Host matching inconsistency with Chrome

[oliverdunk]

This isn't an API I'm familiar with but I started looking to try and understand the problem. A couple of questions:

• The APIs for Chrome and Firefox look fundamentally different (I don't think you can use mode: "pac_script" in Firefox, and instead you specify autoConfigUrl). Is this accurate?
• It looks like you can already achieve what bypassList would do for pac_scripts by implementing a bypass list within the script (i.e if (host == 'foobar.com') return 'DIRECT';). Do you agree or is there a use case I'm missing?

[erosman]

• The APIs for Chrome and Firefox look fundamentally different (I don't think you can use mode: "pac_script" in Firefox, and instead you specify autoConfigUrl). Is this accurate?

Correct. However that is only a naming difference which is not as important as a difference in functionality.

• It looks like you can already achieve what bypassList would do for pac_scripts by implementing a bypass list within the script (i.e if (host == 'foobar.com') return 'DIRECT';). Do you agree or is there a use case I'm missing?

Of course there are workarounds but the topic is about inconsistency.
Additionally, it requires a lot more code to implement CIDR notation e.g. "192.168.1.1/16" or IPV6 patterns in a PAC script.

Bypass list pattern processing is already part of Chrome code but it is not applied to PAC.

Furthermore, a PAC can be a remote URL, where it wont be possible to implement any workarounds.

[oliverdunk]

Thanks for the quick reply!

However that is only a naming difference which is not as important as a difference in functionality.

It seems like Chrome lets you specify the script inline, whereas Firefox requires you to point to a URL (and under a different property name)? I'm not trying to catch you out, I just want to understand the extent of the differences :)

Additionally, it requires a lot more code to implement CIDR notation e.g. "192.168.1.1/16" or IPV6 patterns in a PAC script.

Ah, this makes a lot of sense. I can definitely see why that's useful.

So far I'm thinking the APIs are different enough that this may be two separate feature requests for the Chrome and Firefox issue trackers. Any thoughts on that?

[erosman]

It seems like Chrome lets you specify the script inline, whereas Firefox requires you to point to a URL (and under a different property name)? I'm not trying to catch you out, I just want to understand the extent of the differences :)

In Chrome, there are different properties for the above i.e. pacScript & url

In Firefox's proxy.settings, you can use a URL or data URL, e.g. data:text data:application for the autoConfigUrl.

const pac = `function FindProxyForURL(url, host) { return "HTTPS 1.2.3.4:443"; }`;
const proxySettings = {
  proxyType: "autoConfig",
  autoConfigUrl: `data:application/x-ns-proxy-autoconfig;charset=utf-8,${encodeURIComponent(pac)}`,
};
browser.proxy.settings.set({ value: proxySettings });

So far I'm thinking the APIs are different enough that this may be two separate feature requests for the Chrome and Firefox issue trackers. Any thoughts on that?

The proxy process runs natively in the browsers. PAC is synchronous. For example dnsResolve() has to run synchronously.

The proxy configuration runs very early, before all other request type APIs. The configuration becomes part of browser configuration (not extension configuration).

The proxying would usually entail:

1- Check proxy configuration before any connection made
2- Process Bypass list if there is one
3- If matched, send request DIRECT (break here)
4- Process connection via set proxy

I would imagine, step 3 could be added to the Chrome PAC handling without any major compilations.
The main changes would be in the native Chrome code. The API changes should be fairly superficial.

At the moment ....

Chrome

• request -> check bypass list -> fixed server
• request -> PAC URL/String

Firefox

• request -> check bypass list -> fixed server or PAC URL/String

[erosman]

@oliverdunk To simplify the matter, the request is to add bypassList to PacScript properties (as in ProxyRules properties).

proxy.json

From:

{
  "id": "PacScript",
  "type": "object",
  "description": "An object holding proxy auto-config information. Exactly one of the fields should be non-empty.",
  "properties": {
    "url": {"type": "string", "optional": true, "description": "URL of the PAC file to be used."},
    "data": {"type": "string", "optional": true, "description": "A PAC script."},
    "mandatory": {"type": "boolean", "optional": true, "description": "If true, an invalid PAC script will prevent the network stack from falling back to direct connections. Defaults to false."}
  }
},

Update to:

{
  "id": "PacScript",
  "type": "object",
  "description": "An object holding proxy auto-config information. Exactly one of the fields should be non-empty.",
  "properties": {
    "url": {"type": "string", "optional": true, "description": "URL of the PAC file to be used."},
    "data": {"type": "string", "optional": true, "description": "A PAC script."},
    "mandatory": {"type": "boolean", "optional": true, "description": "If true, an invalid PAC script will prevent the network stack from falling back to direct connections. Defaults to false."},
    "bypassList": {"type": "array", "items": {"type": "string"}, "optional": true, "description": "List of servers to connect to without a proxy server."}
  }
},

Obviously, the associated processes have to be updated accordingly.

[oliverdunk]

@oliverdunk To simplify the matter, the request is to add bypassList to PacScript properties (as in ProxyRules properties).

I think we're in agreement - that feels like a feature request that only applies to Chrome though, right? If so it would be great if you could open one a bug to track the feature request.

Given how different these APIs are I would say it probably makes sense to close this issue once the bug is opened - I appreciate you bringing it up, but I'm not sure there is any cross-browser discussions to be had (other than agreeing on one API over the other, which feels out of scope for this particular issue).

Let me know if that makes sense 😄

[erosman]

@oliverdunk TBH, it is not a feature request per se.

It is an inconsistency affecting Chrome. Firefox already has this behaviour.
If you agree with the proposed implementation, please feel free to post a bug (I am not registered on bugs.chromium.org).

I am also not sure how other browsers are handling the bypass list. It would be good to have a consensus on the implementation.

Subsequently, this inconsistency topic can be closed.

For reference: Consensus on processing the bypass list in case of single proxy, and PacScript (URL/Script)

[oliverdunk]

Thanks for the understanding, I've filed a bug here: https://bugs.chromium.org/p/chromium/issues/detail?id=1495756

I'm going to close this out but I really do appreciate the suggestion - hopefully we can make some progress on it.

[erosman]

Support bypassList for PAC scripts in the chrome.proxy API

Status: Won't Fix (Infeasible)

Hi Oliver,

Thanks for looking into this!

I'm copying my reply from

CL:5227338:

"It looks like one of the issues this CL is trying to fix is the inconsistency with Firefox but I'm worried that trying to match the extension behaviour in the Chrome browser with Firefox will create inconsistent behaviour in Chrome OS:

• With this change, Chrome and Chrome OS platform daemons will honour the PAC bypass list, but not the PlayStore on Chrome OS since it's running on Android.
• There's no UI on Chrome OS to show the PAC bypass list so the proxy configuration will not be transparent to the user - this may be a privacy concern for consumers.
• This change will create inconsistencies between proxy configs coming from extensions vs proxy configs coming from enterprise policies.
• Even within the extension API, only explicitly configured PAC will have a bypass list option; PAC scripts discovered via the Web Proxy Auto-Discovery (WPAD) protocol will not."

In the end it will increase complexity to provide a workaround for modifying PAC script behaviour for scripts which the developer does not own (which does not seem like a good security practice).

Jeongwoo, I'm very happy to give early feedback on anything which may overlap with commercial networking; CL:5227338 is solid technical work but does not align with the Chrome OS networking model.

@oliverdunk Regardless of the final decision by the Chromium developers, please note the following:

Concept

"It looks like one of the issues this CL is trying to fix is the inconsistency with Firefox but I'm worried that trying to match the extension behaviour in the Chrome browser with Firefox will create inconsistent behaviour in Chrome OS:

The purpose of this topic is to make extension development more consistent across browsers and furthermore, to demonstrate that the process works successfully in another browser.

There is no conceptual difference between single proxy & PAC. PAC is simply a method to add conditions to the process.

• Single Proxy: browser -> proxy server A
• PAC: browser ->
  • if X -> proxy server A
  • if Y -> proxy server B

Bypass List

What is fine for a single proxy, is also fine for a conditional single proxy (i.e. PAC).

• Single Proxy: browser -> if not in bypass list-> proxy server A
• PAC: browser -> if not in bypass list ->
  • if X -> proxy server A
  • if Y -> proxy server B

Android

• With this change, Chrome and Chrome OS platform daemons will honour the PAC bypass list, but not the PlayStore on Chrome OS since it's running on Android.

Android is NOT designed to work well with proxies, single or PAC. Users mostly use a VPN on Android. Nonetheless, what is fine for a single proxy, is also fine for a conditional single proxy (i.e. PAC).

UI

• There's no UI on Chrome OS to show the PAC bypass list so the proxy configuration will not be transparent to the user - this may be a privacy concern for consumers.

Chrome doesn't have a UI for the bypass list for the single proxy either.

Enterprise Policy

• This change will create inconsistencies between proxy configs coming from extensions vs proxy configs coming from enterprise policies.

What is fine for a single proxy, is also fine for a conditional single proxy (i.e. PAC).
Enterprise Policy & extension can set the same preferences.

WPAD

• Even within the extension API, only explicitly configured PAC will have a bypass list option; PAC scripts discovered via the Web Proxy Auto-Discovery (WPAD) protocol will not."

chrome.proxy does not have any options under auto_detect to apply additional preferences.
Therefore, extensions can not affect how auto_detect behaves.
However, options exists under pac_script & fixed_servers.