Switch reporting to Structured Headers

[clelland]

Per #158, we should consider switching to structured headers for the Report-To header

Using this issue as a bikeshed forum for SH concerns.

[clelland]

Given an example from the existing spec (with a couple of other options added):

Report-To: { "group": "csp-endpoint",
              "max_age": 10886400,
              "endpoints": [
                { "url": "https://example.com/csp-reports", "priority": 1 }
                { "url": "https://backup.com/csp-reports", "priority": 2 }
              ] },
            { "group": "hpkp-endpoint",
              "endpoints": [
                { "url": "https://example.com/hpkp-reports" }
              ] }

There seem to be a few ways that we can express this with SH:

1. As a pure dictionary, where just map endpoint group names to lists of URIs. We lose the ability to attach items like 'priority' to individual endpoints, but can still assign params to each endpoint group.

csp-endpoint=("https://example.com/csp-reports"
              "https://backup.com/csp-reports");max_age=10886400,
hpkp-endpoint=("https://example.com/hpkp-reports")

2. Using a parameterized inner list (based on the suggestion in [SH] Allow parameterized inner-list values. httpwg/http-extensions#907 the inability to attach parameters to both inner and outer list items means that we need to duplicate parameters like max_age here:

csp-endpoint=("https://example.com/csp-reports";priority=1;max_age=10886400
              "https://backup.com/csp-reports";priority=2;max_age=10886400),
hpkp-endpoint=("https://example.com/hpkp-reports")

3. Parameterized inner list with parameters at each level. This imagines a world where the restriction in (2) is not present:

csp-endpoint=("https://example.com/csp-reports";priority=1
              "https://backup.com/csp-reports";priority=2);max_age=10886400,
hpkp-endpoint=("https://example.com/hpkp-reports")

4. Using two headers; one to define endpoints, and one to group them.

Report-Endpoints: primary="https://example.com/csp-reports";priority=1,
    backup="https://backup.com/csp-reports";priority=2,
    hpkp="https://example.com/hpkp-reports"
Report-Endpoint-Groups: csp-endpoint=(primary backup);max_age=10886400,
    hpkp-reports=(hpkp)

[dcreager]

I'm game for migrating to Structured Headers. We'll want to handle the migration from JSON to SH carefully, since the JSON format has been shipped in Chrome for almost a year now. Presumably a separate header name, so that site owners can provide both a legacy Report-To and a new Report-Endpoints (or whatever) for some period of time?

[clelland]

Chromium could probably support both in the same header; they should be easily distinguished; if the first non-whitespace character is '{', then parse as JSON, otherwise parse as SH.

[clelland]

@mnot - the syntax in (3) is what I'd love to be able to use, but requires changes to Structured Headers. Do you know if HTTPWG would support such changes?

[igrigorik]

Agree, (3) would be ideal.

Wondering if we ought to take this discussion directly to httpwg/http-extensions#907 or open a new issue on httpwg/http-extensions: I think we'll get more visibility on this over there.

[mnot]

Will consider on httpwg/http-extensions#907, but a couple of questions / thoughts:

• Why does this need to be one header field? It seems very odd to have different headers for the mechanisms, but a shared header for the reporting data. E.g., for the case above, why not CSP-Reports and HKPK-Reports headers?

• Why does priority need to be an explicit number, instead of using the natural list ordering?

• From discussion at TPAC, it seems like Origin Policy is starting to move again -- would it be a better fit?

[dcreager]

Why does priority need to be an explicit number, instead of using the natural list ordering?

There can be multiple endpoints with the same priority. Uploads will be balanced across all of them according to their weight properties.

[dcreager]

From discussion at TPAC, it seems like Origin Policy is starting to move again -- would it be a better fit?

I think we are considering moving the content into Origin Policy, but IIUI that just changes how the headers would be delivered, not the syntax of their contents, right?

[clelland]

for the case above, why not CSP-Reports and HKPK-Reports headers?

The split between CSP and HKPK endpoint groups was from the original example, and certainly isn't the only way that reporting could be split up; I could see scenarios where different kinds of reports get sent to different parts of an organization. I also expect that many site owners will just want to have one endpoint group defined and use that for all of the reports types, rather than having to re-state all of the endpoints for each type.

Also, reporting allows you to define a 'default' endpoint group, where deprecation, intervention and crash reports are sent to, without having to have an actual header defined for each of those.

From discussion at TPAC, it seems like Origin Policy is starting to move again -- would it be a better fit?

I think we are considering moving the content into Origin Policy, but IIUI that just changes how the headers would be delivered, not the syntax of their contents, right?

Origin policy is definitely a possibility, and it could change the syntax; it has moved from being just a pure header-delivery mechanism, to being able to define policies/configuration more directly.

One thing that I definitely expect to see is that Network Error Logging will get its configuration from origin policy rather than the Report-To header. (See #158.) Report-To should continue to be used for setting up in-document reporting, which doesn't require configuration to outlive the document.

In that case, we might be able to move max_age from this header into Origin Policy, but I haven't thought through everything that will be required for that. It might mean that for this header,we can use

csp-endpoint=("https://example.com/csp-reports";priority=1
              "https://backup.com/csp-reports";priority=2),
hpkp-endpoint=("https://example.com/hpkp-reports")

which only requires inner-list parameters.

[clelland]

httpwg/http-extensions#907 is closed now; the latest drafts of Structured Headers now support parameters on all Items, as well as on inner lists. This means that we can now actually implement the full syntax as a list serialization, as in example (3):

csp-endpoint=("https://example.com/csp-reports";priority=1
              "https://backup.com/csp-reports";priority=2);max_age=10886400,
hpkp-endpoint=("https://example.com/hpkp-reports")

[igrigorik]

Yay, that's great! I guess that removes the last hurdle for us to make the switch.. 👍

[dcreager]

This is great! Thanks for driving that forward. So it sounds like the backwards-compatibility story would that Chromium would support both formats in Report-To for a release or two. And at some point during that time, site owners would have to migrate from the JSON format to the SH format. Does that sound right? That sounds broadly okay to me, though it means that site owners would lose reporting from old Chromium versions after they make the switch.

[clelland]

https://github.com/w3c/reporting/tree/structured-headers is a first attempt at switching to this (though it's currently rebased on top of #191, so it doesn't have, for example, max_age)

One issue with the syntax as proposed is that dictionary entries must have a key, so it's not possible to simply have 'default' assumed if it is missing. A group explicitly named 'default' must currently be defined in order to enable crash/deprecation/intervention reports.

[clelland]

Re: #177 (comment),

I've thought about it some more, and I don't think we can support two parsers for the same field in this case, and we definitely can't drop support for JSON until Origin Policy is available.

It's not a problem with ambiguity, but the fact that until Origin Policy is ready, users are going to need to continue to be able to configure Network Error Logging endpoints with the Report-To header, and that syntax is not being migrated to structured headers, and NEL configuration isn't even staying in the header. That means that unless we make NEL users migrate twice (once to a temporary structured-headers format, and then again to Origin Policy), they can't switch at all to structured headers, even for document-centred reporting mechanisms.

In that case, we probably do need to define a Report-Endpoints or Reporting header, and deprecate Report-To, while maintaining support until an alternative, persistent, origin-wide configuration mechanism is available.

[mnot]

FYI Structured Headers is in Working Group Last Call.

[clelland]

Thanks, @mnot; I'm taking a look at the most recent draft now.

It looks like its in pretty good shape; I've submitted issues for a couple of minor nits.

The only thing that would be on my wish list, as far as reporting goes, would be that something like * would be a valid dictionary member name (not just a token), to substitute for the previously implicit "default" endpoint.

Is that worth opening an issue for, or is LC too late for that kind of change?

[mnot]

Open an issue; worth a discussion.

…

On 7 Feb 2020, at 5:18 am, Ian Clelland ***@***.***> wrote: Thanks, @mnot; I'm taking a look at the most recent draft now. It looks like its in pretty good shape; I've submitted issues for a couple of minor nits. The only thing that would be on my wish list, as far as reporting goes, would be that something like * would be a valid dictionary member name (not just a token), to substitute for the previously implicit "default" endpoint. Is that worth opening an issue for, or is LC too late for that kind of change? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[clelland]

Closed via 7da9f80.