-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
High volume of requests to add domains to the PSL #78
Comments
Hi Ben! I’m not commenting on any policies on specific platforms, only on this proposed web standard. To understand this a bit better, are merchants registering new eTLDs to be able to track clicks to specific product pages, such as When we’ve discussed this previously, the two defenses have been 1) the registrable domain showing in the URL bar, and 2) re-engagement on the personalized registrable domain would be very hard unless a link was again followed from the same click source or the user leveraged a bookmark or their browsing history. However, for triggering events that happen on first engagement (=direct conversion), only the URL bar would be the defense and there may be merchants who only expect or care about measuring direct conversions. Flooding the PSL is clearly a problem given the link you provided. Browsers also don’t update their copy of the PSL on any guaranteed schedule that I’m aware of. Perhaps the only solution is to not support eTLDs in PCM and only support TLDs. That is my current thinking. Thoughts? |
That's a non-solution. That would cause tremendous harm to all the small businesses who operate on subdomains of TLDs like myshopify, and for what? We need a less binary solution here. If there ought to be some sort of vetting process to determine who is using subdomains in a way that is aligned with the intended purpose of the PSL, then we should actually invest in making that happen, not just throw up our hands and let thousands of businesses suffer. We both agree that cross-site tracking through personalized registrable domains is abusive, and is not an appropriate use of the PSL. I'm assuming you share the same desire to make this work only for those with a legit use-case. I'd propose a "facts and circumstances" approach. Let's actually take a look at which subdomains of the TLD are in use. Let's do an analysis to see if they appear to represent wholly different businesses, or if they appear to be something personalized. Let's invest in some education about what is, and isn't an appropriate use. Let's invest in some kind of a form people fill in to generate a request, that can weed out requests that aren't going to fly. And let's invest in some kind of ongoing monitoring to automatically remove TLDs that change their behavior post addition to the PSL. Let's try to find a more even-handed approach that enables legitimate usage. |
Who will vet such a list continuously at a global scale? The people maintaining PSL have already rejected the assignment purely on the volume of requests to be added. They have not looked at any actual use in my understanding, and are probably not obliged to do so. In my view, both |
There are three different examples to consider:
I don't believe there is any disagreement that both the 2nd and 3rd cases are problematic in the view of the intentions stated by PCM. However, the first case seems reasonable to support, in terms of the intention to allow for measurement of clicks to a site. It seems that the PSL was initially thought to be a way to support the first case, but that has caused the issued linked from their repository. While it may be difficult to technically differentiate these three examples above, it's at least useful to acknowledge that it would be useful to solve the 1st case while preventing the 2nd and 3rd. |
Agreed. Number one in your list is the "Etsy Case" tracked in #60. However, as discussed in that thread, doing it through PSL means users can not be logged in to, or potentially even browse to I have always assumed that sites like Etsy are not looking to become services in that sense but Etsy is using |
Apple should.
|
I can't speak for any individual multi-tenant website, but for the sites driving the current influx to the PSL are presumably making that decision. For example, this issue is one such service which describes its need as:
This specific pull request did get merged, but it seems that the PSL needs some support in handling the domains making the similar decision to "make it work with the PSL." |
There’s a fourth example to consider too, which is that nothing would stop retail-as-a-service.example from implementing a feature where they use their status on the PSL to set up a tracking feature that does exactly what gymshop.example does, using a prefix or suffix to distinguish tracked requests from ordinary requests? Also the second example - 5kg-kettlebells-in-red.gymshop.example - seems a bit problematic to prevent: leaving out companies with their own TLD (I’m not sure how that works…), they might also buy standard top-level domains for specific campaigns, products, etc. Buying a domain for a user session would be extreme unless free trials or refunds are involved but doing so for a limited set of products seems like something that’s hard to prevent even if we exclude use of the PSL list? It’s just PSL makes it cheaper, of course… |
Appreciate this discussion is happening. Any help in not flash-mobbing our volunteers and bloating the PSL is appreciated in how you address this. Please also see this: https://github.com/sleevi/psl-problems |
Looping back around on the original rationale behind this from the Facebook perspective ahead of our privacy-cg discussion later today:
It seems like we have one consensus point, one fact of how business operate, and three questions: |
Hi - a plea from a PSL maintainer - what's the latest on this stuff? I am now getting threatening calls from people freaking out that their Facebook Pixel stuff is going to break if they are not added and other visceral conversations out of band from the github repo for PSL PR's on this matter. What's an update? |
Hi @dnsguru, Firstly, sorry that you're getting threatening calls over this... that should never happen. The discussion in privacy-CG this week closed out that PSL is the "least bad" way of handling this today; due to the definition of eTLD+1 and the privacy requirement of cookie separation, there are going to be some use cases that fall into a space of needing PSL registration. Facebook could handle this on our side directly, however without the guarantee of cookie separation, this does not have the desired privacy guarantees. We clarified our documentation on this last week. Our intent was always for this to be for platforms and multi-faceted organisations such as governments as described above and our expectation was that if you weren't already on the PSL, then it is likely you do not need to be added to the PSL. You can see our updated guidance here: https://www.facebook.com/business/help/331612538028890 Specifically we added the paragraph:
From the Facebook side, @benjaminsavage, @n8schloss, and myself can volunteer to help you vet the PSL applications. I'm hoping that @johnwilander can nominate someone to support from Apple as well. I'm going to drop you an email for us to chat and agree how to proceed. In the meantime, please continue to reject iOS14/FB related PSL registrations as you have been doing and feel free to point advertisers who do apply in the direction of our updated help centre guidance. |
Sharing perspective from Shopify which is mentioned a few times in above discourse. Shopify submitted Per the above nomenclature, on the Shopify platform, When Apple announced PCM and Facebook subsequently adopted it as a standard into their platform, we advocated to Facebook that they support our merchants operating on Our hope is that there is no regression through these discussions for legitimate entities leveraging PSL appropriately and further looking to adopt privacy preserving mechanisms of advertising, such as PCM. |
Thanks! I went to a shopping site under |
Thanks John. We’d like to prevent this as well, your observed behaviour is likely from an embedded 3P integration. Once Webkit updates its implementation of the PSL, this should no longer be possible. I’ve filed a bug in webkit and would greatly appreciate any support, do not wish to hijack this issue. |
Hello, First of all thank you to all of the volunteers who actively participate to serve and maintain this list. Secondly, it's so irresponsible for Facebook to push this issue onto PSL volunteers. They have an internal tool for verifying subdomains which I have used, but the pixels installed on sites can only point to a root domain. It seems that if a root domain ownership is verified, the owner would be help accountable/responsible for the subdomains as well, the functionality to tie an FB pixel to a subdomain would be simple to implement for FB devs. IMO Facebook isn't really invested in better security, only reacting to ridicule over malicious, misinformation-based content. |
Please refrain from accusations and speculation (the IMO part). There’s been a bit of that above too so you’re not alone. Let’s stick to facts and the technology since this is a standards proposal conversation. There are plenty of places where you can discuss your opinion on certain actors. Thanks! |
The PSL Pull Requests that are coming in are manifesting as being more iterative than the typical ones - we seem to get stuff like this request below where the root domain name is requested and they need extra hand-holding on not mucking up their root domain cookie stuff and they have clearly done a copy/paste and threw this over the fence to the volunteers. Some are just copy/pasting from (new or old), other PRs, or otherwise submitting things that would not typically get approved. So the icing on the 'this ain't chocolate but still is brown' cake that we're getting served is that these are more process expensive, iterative, high-maintenence walkthroughs. Hoping to see some text for the wiki from FB folks to help direct people but this likely needs to end up being something FB is handling end to end akin to where Let's Encrypt have their own list and request form for their customers seeking to be working around their limits themselves. |
From the PSL volunteers campside, we're still awaiting some sample text for an FAQ or wiki entry from Facebook to help folks understand who to talk to AT FACEBOOK to address this. Meanwhile the team at Facebook are peeking at the PSL PRs and landing their helicopter from time to time and engaging with the request pool, but we'd like to see a bit more of that proactive spirit. A lot of the requests, if they had just been approved and merged, would have mucked up the root domain in ways the requestor would not have wanted, and due to the time-slide on when different browsers (in this case Apple) integrate and subsequently push their eTLD+1 derivative list, during System Updates. It is best that the PSL volunteers are wontfix closing the requests to workaround the 8 subdomain limit that FB has... Those requestors would have been in bad shape.if they would have to wait for post 14.5 IOS update 1 to discover the breakage and then make yet another PSL request and then wait for the subsequent IOS update to fix the fix. |
We need to make sure the organizations wanting to register understand the full implications of getting on the PSL. Mozilla has this documentation: https://wiki.mozilla.org/Public_Suffix_List/Use_Cases As you can see, getting on the PSL has consequences for wildcard TLS certificates, possibly invalidating existing ones and creating issues for existing certificate update procedures. Do the sites know about that? |
It's a mixed bag. Some of the requests are legitimate and understand this, others are legitimate with no understanding of this, others still are inappropriate for a PSL addition. We've just updated our guidance today found here that goes into more details. Specific section on PSL limitations below, and we've also clarified further that if you're not already on the PSL it most likely does not make sense to be on it.
|
Thanks! I haven't seen it documented anywhere but I think it's reasonable to assume that browsers will not allow a user to even visit/load an entry on the PSL. It's treated like a TLD and afaik you can't visit https://com. That's something for them to take into consideration, including any existing links, subresource URLs, bookmarks, QR codes etc. |
Yes, we were also not 100% certain on this for every browser, but it stands to reason. This is why we put in the line specifically around:
Many businesses don't understand this detail and the risk with different browser cadences for updating their own PSLs is that this may appear to be ok right now, but would be hard/nearly impossible to back out later, so definitely better to "measure twice, cut once" in this scenario |
Hey @johnwilander, just looking into this a bit more and would like to understand exactly how PCM would handle something on the PSL. From what I can see, the browser does in fact load a page that is PSL listed. The example I tried is http://gov.au which does not redirect, and is on the PSL. Now it could be a more recent addition to the PSL than the Safari list, but I believe it's been there quite a while so far as I can tell. In this circumstance (beyond just the limited functionality that the PSL site may have), if we had a PCM flow with |
If it's WebKit specific, please file a bug on https://bugs.webkit.org and cc me (thanks!). If it works the same in all browsers, we should bring it up with in the storage partitioning repo. |
It isn't webkit specific from what I can tell. Chrome and Firefox both allow for similar behaviours. How do we raise this with storage partitioning? Separately I think we should figure out how PCM (and actually all other proposals that use eTLD+1) handle the edge case of something that's on the PSL since the domain seems to still actually load. Let's have the storage partitioning conversation first and go from there |
It breaks partitioning and cookie scoping. Please file here: https://github.com/privacycg/storage-partitioning and reference this bug. Thanks! |
Raised and requested for next privacy-cg agenda: privacycg/storage-partitioning#24 |
Continuing to wontfix requests to the PSL, which are stacking up. Requestors have 'energy' to express about that and they are sharing with PSL. Really not appreciating this whole mess, folks. really. REALLY. R E A L L Y! The dialog seems fairly casual ... While FB folks have been helping a bit with triage, the rate increase on PR due to this change is really dumping on the PSL volunteer pool, and I had planned to attempt to address some of the backlog of automation and other improvements, so this whole matter has been incredibly disruptive - ultimately to the PSL volunteers of which I currently am the heaviest lifter. grrr to whomever put this in motion. And I want your cel number to pass along so that you're not denied the dignity of hearing from people affected and how expressive they are |
FB are standing up a process to alleviate the burden of validation of requests from PSL maintainers, which will hopefully reduce the load for the time being, however as discussed here, PCM and all privacy preserving proposals that currently rely on eTLD+1 and the PSL to determine what a business entity is need to find a better way. I don't see this problem going away, or diminishing in volume as more of these types of proposals progress from ideation to trial to production. It's clear from the situation here that this is not something easily taken on by the existing PSL structure. cc @johnwilander @csharrison @michaelkleber @krgovind for visibility |
@bedfordsean I plead... please 'stand up' faster.... perhaps at very least immediately update this web page : Here is the exact text:
Its really basic and folks continue to submit PR that are going to break their stuff. The guidance is absent any of the dialog or evolutions that we have been discussing in good faith to discourage the PSL as a workaround, does not identify the hazards about breaking their core domain inadvertantly or how browsers do their own thing per browser at their pace on propogation of changes or rollbacks, As worded, it will continue to push higher effort pull requests to the PSL in order for that Facebook customer to solve their issue. I do not work for Facebook, I am not paid to work for Facebook. I do not want to do unpaid work for Facebook. Straight up, that guidance is abandoning Facebook's responsibility to THEIR customer, and causing me and other volunteers to work for Facebook for free and I feel that is just reprehensible. While you and your colleagues have been monitoring some of the PR and walking people through their PR at the PSL repo on github, and that is appreciated, the root cause (the FB Guidance) needs to change.
I could see it alleviated a bit by making the guidance less basic on that URL pretty please
Just want FB to not hold a Fyre fesival PSL Island |
@dnsguru inbound form will be coming this week. I'll ensure that page is up to date ASAP |
@bedfordsean I think the wording on the Facebook page needs to reflect the reality of PSL: That there are no obligations for any users of the PSL to update their copy of PSL in any set timeframe. Hence a wording like:
Should get a strong clarification like:
As well as get new clarifying texts about additions added to it:
Right @dnsguru? Also: If there needs to be a registry for all of the Documenting all of the |
@voxpelli that information is already detailed on our updated guidance that I posted about here #78 (comment) This other page has been missed |
The updated guidance pages have reduced the pace of requests already - which is helpful. I do like @voxpelli suggestion on the 'historic' word being included to help further smush down the pace of requests. Once we see what kind of process FB has in place for "review, not refer" and can gauge the request quality, there will be a review of acceptance criteria and perhaps those caught in the queue of PR that accumulated can move forward. I am especially intreagued with the potential of a registry for the private section stuff - we have avoided requiring payments to also avoid the associated expectations that come along with them, but given this has grown a lot since we made that choice it would make sense to re-explore it. The concern I have is that charging money may unravel the voluntary nature of browser participation as much as accepting any volume of workarounds for security or rate limits might. I digress... We will likely see a few bursts again in cycles when affiliate types who are more 'set and forget' peek at their referral reports and scratch their heads as to what happened and discover the IOS change impacts. Any vetting pre PSL PR is going to help reduce the queue load, and mid PR interventions have been helpful to addressing the increased load. Yes, I have been [understandibly] cranky about this update of the FB help page situation due to how long it took before these would update to change from 'come to PSL for remedy' to current as FB's help for their clients adjusts away from punting towards helping, but I wanted to flip around and instead frame things from a position of gratitude - to also point out that we have had members of the team at FB stepping in to triage PR and reason out the requests so that they are well reasoned and proportionate. That has been helpful with the symptomatic increase in PR and the load - and the recent updates to the help pages should prove to be curative or slow the herd. The biggest issue I have had with all of this was that the voluntary nature of browser participation in use of the PSL has been powerful as a means to help namespace operators ensure their users experience was happening in an intentional manner. Volunteering towards that objective has always been fulfilling because it allows for making real beneficial change and enablement of communities and innovation at scale. I am biased in that I give fewer effs if some marketing affiliate entrepreneur wants to not spend $10 to buy their own domain to make thirty cents off a take out order they referred than I do in ensuring that remote learning functions as expected in say the entire country of Ecuador. Both are one line in a file. The latter feeds my soul a bit more for the same amount of effort as a volunteer on the PSL, and it helps principly to continue a legacy of graceful service that Gerv Markham represented while he was with us. |
One further update here; we have updated our page at https://developers.facebook.com/docs/sharing/domain-verification/ to point to the Help Center article (https://www.facebook.com/business/help/126789292407737) where we had already listed much more detailed guidance on PSL usage.
That Help Center page also includes the wording "pre-existing Public Suffix List domain registrations or eTLDs" to try to encourage not blindly making a PSL addition. You'll note that we also now have a request form present and linked on the Help Center page. We're going to be finalising the process that happens after submitting that form with @dnsguru and others via email in the next 24h for what to do for pre-validation of PSL requests. Once we've agreed on that we're going to follow up and post on every PR linked to publicsuffix/list#1245 and handle them again through the FB centric flow. I hope this will put us on to a more even keel and we can alleviate the burden on the PSL volunteers in the near term. In the longer term once we learn more about common business use cases that seem to require currently being on the PSL, we can bring these to future W3C meetings discussing PCM/privacy sandbox proposals and (hopefully) find a better way for the long term |
@bedfordsean @johnwilander Excellent - so will PSL folks be invited to tomorrow or thursday's virtual f2f? I pledge to not darth vader lift anyone across zoom |
One possible way to get a volunteer-like payment speed bump is asking for a credit card, charging something ($100, perhaps) and later refunding it (less processing costs). This at least makes people to think if they need it and qualify for it. If that's too much for the Global South, just use IP-based location to pick an appropriate value. |
I'm going to suggest we wait a couple more weeks for the following reasons:
There's a privacy-CG meeting every 2-3 weeks and a web-adv meeting every week, so suggest we collate this information and come in to have a smaller group chat with the parties this is most relevant to, as well as incorporating our learnings and understanding of any new proposals that come out of the Google and Apple developer events. |
NO, THANK YOU |
We have avoided charging for changes, historically, but that was when there had been 1-2 PR a month or so and could manage to resource it. |
The charge would be minimal after the refund, and the system would still have 0 income. |
At this point, charging an astronomical price for a rollback on casual requestors is making more and more sense, as those are the worst waste of PSL volunteer cycles to generate zero net outcome On the better news side of things, FB's lead is back from their two weeks off (sigh, they took that two weeks from ME ;) ) and they have now added an intake form to hopefully process to help qualify requestors. though the vetting process or acceptance criteria on those requests that come through their sieve are yet to be defined. I have created a label on PRs called 'IOS-FB?' to help identify PRs that were/are directed at the PSL for false salvation. There have been a number of request/rollbacks in the past month - so qualifying requests is really, really fundamentally important. It is also important to note that PSL inclusion is not any guarantee that anything propogates downstream to browsers, certs, DMARC, libs etc. They do what they do, when they do. So the rollback timing can have big ouch factors where the core business domain name is used and a ham-fisted request comes in on the domain. |
A requirement for PSL should be that "foo.myshopify.com" have sub-user A records or NS record control, to aim their site off Only true webhosting/VMs/web dev playgrounds/free sub domains and dynamic DNS make sense on PSL. If the major cause for PSL is for ad-tracking/e-commerce, the sub-users REALLY need to move to a proper eTLD/well known PSL TLD, if the sub-user a commercial biz, they can afford $30-$60 a year for the well known TLD domain. It looks more professional to clients/eyeballs. Most CDNs/platforms nowadays have The minimum future-TLD domain registration renewel requirement (years into the future) sounds smart, or my idea a 1 time payment for PSL listing if owner is a for-profit LLC, voided if the future-TLD has >1 year history by google search of use as sub-domains or void the PSL listing fee if proof of non-profit status of future-TLD owner background. PSL really needs to be free subdomains with A/NS/SOA/TXT records, not subdomains of "full stack" platforms to "1 company edge-proxies". The full stack, all of our parts/services, or none of our parts/services, platforms, control their sub-users so much anyways, and all use edge proxies/anti-DDOS/edge TLS termination anyways, the full-stack platforms don't really need to be on the PSL, or they can pay a PSL listing fee, that allocates resources for a PSL volunteer in the future to remove their eTLD from PSL if faux-eTLD domain owner goes bankrupt. |
Hi John,
We have a problem. As this GitHub issue explains, there is an increased volume of requests to add entries to the public suffix list.
One problem is with resourcing. The PSL is maintained by volunteers, and they are not prepared for a large volume of requests. This is definitely a problem we should all discuss at the next Privacy-CG.
But that is not the only problem. This is being mis-characterized as a “workaround for [...] security measures”.
Firstly, anyone who thinks that they have discovered a “workaround” is going to be sorely disappointed. And as you yourself pointed out in a recent tweet: “Today might be a good day to remind folks that adding a domain as an eTLD to the Public Suffix List (PSL) affects cookies. E.g. if you register shoppe.example to the PSL so that coffeemug.shoppe.example can be an eTLD+1, shoppe.example can no longer use cookies.”
I think we would both agree that it’s not feasible to use the PSL as a “workaround” for tracking users - it makes it so that the subdomains act, in effect, like separate websites. I think it might help if you could chime in on the issue to explain that there is no “security workaround” here.
So what is going on? Here’s my understanding:
Once iOS14 starts enforcing the ATT prompt, a large number of people will likely “opt-out” of “tracking”. Advertisers will still need to measure the aggregate count of conversions driven by their paid advertising, including conversions from people who have “opted-out” of “tracking”.
This is not a Facebook specific problem at all. They will face this same challenge everywhere they run ads. As such, businesses need to find alternatives ways to measure their ads that are allowed by Apple’s policies, such as “Private Click Measurement”.
“Private Click Measurement” conflates “business entity” with “registrable domain”. There are many small businesses who buy ads (on Facebook and elsewhere) that do not operate on their own eTLD+1 and instead operate on subdomains of websites like “foo.myshopify.com”. Unless they do something - they will no longer be able to measure their paid advertising.
The blog post which introduces PCM provides a possible solution to their dilemma. The blog post explains in the documentation of the “attributeon” parameter that PCM only supports “registrable domains” and links to this page, which specifically makes mention of the “Public Suffix List”.
Websites like “myshopify.com” that offer hosting to many separate businesses, as subdomains of that root, need to go and register themselves on the PSL prior to ATT enforcement.
Now clearly, this is only appropriate if there is no need for data-sharing between subdomains, and they are OK without having any cookies on the root domain.
So what is Facebook’s involvement here?
Facebook finds itself in the position of trying to help advertisers navigate Apple’s ATT changes - answering a wide variety of questions. We didn’t originally have any guidance around the PSL in our help articles until we received questions from advertisers who noticed PCM was supporting it. We are just trying our best to provide guidance about how to work with PCM. If you’d like to suggest any changes to the wording of that guidance, or to offer your own guidance about the PSL and how it interacts with PCM, we’d be happy to just direct people there.
As for the increased number of requests - this is an important issue worth discussing. The underlying cause of this increased demand is due to Apple’s upcoming ATT changes - so I think it would be sensible for Apple to help provide the PSL maintainers with additional support.
I do understand that a volunteer maintained group cannot be expected to meet SLAs for adding new entries here. But as you understand, with the uncertainty about when iOS’s ATT policy will be enforced, businesses are attempting to prepare in advance to avoid disruption. I’m open to discussing possible solutions to this capacity issue. Perhaps the next Privacy-CG is a good venue to discuss this.
The text was updated successfully, but these errors were encountered: