Allow GET in webhook triggers by swiergot · Pull Request #56446 · home-assistant/core

swiergot · 2021-09-20T08:37:54Z

Proposed change

Accept HTTP GET method in webhook triggers. So far only POST, PUT and HEAD have been allowed, but many devices which can be configured to send HTTP commands support only GET.

Type of change

Dependency upgrade
Bugfix (non-breaking change which fixes an issue)
New integration (thank you!)
New feature (which adds functionality to an existing integration)
Breaking change (fix/feature causing existing functionality to break)
Code quality improvements to existing code or addition of tests

Additional information

This PR fixes or closes issue: fixes #
This PR is related to issue:
Link to documentation pull request:

Checklist

The code change is tested and works locally.
Local tests pass. Your PR cannot be merged unless tests pass
There is no commented out code in this PR.
I have followed the development checklist
The code has been formatted using Black (black --fast homeassistant tests)
Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

The manifest file has all fields filled out correctly.
Updated and included derived files by running: python3 -m script.hassfest.
New or updated dependencies have been added to requirements_all.txt.
Updated by running python3 -m script.gen_requirements_all.
For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

No score or internal
🥈 Silver
🥇 Gold
🏆 Platinum

To help with the load of incoming pull requests:

I have reviewed two other open pull requests in this repository.

homeassistant · 2021-09-20T08:37:56Z

Hi @swiergot,

It seems you haven't yet signed a CLA. Please do so here.

Once you do that we will be able to review and accept this pull request.

Thanks!

elliotmoso

Looks good

MartinHjelmare · 2021-09-22T13:23:50Z

See these rejected PRs for history in this matter:
#17782
#19648

frenck · 2021-09-27T11:34:42Z

I think we should maybe consider allowing for this, for 2 reasons:

As said in this and previous PRs, some devices may only support GET requests
Webhooks can be used to trigger "anything", and doesn't have to provide information even. I can see possibilities to use simple GET requests to trigger automation.

esev · 2021-12-16T04:11:43Z

If this PR is to apply to all webhooks, is it possible to add CSRF protection on these URLs?

davet2001 · 2022-01-30T18:25:28Z

+1 from me. QR codes and RFID tag URLs could then be used to trigger automations without apps.

davet2001 · 2022-01-30T19:19:19Z

@esev what type of CSRF protection did you have in mind? I don't think any of the use cases wanting this can set header tokens.

esev · 2022-02-03T00:01:41Z

@esev what type of CSRF protection did you have in mind? I don't think any of the use cases wanting this can set header tokens.

I agree. And I don't see any good solution except for using long random URLs that aren't shared publicly.

Just to clarify. I do see the value in this PR! It makes some integrations a lot simpler. My goal isn't to stop the PR. I just want to raise awareness of the issues. As long as users are aware of the abuse vector, they can take steps to avoid it.

I just fear folks will use this without considering the consequences. http://homeassistant.local:8123/api/webhook/ is a fairly common URL that will work on many people's HA instances. These are unauthenticated URLs. It doesn't take much to abuse this if folks aren't always generating secret random webhook IDs, or if they print the URLs on QR codes/NFC stickers. Any web page or email that contains html similar to the following will be enough. It doesn't matter where the html is hosted; all that matters is where the user's browser is running.

<html>
  <body>
    <img src="http://homeassistant.local:8123/api/webhook/Unlock_Door" />
    <img src="http://homeassistant.local:8123/api/webhook/Open_Garage" />
  </body>
</html>

Can a blueprint contain a webhook trigger? If so that might be another abuse vector to consider (blueprints with common webhook names, or attacker controlled blueprints). Especially with the easy-to-click "Add Blueprint" buttons.

That said, AFAICT the CSRF issue is already present with webhooks as the CORS headers already allow this (cors_allowed = True). This PR just makes it easier to abuse directly from any static email/web page; no javascript required.

This might get safer in the future though. See https://wicg.github.io/private-network-access/. I believe Chrome intends to support this (https://developer.chrome.com/blog/private-network-access-update/).

emufan · 2022-02-03T08:02:39Z

is a fairly common URL that will work on many people's HA instances

But you are aware that everything you wrote is the same for POST webhooks?

This request is to get GET and not to remove webhooks. 😉

emufan · 2022-02-03T17:39:57Z

Why are we discussing webhooks in this PR? This is completely not related to it.

Everything what you wrote is already there. Warning, off by default, created only if you create an automation, etc.

https://www.home-assistant.io/docs/automation/trigger/#webhook-trigger

And I saw this "only local" in the changelog from 2022.2.0 as well. At least I think so.

esev · 2022-02-03T18:23:00Z

The difference in this PR is that GET requests have no CORS preflight checks. The GET verb was intended to be a read-only action without side-effects. By adding GET, the webhooks are automatically triggerable from any static web page. That was not possible before this PR; you had to use some javascript or user interaction to trigger the webhook from a web page.

Example: With this PR, if I add a webhook with the ID Open_Garage. All a bad actor needs to do is trigger a GET to http://homeassistant.local:8123/api/webhook/Open_Garage. This can be in an email, a malicious ad, a URL shortener service, a PDF/document, a wiki/forum post that allows images, or any website. The user only needs to open the web page from a device on their local network, and doesn't need to interact with the web page. And no javascript is required. This bypasses the browser's built-in security mechanisms. And there was no indication in the UI that this might be unsafe.

This can't be prevented by checking if the device is on the local subnet. Until https://wicg.github.io/private-network-access/ is implemented in browsers, all that is required is that your browser (likely on the same subnet as HA) be able to access Home Assistant locally. And that only considers browsers. It could still be triggered by malicious content in document/PDF viewers, native apps, chat messages, etc.

Admittedly though, I think most browsers would block this behavior from secure sites. Browsers tend to not allow mixed content; http requests from https sites. So that would mitigate the risk a bit. But it would be great if we could also consider users who setup SSL for themselves too, and make them safe also.

Personally I'd like to see:

The cors_allowed = True be an option on each webhook individually, and for the default to be False (not related to this PR, but it would close the pre-existing CSRF browser-based security issue by default)
Another per-webhook option (Allow Web Browsers) to allow common browser User Agents to be permitted to trigger the webhook. And for that option to default to False. Or, as @davet2001 suggested, only allow specific User Agents. (Could be used to make this PR safer, but not a bullet-proof solution)
For this PR specifically: A third per-webhook option to allow 'GET' requests to trigger a webhook, as these bypass the CORS preflight checks in the browser. Again, I'd like to see this default to False.
Add an option to restrict the webhook to a specific IP address/ranges (suggested by @davet2001). Maybe default to the local subnet, but with the understanding that it doesn't provide protection for browsers in the default state (could be used to make this PR safer as long as the device at that IP isn't running a browser. Would be a big win for users who expose HA directly to the internet. Probably needs to consider reverse proxies: 'X-Forwarded-For')
Provide a default, randomly-generated, webhook ID in the UI. Add a warning in the UI if the webhook ID doesn't contain much entropy/randomness. Similar to how some websites have password complexity tests. (not directly related to this PR)
Disallow a webhook ID to be defined in a blueprint. Randomly generate one or let the user choose one. (assumes blueprints can define webhook triggers - I haven't verified this. This isn't directly related to this PR)

Each of these options could be described in the docs with examples of how they could be abused if enabled.

I'm not on the HA core development team, so don't take anything I've written as a requirement for this PR. I'm not suggesting the PR be blocked; I see how it is very useful. I'm only suggesting ideas to make things safe by default (from a browser security perspective) when using webhooks so users can enable them without adding security risks. IMHO I don't think users understand/consider the CSRF implications for browsers before using webhooks. I'd just like users to be aware of the danger, and need to perform an extra action, before using webhooks in an unsafe way. In it's current form, this PR increases the danger by bypassing CORS preflight checks in the browser, without providing any safe-by-default options. This increases risk for all previously defined webhooks that only expected POST/PUT/HEAD.

alderete · 2022-02-05T01:48:23Z

As someone who has requested GET support for webhooks, in several places, I think the additions proposed by @esev all sound like good ideas. In particular, the last four bullets:

Per-webhook option for GET — false by default
Per-webhook option for array of allowed IP addresses — local subnet by default?
In the Automation editor UI, generate a random webhook ID by default (but allow me to override)
Disallow webhook ID in blueprints, replace with random ID

All sound like terrific, and hopefully not terribly complicated additional development.

I'm sad that that might mean this PR requires some re-work. I have Shelly Button 1's that want GET webhooks today to make it easier for my wife to turn off the "smart" bedroom light switch without getting out of bed. (The switch is by the door; the bed is not.)

But safety first!

I'll personally commit to writing the doc to go with the changes, if the appropriate person wants to loop me in. (I'm a tech writer by day, and I have one commit for the webhooks doc already.)

esev · 2022-02-05T20:32:39Z

@alderete I've sent a PR for randomizing the webhook_id by default. home-assistant/frontend#11568 Could you take a look, and update the docs if you think there is anything needed for that?

The copy button on the right-side copies the URL for the webhook to the clipboard (http://homeassistant.local:8123/api/webhook/bnWvYB5KLRBbkT9XMCl9zXoX). I figured that makes it a bit easier to use too.

Thanks for volunteering to update the docs!

alderete · 2022-02-06T03:05:56Z

I just submitted a PR for the webhooks doc: home-assistant/home-assistant.io#21506

I attempted to add security details in the form of best practices and considerations (don't open your garage door, etc.), and other concerns discussed in this thread.

And I'm happy to continue to update the docs as this and other open webhook-related PRs move forward. (That is, please don't let doc be the blocker on this PR. I will fix it if that's what's needed.)

esev · 2022-02-07T01:53:49Z

I'm putting together a PR to:

Add an option to allow webhook triggers to be accessed from the internet, and making the default False.
Make all HTTP methods optional, and enabling enabling 'POST' & 'PUT' by default.

For 1, I'm making it so that Nabu Casa's webhooks still work regardless & any webhooks triggered via the websocket API still work too. For both of these, the user would have needed to explicitly enable it, so that seems safe to me.

Number 2 will be easy to adapt for this PR, and add 'GET' as an option too.

trigger:
  - platform: webhook
    webhook_id: bnWvYB5KLRBbkT9XMCl9zXoX
    allow_methods:
      - PUT
      - POST
    allow_internet: false

I'm going to wait to see if the core developers give any feedback on the two outstanding PRs, or this PR, before proceeding on this next change though. I'll need some help updating the docs and creating a note about this being a breaking change too, assuming this all looks okay.

balloob · 2022-02-08T22:35:02Z

I like the idea of opt-in GET. We should make sure it applies to the hooks coming in via cloud/websocket too.

davet2001 · 2022-09-06T20:50:03Z

@byroncoetsee @FutureTense I don't think anyone is objecting to this feature.

It's just not been reviewed yet.

You could help by reviewing the PRs - even a partial review helps. 😀

byroncoetsee

Looks good to me - pretty straight forward 😃

byroncoetsee · 2022-10-02T10:54:48Z

@davet2001 reviews added and awaiting merge... 👍🏼

esev · 2022-10-02T18:50:37Z

Are we saying that it's better to strip ability and responsibility rather than give someone an opt-in feature (which could have like 3 warning popups if you're that concerned about this stranger's inability to make their own decisions)?

I'd like to have this as an opt-in feature so that folks who previously setup webhooks are not impacted by this change. This PR will decrease security for all previously added webhooks. That's why I proposed the other PRs, to give users an option to enable GET if they need it.

frenck · 2023-04-12T14:02:49Z

I've rebased the PR onto the latest dev branch (it was 21815 commits behind) and resolved the merge conflict.

pvizeli · 2023-04-12T15:49:31Z

I think before we add this, we should consider: #66494

frenck · 2023-04-12T15:50:19Z

@pvizeli yes, that has been discussed and is next up.

frenck

Thanks, @swiergot 👍

../Frenck

frenck · 2023-04-12T17:22:13Z

This PR is good to go, but marking it as draft until #66494 is ready to go as well.
The aim is to get them in the same release.

frenck · 2023-04-14T07:45:49Z

Alright going to merge this one in, now #66494 is ready to go too.
Thanks again @swiergot 👍

../Frenck

epenet · 2023-04-14T11:54:23Z

~~This appears to have broken CI~~

Error: tests/components/webhook/test_init.py:183:11: F811 Redefinition of unused `test_webhook_get` from line 134

Edit: not this PR.

homeassistant added the cla-needed label Sep 20, 2021

probot-home-assistant bot added core has-tests integration: webhook new-feature small-pr PRs with less than 30 lines. labels Sep 20, 2021

homeassistant added cla-signed and removed cla-needed labels Sep 20, 2021

elliotmoso approved these changes Sep 21, 2021

View reviewed changes

mountainsandcode mentioned this pull request Feb 1, 2022

Webhooks missing support for GET requests #65348

Closed

This comment has been minimized.

Sign in to view

This comment was marked as off-topic.

Sign in to view

esev mentioned this pull request Feb 5, 2022

Generate random webhook_id and add copy button home-assistant/frontend#11568

Merged

9 tasks

esev mentioned this pull request Feb 6, 2022

Disallow cross-origin requests for webhooks #65928

Closed

22 tasks

esev mentioned this pull request Feb 14, 2022

Add webhook trigger allowed_methods/local_only options #66494

Merged

22 tasks

byroncoetsee reviewed Sep 7, 2022

View reviewed changes

byroncoetsee approved these changes Sep 7, 2022

View reviewed changes

This comment was marked as spam.

Sign in to view

Mincka mentioned this pull request Dec 4, 2022

Allow clickAction to be used as a regular action to send back events home-assistant/android#3134

Closed

CR6-SE approved these changes Jan 5, 2023

View reviewed changes

frenck self-requested a review January 6, 2023 11:25

frenck self-assigned this Jan 6, 2023

This comment was marked as off-topic.

Sign in to view

Allow GET in webhook triggers

5f4806e

frenck force-pushed the webhook_get2 branch from 892641b to 5f4806e Compare April 12, 2023 14:02

frenck requested a review from a team as a code owner April 12, 2023 14:02

frenck added the noteworthy Marks a PR as noteworthy and should be in the release notes (in case it normally would not appear) label Apr 12, 2023

frenck approved these changes Apr 12, 2023

View reviewed changes

frenck marked this pull request as draft April 12, 2023 17:22

frenck marked this pull request as ready for review April 14, 2023 07:44

frenck merged commit 47f5160 into home-assistant:dev Apr 14, 2023

github-actions bot locked and limited conversation to collaborators Apr 15, 2023

Uh oh!

Conversation

swiergot commented Sep 20, 2021

Proposed change

Type of change

Additional information

Checklist

Uh oh!

homeassistant commented Sep 20, 2021

Uh oh!

elliotmoso left a comment

Choose a reason for hiding this comment

Uh oh!

MartinHjelmare commented Sep 22, 2021

Uh oh!

frenck commented Sep 27, 2021

Uh oh!

esev commented Dec 16, 2021

Uh oh!

davet2001 commented Jan 30, 2022

Uh oh!

davet2001 commented Jan 30, 2022

Uh oh!

This comment has been minimized.

esev commented Feb 3, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

emufan commented Feb 3, 2022

Uh oh!

This comment was marked as off-topic.

emufan commented Feb 3, 2022

Uh oh!

esev commented Feb 3, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alderete commented Feb 5, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

esev commented Feb 5, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alderete commented Feb 6, 2022

Uh oh!

esev commented Feb 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

balloob commented Feb 8, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

davet2001 commented Sep 6, 2022

Uh oh!

byroncoetsee left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

byroncoetsee commented Oct 2, 2022

Uh oh!

esev commented Oct 2, 2022

Uh oh!

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

frenck commented Apr 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pvizeli commented Apr 12, 2023

Uh oh!

frenck commented Apr 12, 2023

Uh oh!

frenck left a comment

Choose a reason for hiding this comment

Uh oh!

frenck commented Apr 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

frenck commented Apr 14, 2023

Uh oh!

esev commented Feb 3, 2022 •

edited

Loading

esev commented Feb 3, 2022 •

edited

Loading

alderete commented Feb 5, 2022 •

edited

Loading

esev commented Feb 5, 2022 •

edited

Loading

esev commented Feb 7, 2022 •

edited

Loading

balloob commented Feb 8, 2022 •

edited

Loading

byroncoetsee left a comment •

edited

Loading

frenck commented Apr 12, 2023 •

edited

Loading

frenck commented Apr 12, 2023 •

edited

Loading

epenet commented Apr 14, 2023 •

edited

Loading