Add application credentials platform

[allenporter]

Proposed change

Initial implementation of a OAuth application credentials platform, from architecture discussion, initially supporting:

• Platform for integrations to provide authorization server information, and backwards compatibility for yaml based credentials
• Websocket list/add/delete
• Add developer credentials protocol from yaml config
• Handle OAuth credential registration and de-registration
• Tests for websocket and integration based registration
• Support for XBOX as initial test case

Upcoming considerations:

• Updated scaffold script and developer documentation
• "Import" and migration path for credentials
• Move existing local auth implementations to application credentials
• APIs for lower level manipulation for more complex cases (e.g. for nest)
• APIs needed for improved user experience (e.g. tighter config flow integration, smoother add/remove)
• Frontend changes to show the user interface (Add configuration panel for Application Credentials frontend#12344)

Type of change

• Dependency upgrade
• Bugfix (non-breaking change which fixes an issue)
• New integration (thank you!)
• New feature (which adds functionality to an existing integration)
• Breaking change (fix/feature causing existing functionality to break)
• Code quality improvements to existing code or addition of tests

Additional information

• This PR fixes or closes issue: fixes #
• This PR is related to issue:
• Link to documentation pull request: Documentation for Application Credentials platform initial features developers.home-assistant#1279

Checklist

• The code change is tested and works locally.
• Local tests pass. Your PR cannot be merged unless tests pass
• There is no commented out code in this PR.
• I have followed the development checklist
• The code has been formatted using Black (black --fast homeassistant tests)
• Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

• Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
• Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

• No score or internal
• 🥈 Silver
• 🥇 Gold
• 🏆 Platinum

To help with the load of incoming pull requests:

• I have reviewed two other open pull requests in this repository.

[balloob]

We prevent the router from getting frozen so this is not an issue to work around.

[allenporter]

I am having trouble reconciling your comment with the error message I see:

homeassistant/data_entry_flow.py:205: in async_init
    flow, result = await task
homeassistant/data_entry_flow.py:232: in _async_init
    result = await self._async_handle_step(flow, flow.init_step, data, init_done)
homeassistant/data_entry_flow.py:335: in _async_handle_step
    result: FlowResult = await getattr(flow, method)(user_input)
homeassistant/helpers/config_entry_oauth2_flow.py:319: in async_step_user
    return await self.async_step_pick_implementation(user_input)
homeassistant/helpers/config_entry_oauth2_flow.py:236: in async_step_pick_implementation
    implementations = await async_get_implementations(self.hass, self.DOMAIN)
homeassistant/helpers/config_entry_oauth2_flow.py:370: in async_get_implementations
    async_register_local_apis(hass)
homeassistant/helpers/config_entry_oauth2_flow.py:349: in async_register_local_apis
    hass.http.register_view(OAuth2AuthorizeCallbackView())
homeassistant/components/http/__init__.py:295: in register_view
    view.register(self.app, self.app.router)
homeassistant/components/http/view.py:95: in register
    routes.append(router.add_route(method, url, handler))
venv/lib/python3.9/site-packages/aiohttp/web_urldispatcher.py:1100: in add_route
    resource = self.add_resource(path, name=name)
venv/lib/python3.9/site-packages/aiohttp/web_urldispatcher.py:1085: in add_resource
    self.register_resource(resource)

...
resource = <PlainResource  /auth/external/callback>

    def register_resource(self, resource: AbstractResource) -> None:
        assert isinstance(
            resource, AbstractResource
        ), f"Instance of AbstractResource class is required, got {resource!r}"
        if self.frozen:
>           raise RuntimeError("Cannot register a resource into frozen router.")
E           RuntimeError: Cannot register a resource into frozen router.

My impression is that we are in fact freezing the router and this registration must be done at startup time, rather than just in time when the first local oauth implementation is registered. Should I interpret your comment to mean the test harness is incorrect?

[allenporter]

The above error happens if i do the API registration lazily, like how the non-provider does it for local oauth implementations. An alternative framing is that the integration just needs to call this to have the APIs exposed at all when using the provider interface, so i removed the comment.

[balloob]

https://github.com/home-assistant/core/blob/dev/homeassistant/components/http/__init__.py#L441-L446

[allenporter]

Well .. that nice "creative" :) solution to avoid freezing does not run during tests. I am not going to dig into this, as moving the registration to the a component seems like the best fix. I'll revert changes to this file in this PR.

[balloob]

Yeah this is one of those ticking time bombs that will one aiohttp update explode in our face. I think that we can workaround it using a resource, like we did for the frontend index, but never dug in to solve it.

[MartinHjelmare]

Great! Can be merged when frontend PR is happy with this.

[allenporter]

Great! Can be merged when frontend PR is happy with this.

Thanks for all the review time on this.

home-assistant/frontend#12344 (comment)
"When the backend gets merged can you set it back to ready for review?"

:) I'll poke and see if frontend is close.

[allenporter]

Frontend PR is good to go.

I updated how importing works to register imported credentials with the specified auth domain, rather than using the unique id auth domain based on the client id. This is needed for compatibility with existing config entries and was found when i was testing with other integrations besides xbox (future PRs). Have another look.