Provide api to see which integrations are being loaded

[bdraco]

Needs #48269 since device trackers are not being added to the components

Proposed change

This is much easier to review without whitespace changes due to the context manager addition

Goals

• Show the integration that is taking the most time to startup during startup
• Show how longer each integration took in total on the config info page to call out slow or misbehaving integrations so Home Assistant doesn't take the blame for slow ones (esp custom ones)

Frontend home-assistant/frontend#8720

2021-03-26 08:59:01 INFO (MainThread) [homeassistant.bootstrap] Integration setup times: {'logger': 0.001837, 'onboarding': 0.002468, 'search': 0.002993, 'lock': 0.004147, 'webhook': 0.004786, 'frontend': 0.005408, 'alexa': 0.00748, 'stream': 0.009616, 'http': 0.011046, 'image': 0.012741, 'cover': 0.0138, 'system_log': 0.017852, 'device_automation': 0.019021, 'config': 0.044177, 'sensor': 0.045179, 'fan': 0.051967, 'cloud': 0.053339, 'my': 0.054587, 'recorder': 0.055922, 'vacuum': 0.080255, 'api': 0.081303, 'auth': 0.088869, 'light': 0.091841, 'default_config': 0.093249, 'rest_command': 0.100522, 'lovelace': 0.103624, 'dhcp': 0.105445, 'websocket_api': 0.110111, 'script': 0.113537, 'emulated_hue': 0.120521, 'binary_sensor': 0.145037, 'weather': 0.197711, 'media_player': 0.204634, 'switch': 0.241131, 'cast': 0.313861, 'person': 0.341786, 'homekit': 0.344398, 'ssdp': 0.444936, 'homekit_controller': 0.670873, 'doorbird': 0.714473, 'climate': 0.740771, 'history': 0.783979, 'blueprint': 0.806934, 'media_source': 0.869515, 'sun': 0.876625, 'shell_command': 0.880883, 'threading': 0.881798, 'remote': 0.885051, 'map': 0.913215, 'trace': 0.937068, 'autelis': 0.937501, 'filter': 0.958175, 'enphase_envoy': 0.961878, 'somfy': 0.96544, 'automation': 0.967931, 'homeassistant': 1.117395, 'sonos': 1.231818, 'systemmonitor': 1.297367, 'zeroconf': 1.421767, 'logbook': 1.541961, 'system_health': 1.574957, 'flume': 1.658237, 'scene': 1.676498, 'profiler': 1.678715, 'tag': 1.729499, 'upnp': 1.809989, 'heos': 1.91246, 'unifi': 1.951867, 'timer': 1.978993, 'camera': 2.00041, 'ffmpeg': 2.001187, 'denonavr': 2.070011, 'mobile_app': 2.25135, 'template': 2.261124, 'nut': 2.53838, 'apple_tv': 2.602957, 'counter': 2.717594, 'input_datetime': 2.730172, 'tts': 2.778089, 'updater': 2.853858, 'openweathermap': 3.065485, 'samsungtv': 3.203091, 'group': 3.27649, 'harmony': 3.696712, 'rachio': 4.201295, 'ipp': 4.290456, 'roku': 4.433694, 'input_text': 4.929488, 'synology_dsm': 4.958669, 'zone': 4.987181, 'input_number': 4.996274, 'bond': 5.327438, 'input_boolean': 5.853331, 'screenlogic': 5.921358, 'notify': 5.929316, 'lutron_caseta': 6.010032, 'input_select': 6.024482, 'rest': 6.607305, 'sense': 6.637527, 'smartir': 6.648064, 'august': 6.659398, 'faa_delays': 6.986708, 'icloud3': 7.141761, 'device_tracker': 7.340055, 'unifiprotect': 7.423218, 'roomba': 7.626561, 'senseme': 7.818574, 'hacs': 8.534149, 'tesla': 9.586424, 'emonitor': 9.692194, 'isy994': 11.400433, 'ping': 15.259365, 'esphome': 21.798107, 'broadlink': 32.379421}

This has already helped identify some integrations that could be sped up:

• ping: Prevent ping integration from delaying startup #43869
• senseme: Use synchronization primitives to avoid sleeping during first update bdraco/aiosenseme#10
• do not load disabled/ignored integrations: Do not load ignored or disabled integrations at startup #48355
• small optimizations of adding entities: Small speed up to adding entities #48353
• group creating storage collection entities: Optimize storage collection entity operations with asyncio.gather #48352
• broadlink storage loads: Lazy load broadlink storage #48391
• bond setup concurrency Speed up bond setup with gather #48454

Type of change

• Dependency upgrade
• Bugfix (non-breaking change which fixes an issue)
• New integration (thank you!)
• New feature (which adds functionality to an existing integration)
• Breaking change (fix/feature causing existing functionality to break)
• Code quality improvements to existing code or addition of tests

Example entry for configuration.yaml:

# Example configuration.yaml

Additional information

• This PR fixes or closes issue: fixes #
• This PR is related to issue:
• Link to documentation pull request:

Checklist

• The code change is tested and works locally.
• Local tests pass. Your PR cannot be merged unless tests pass
• There is no commented out code in this PR.
• I have followed the development checklist
• The code has been formatted using Black (black --fast homeassistant tests)
• Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

• Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

• No score or internal
• 🥈 Silver
• 🥇 Gold
• 🏆 Platinum

To help with the load of incoming pull requests:

• I have reviewed two other open pull requests in this repository.

[bdraco]

I think I should make DATA_SETUP_STARTED a context manager so we can add it more places to increase what can be tracked.

With async_start_setup(hass, names):

Pushes into DATA_SETUP_STARTED, pop on finish

On finish into DATA_SETUP_COMPLETED so we can see on config info page

Places we need it:

• Setup integration -- DATA_SETUP_STARTED already populates integration
• Entity platform (platform.integration)
• device tracker legacy (device_tracker.integration)
• notify platforms (notify.integration)

[bdraco]

2021-03-24 19:57:35 DEBUG (MainThread) [homeassistant.bootstrap] Setup times: {'logger': datetime.timedelta(microseconds=1423), 'http': datetime.timedelta(microseconds=17148), 'system_log': datetime.timedelta(microseconds=1087), 'recorder': datetime.timedelta(microseconds=83353), 'webhook': datetime.timedelta(microseconds=75709), 'websocket_api': datetime.timedelta(microseconds=74605), 'auth': datetime.timedelta(microseconds=63292), 'device_automation': datetime.timedelta(microseconds=61863), 'api': datetime.timedelta(microseconds=45899), 'alexa': datetime.timedelta(microseconds=3386), 'search': datetime.timedelta(microseconds=15968), 'config': datetime.timedelta(microseconds=38842), 'lovelace': datetime.timedelta(microseconds=86972), 'image': datetime.timedelta(microseconds=7629), 'person': datetime.timedelta(microseconds=313228), 'cloud': datetime.timedelta(microseconds=49667), 'onboarding': datetime.timedelta(microseconds=2139), 'frontend': datetime.timedelta(microseconds=4019), 'blueprint': datetime.timedelta(microseconds=819283), 'shell_command': datetime.timedelta(microseconds=818958), 'trace': datetime.timedelta(microseconds=817333), 'updater': datetime.timedelta(seconds=2, microseconds=709575), 'remote': datetime.timedelta(microseconds=793298), 'rest_command': datetime.timedelta(microseconds=777519), 'my': datetime.timedelta(microseconds=775199), 'threading': datetime.timedelta(microseconds=761732), 'autelis': datetime.timedelta(microseconds=759321), 'climate': datetime.timedelta(microseconds=747819), 'sun': datetime.timedelta(microseconds=743480), 'map': datetime.timedelta(microseconds=711164), 'history': datetime.timedelta(microseconds=654846), 'dhcp': datetime.timedelta(microseconds=44424), 'binary_sensor': datetime.timedelta(microseconds=47389), 'homeassistant': datetime.timedelta(seconds=1, microseconds=239579), 'emulated_hue': datetime.timedelta(microseconds=575971), 'system_health': datetime.timedelta(seconds=1, microseconds=455119), 'logbook': datetime.timedelta(seconds=1, microseconds=395573), 'profiler': datetime.timedelta(seconds=1, microseconds=322999), 'media_source': datetime.timedelta(seconds=1, microseconds=320622), 'scene': datetime.timedelta(seconds=1, microseconds=300893), 'input_datetime': datetime.timedelta(seconds=1, microseconds=585562), 'camera': datetime.timedelta(seconds=1, microseconds=537392), 'timer': datetime.timedelta(seconds=1, microseconds=529638), 'ffmpeg': datetime.timedelta(seconds=1, microseconds=518741), 'counter': datetime.timedelta(seconds=1, microseconds=498190), 'tag': datetime.timedelta(seconds=1, microseconds=109656), 'zeroconf': datetime.timedelta(seconds=1, microseconds=348315), 'switch': datetime.timedelta(microseconds=465537), 'flume': datetime.timedelta(seconds=2, microseconds=206408), 'openweathermap': datetime.timedelta(seconds=4, microseconds=26078), 'apple_tv': datetime.timedelta(seconds=3, microseconds=81848), 'somfy': datetime.timedelta(seconds=1, microseconds=33030), 'group': datetime.timedelta(seconds=2, microseconds=981550), 'ping': datetime.timedelta(seconds=32, microseconds=226043), 'ssdp': datetime.timedelta(microseconds=403898), 'zone': datetime.timedelta(seconds=3, microseconds=633598), 'media_player': datetime.timedelta(microseconds=501019), 'nut': datetime.timedelta(seconds=2, microseconds=71835), 'sonos': datetime.timedelta(seconds=1, microseconds=558537), 'samsungtv': datetime.timedelta(seconds=4, microseconds=222867), 'weather': datetime.timedelta(microseconds=34404), 'input_number': datetime.timedelta(seconds=4, microseconds=81606), 'input_text': datetime.timedelta(seconds=4, microseconds=40730), 'input_boolean': datetime.timedelta(seconds=4, microseconds=9207), 'upnp': datetime.timedelta(seconds=1, microseconds=641862), 'screenlogic': datetime.timedelta(seconds=5, microseconds=448981), 'doorbird': datetime.timedelta(microseconds=664446), 'tts': datetime.timedelta(seconds=2, microseconds=736614), 'roomba': datetime.timedelta(seconds=6, microseconds=832777), 'sensor': datetime.timedelta(microseconds=39270), 'denonavr': datetime.timedelta(seconds=1, microseconds=671809), 'vacuum': datetime.timedelta(microseconds=37323), 'template': datetime.timedelta(seconds=1, microseconds=295748), 'unifi': datetime.timedelta(seconds=1, microseconds=982090), 'homekit': datetime.timedelta(microseconds=376393), 'esphome': datetime.timedelta(seconds=7, microseconds=742603), 'senseme': datetime.timedelta(seconds=8, microseconds=999276), 'smartir': datetime.timedelta(seconds=7, microseconds=200950), 'systemmonitor': datetime.timedelta(microseconds=872095), 'fan': datetime.timedelta(microseconds=33225), 'rachio': datetime.timedelta(seconds=3, microseconds=238924), 'homekit_controller': datetime.timedelta(microseconds=721161), 'heos': datetime.timedelta(seconds=2, microseconds=51075), 'automation': datetime.timedelta(seconds=1, microseconds=23583), 'light': datetime.timedelta(microseconds=341107), 'input_select': datetime.timedelta(seconds=4, microseconds=113446), 'cover': datetime.timedelta(microseconds=51583), 'faa_delays': datetime.timedelta(seconds=4, microseconds=889299), 'filter': datetime.timedelta(microseconds=597656), 'enphase_envoy': datetime.timedelta(microseconds=604108), 'script': datetime.timedelta(microseconds=50113), 'rest': datetime.timedelta(seconds=4, microseconds=648672), 'cast': datetime.timedelta(microseconds=266041), 'roku': datetime.timedelta(seconds=2, microseconds=426973), 'notify': datetime.timedelta(seconds=5, microseconds=46318), 'mobile_app': datetime.timedelta(seconds=1, microseconds=157026), 'stream': datetime.timedelta(microseconds=11007), 'default_config': datetime.timedelta(microseconds=71706), 'bond': datetime.timedelta(seconds=3, microseconds=772802), 'unifiprotect': datetime.timedelta(seconds=5, microseconds=950214), 'broadlink': datetime.timedelta(seconds=6, microseconds=301112), 'ipp': datetime.timedelta(seconds=4, microseconds=118290), 'august': datetime.timedelta(seconds=5, microseconds=115461), 'lutron_caseta': datetime.timedelta(seconds=4, microseconds=469076), 'lock': datetime.timedelta(microseconds=3285), 'harmony': datetime.timedelta(seconds=2, microseconds=752673), 'sense': datetime.timedelta(seconds=5, microseconds=612323), 'synology_dsm': datetime.timedelta(seconds=4, microseconds=269406), 'tesla': datetime.timedelta(seconds=6, microseconds=867953), 'hacs': datetime.timedelta(seconds=6, microseconds=562388), 'icloud3': datetime.timedelta(seconds=6, microseconds=528373), 'isy994': datetime.timedelta(seconds=11, microseconds=701235), 'device_tracker': datetime.timedelta(seconds=13, microseconds=54139)}

[bdraco]

rebase to fix conflict only

[MartinHjelmare]

Looks good!

Would be good with some more eyes and thoughts.

[bdraco]

Thanks. Retested 👍

Frontend PR adjusted to the new name.