Skip to content

Reduce orm overhead by grouping object expiration#41394

Merged
balloob merged 1 commit intohome-assistant:devfrom
bdraco:expire_overhead
Oct 7, 2020
Merged

Reduce orm overhead by grouping object expiration#41394
balloob merged 1 commit intohome-assistant:devfrom
bdraco:expire_overhead

Conversation

@bdraco
Copy link
Copy Markdown
Member

@bdraco bdraco commented Oct 7, 2020

Proposed change

While #40982 solved the performance overhead of expiring every time
we commit the event session, it caused a regression which was
fixed in #41349.

Ideally we could avoid the overhead of expiring objects on commit
since we are never going to use them again by using expunge after
the commit to remove the objects we no longer need. Unfortunately
this causes sqlalchemy to spend quite a bit of time sorting out
state when adding new objects to the event session after an
expunge_all so it wasn't a viable option.

As not expring the objects causes unexpected side effects, we can
limit the impact of having to expring them by only doing the expire
every 120 commits (by default every 2 minutes) instead of every commit
(by default every second). This works because the expire operation
itself is expensive reguardless of the number of objects it is expiring.

To verify this approach does not leak memory, a scale test integration
was created (https://github.com/bdraco/scaletest) and run for hours
along with an objgraph dump setup every 30 seconds to verify States
and Events were being properly disposed of
(https://mg.pov.lt/objgraph/objgraph.html#locating-and-filtering-objects)

In testing this took a machine running the scaletest integration
from 100% cpu to 4% cpu

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Example entry for configuration.yaml:

# Example configuration.yaml

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request:

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • The code has been formatted using Black (black --fast homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

  • No score or internal
  • 🥈 Silver
  • 🥇 Gold
  • 🏆 Platinum

To help with the load of incoming pull requests:

While home-assistant#40982 solved the performance overhead of expiring every time
we commit the event session, it caused a regression which was
fixed in home-assistant#41349.

Ideally we could avoid the overhead of expiring objects on commit
since we are never going to use them again by using expunge after
the commit to remove the objects we no longer need. Unfortunately
this causes sqlalchemy to spend quite a bit of time sorted out
state when adding new objects to the event session after an
expunge_all so it wasn't a viable option.

As not expring the objects causes unexpected side effects, we can
limit the impact of having to expring them by only doing the expire
every 120 commits (by default every 2 minutes) instead of every commit
(by default every second). This works because the expire operation
itself is expensive reguardless of the number of objects it is expiring.

To verify this approach does not leak memory, a scale test integration
was created (https://github.com/bdraco/scaletest) and run for hours
along with an objgraph dump setup every 30 seconds to verify States
and Events were being properly disposed of
(https://mg.pov.lt/objgraph/objgraph.html#locating-and-filtering-objects)

In testing this took a machine running the scaletest integration
from 100% cpu to 4% cpu
@bdraco bdraco linked an issue Oct 7, 2020 that may be closed by this pull request
@balloob balloob merged commit 113d738 into home-assistant:dev Oct 7, 2020
@bdraco bdraco modified the milestone: 0.116.0 Oct 7, 2020
bdraco added a commit to bdraco/home-assistant that referenced this pull request Oct 7, 2020
bdraco added a commit to bdraco/home-assistant that referenced this pull request Oct 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Higher CPU continues in 0.118

3 participants