Skip to content

Add backblaze b2 backup integration#149627

Merged
emontnemery merged 72 commits into
home-assistant:devfrom
ElCruncharino:integration-backblaze-b2
Oct 30, 2025
Merged

Add backblaze b2 backup integration#149627
emontnemery merged 72 commits into
home-assistant:devfrom
ElCruncharino:integration-backblaze-b2

Conversation

@ElCruncharino
Copy link
Copy Markdown
Contributor

@ElCruncharino ElCruncharino commented Jul 29, 2025

Proposed change

Adds a new core integration for Backblaze B2. Can be used as a backup source for HA Backups.

This PR builds upon the foundational work in #145508 by @hugo-vrijswijk and incorporates feedback received on that PR.

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

As @hugo-vrijswijk mentioned in their PR, b2sdk is completely sync, so that needed some refactoring calls wrapped in async_add_executor_job.

Primary changes (based on ongoing discussions in that PR):

  • Separate Metadata Files: Backups now consist of two files in Backblaze B2: the main backup archive (.tar) and a separate JSON metadata file (.metadata.json).
  • Optimized Uploads: Implemented distinct upload strategies for small (direct upload) and large (multipart upload via temporary file) backups, enhancing efficiency and stability. I wanted to avoid keeping uploads in memory.
  • Caching: Introduced caching for the list of backups and file information, mostly to reduce API calls to Backblaze B2 and improving responsiveness in instances with lots of backups.
  • Backup Lifecycle Management: Improved logic for listing, retrieving, and deleting backups by leveraging the new metadata file structure.
  • Stream Handling: Added more explicit closing of file and network streams during downloads and uploads to prevent resource leaks.

Tests were overhauled to get full coverage, but I'm relatively new to writing tests for these types of integrations, so that was a lot of trial and error.

I've also tested locally in the UI with large backups, small backups, and so on with a real B2 bucket, and concluded there were no major functional or performance issues with the code.

Much like @hugo-vrijswijk, this is my first time contributing to HA, so if

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Ruff (ruff format homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.

To help with the load of incoming pull requests:

Copy link
Copy Markdown
Contributor

@home-assistant home-assistant Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ElCruncharino

It seems you haven't yet signed a CLA. Please do so here.

Once you do that we will be able to review and accept this pull request.

Thanks!

@home-assistant
Copy link
Copy Markdown
Contributor

Please take a look at the requested changes, and use the Ready for review button when you are done, thanks 👍

Learn more about our pull request process.

Copy link
Copy Markdown
Contributor

@emontnemery emontnemery left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments. I want to understand how the file iteration is supposed to work before I continue the review.

Comment thread homeassistant/components/backblaze/__init__.py Outdated
Comment thread homeassistant/components/backblaze/__init__.py
Comment thread homeassistant/components/backblaze/repairs.py Outdated
Comment thread homeassistant/components/backblaze/strings.json Outdated
Comment thread homeassistant/components/backblaze/backup.py Outdated
Comment thread homeassistant/components/backblaze/backup.py Outdated
Comment thread homeassistant/components/backblaze/backup.py Outdated
Comment thread homeassistant/components/backblaze/backup.py Outdated
Comment thread homeassistant/components/backblaze/backup.py Outdated
Comment thread homeassistant/components/backblaze/backup.py Outdated
@ElCruncharino
Copy link
Copy Markdown
Contributor Author

@emontnemery Continuing some testing on my side. Will make each conversation as resolved when I have my final implementation in the branch for your review

@emontnemery emontnemery changed the title Integration backblaze b2 Add backblaze b2 backup integrawtion Sep 17, 2025
@emontnemery
Copy link
Copy Markdown
Contributor

@ElCruncharino if you want to discuss something, feel free to message me on Discord

@emontnemery emontnemery changed the title Add backblaze b2 backup integrawtion Add backblaze b2 backup integration Sep 17, 2025
Copy link
Copy Markdown
Contributor

@emontnemery emontnemery left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some follow-up questions and comments.

Comment thread homeassistant/components/backblaze/__init__.py Outdated
Comment thread homeassistant/components/backblaze/__init__.py Outdated
Comment thread homeassistant/components/backblaze/strings.json Outdated
Comment thread homeassistant/components/backblaze/manifest.json Outdated
Comment thread homeassistant/components/backblaze/backup.py Outdated
Comment thread homeassistant/components/backblaze/backup.py Outdated
Comment thread homeassistant/components/backblaze/repairs.py Outdated
Comment thread homeassistant/components/backblaze/__init__.py Outdated
@emontnemery
Copy link
Copy Markdown
Contributor

@ElCruncharino please confirm the domain you want to use for this integration, then mark the PR ready for review.

@emontnemery emontnemery marked this pull request as draft September 24, 2025 09:28
@ElCruncharino
Copy link
Copy Markdown
Contributor Author

@ElCruncharino please confirm the domain you want to use for this integration, then mark the PR ready for review.

Done! "backblaze_b2", for reasons stated in the resolved conversation.

@ElCruncharino ElCruncharino marked this pull request as ready for review September 25, 2025 01:44
@jeroenleenarts
Copy link
Copy Markdown

Tried finding another way to reach out.

Just wanted to say I am impressed by your perseverance in bringing this integration to completion.

@ElCruncharino Thank you. Looking forward to trying your work once completed.

ElCruncharino and others added 19 commits October 28, 2025 21:27
Co-authored-by: Erik Montnemery <erik@montnemery.com>
Co-authored-by: Erik Montnemery <erik@montnemery.com>
Co-authored-by: Erik Montnemery <erik@montnemery.com>
Co-authored-by: Erik Montnemery <erik@montnemery.com>
reduce test file from 808 to 649 lines, add agent fixture to eliminate repetitive setup, consolidate similar error tests using parametrize, simplify test patterns and remove line number references, maintain 100% coverage with all linting checks passing
Co-authored-by: Erik Montnemery <erik@montnemery.com>
Co-authored-by: Erik Montnemery <erik@montnemery.com>
… backup integration

Co-authored-by: Erik Montnemery <erik@montnemery.com>
AsyncIteratorReader was moved from backup.util to util.async_iterator
in PR home-assistant#153194. Add fallback import to support both locations.
The signature changed from (hass, stream) to (loop, stream) in PR home-assistant#153194.
Add runtime check to use the correct signature based on which version is available.
- Made magic number 5 a constant (MAX_CONCURRENT_DOWNLOADS)
- Added test_requests_pool_maxsize_assumption() to verify urllib assumptions
- Parametrized test_semaphore_limits_concurrent_metadata_downloads to test both proper limiting and detection of pool exhaustion
- Updated docstrings for 6 tests to explain why they need direct agent access
Copy link
Copy Markdown
Contributor

@home-assistant home-assistant Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @fp79655

It seems you haven't yet signed a CLA. Please do so here.

Once you do that we will be able to review and accept this pull request.

Thanks!

Comment thread homeassistant/components/backblaze_b2/__init__.py Outdated
Comment thread homeassistant/components/backblaze_b2/backup.py Outdated
Comment thread homeassistant/components/backblaze_b2/__init__.py Outdated
Comment thread homeassistant/components/backblaze_b2/backup.py Outdated
Comment thread homeassistant/components/backblaze_b2/backup.py Outdated
Comment thread homeassistant/components/backblaze_b2/backup.py Outdated
Comment thread homeassistant/components/backblaze_b2/strings.json Outdated
Copy link
Copy Markdown
Member

@MartinHjelmare MartinHjelmare left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

Copy link
Copy Markdown
Contributor

@emontnemery emontnemery left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice @ElCruncharino 👍

@jeroenleenarts
Copy link
Copy Markdown

How can I tell which Home Assistant release this will be a part of?

And great job @ElCruncharino :)

@ehendrix23
Copy link
Copy Markdown
Contributor

How can I tell which Home Assistant release this will be a part of?

And great job @ElCruncharino :)

Concur, excellent job @ElCruncharino and thank you.

@jeroenleenarts, as 1st beta of 2025.11 just went out I would expect this to become part of 2025.12.

@emontnemery
Copy link
Copy Markdown
Contributor

Yes, this will be part of HA Core 2025.12

@joostlek
Copy link
Copy Markdown
Member

@hugo-vrijswijk and @ElCruncharino Can you send me a message on Discord?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants