Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new explicit migration command #2139

Merged
merged 8 commits into from
Jan 9, 2025
Merged

Add new explicit migration command #2139

merged 8 commits into from
Jan 9, 2025

Conversation

DL6ER
Copy link
Member

@DL6ER DL6ER commented Dec 21, 2024

What does this implement/fix?

pihole-FTL migrate v6 is intentionally not included in the help output as it is solely meant to be used internally by the installer script once from v5 -> v6

Related to pi-hole/pi-hole#5830

Related issue or feature (if applicable): N/A

Pull request in docs with documentation (if applicable): N/A


By submitting this pull request, I confirm the following:

  1. I have read and understood the contributors guide, as well as this entire template. I understand which branch to base my commits and Pull Requests against.
  2. I have commented my proposed changes within the code.
  3. I am willing to help maintain this change if there are issues with it later.
  4. It is compatible with the EUPL 1.2 license
  5. I have squashed any insignificant commits. (git rebase)

Checklist:

  • The code change is tested and works locally.
  • I based my code and PRs against the repositories developmental branch.
  • I signed off all commits. Pi-hole enforces the DCO for all contributions
  • I signed all my commits. Pi-hole requires signatures to verify authorship
  • I have read the above and my PR is ready for review.

@pralor-bot
Copy link

This pull request has been mentioned on Pi-hole Userspace. There might be relevant details there:

https://discourse.pi-hole.net/t/broken-installation/74638/25

yubiuser
yubiuser previously approved these changes Dec 21, 2024
Copy link
Member

@yubiuser yubiuser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I can see no functional changes but only putting code into new functions

@DL6ER
Copy link
Member Author

DL6ER commented Dec 23, 2024

Tested on two virtual machines to do what it should. The latest commit ensures we log the migration process to the user so they can quickly review if everything went the way they'd expect it to.

@DL6ER DL6ER marked this pull request as ready for review December 23, 2024 08:00
@DL6ER DL6ER requested a review from yubiuser December 23, 2024 08:00
DL6ER added 2 commits January 3, 2025 22:04
…d in the help output as it is solely meant to be used internally by the installer script once from v5 -> v6

Signed-off-by: DL6ER <[email protected]>
@DL6ER
Copy link
Member Author

DL6ER commented Jan 3, 2025

The most recent commit adds code to ensure a broken (or too new) database is not used when we know that all the SQL statements against it will fail anyway

@DL6ER
Copy link
Member Author

DL6ER commented Jan 3, 2025

Rebased on latest development

…is not available (e.g. broken or more recent than this particular FTL binary can handle (like checking out a very old branch))

Signed-off-by: DL6ER <[email protected]>
yubiuser
yubiuser previously approved these changes Jan 8, 2025
Copy link
Member

@yubiuser yubiuser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I was too fast. Importing v5 backups results in SQL database locked errors. It does not happen every time, so maybe it's a race collission

2025-01-08 12:12:15.997 CET [147804/T148076] INFO: import_json_table(client_by_group.json): JSON array contains 0 entries
2025-01-08 12:12:16.017 CET [147804/T148076] INFO: import_json_table(domainlist_by_group.json): JSON array contains 64 entries
2025-01-08 12:12:16.036 CET [147804/T148076] INFO: import_json_table(group.json): JSON array contains 3 entries
2025-01-08 12:12:16.050 CET [147804/T147886] ERROR: SQLite3: database is locked in "SELECT value FROM info WHERE property = 'updated';" (5)
2025-01-08 12:12:16.058 CET [147804/T148076] INFO: import_json_table(whitelist.exact.json): JSON array contains 26 entries
2025-01-08 12:12:16.083 CET [147804/T148076] INFO: import_json_table(whitelist.regex.json): JSON array contains 3 entries
2025-01-08 12:13:43.029 CET [147804/T148365] INFO: import_json_table(blacklist.regex.json): JSON array contains 27 entries
2025-01-08 12:13:43.048 CET [147804/T148365] INFO: import_json_table(client.json): JSON array contains 0 entries
2025-01-08 12:13:43.069 CET [147804/T148365] INFO: import_json_table(client_by_group.json): JSON array contains 0 entries
2025-01-08 12:13:43.082 CET [147804/T148365] ERROR: Unable to commit transaction: database is locked
2025-01-08 12:13:43.088 CET [147804/T148365] INFO: import_json_table(domainlist_by_group.json): JSON array contains 64 entries
2025-01-08 12:13:43.107 CET [147804/T148365] INFO: import_json_table(group.json): JSON array contains 3 entries
2025-01-08 12:13:43.129 CET [147804/T148365] INFO: import_json_table(whitelist.exact.json): JSON array contains 26 entries
2025-01-08 12:13:43.153 CET [147804/T148365] INFO: import_json_table(whitelist.regex.json): JSON array contains 3 entries

DL6ER added 3 commits January 8, 2025 19:32
…st-updated timestamp from the gravity database

Signed-off-by: DL6ER <[email protected]>
…ON tables to the gravity database

Signed-off-by: DL6ER <[email protected]>
@DL6ER
Copy link
Member Author

DL6ER commented Jan 8, 2025

Yes, you are right with your assumption. You have found two of the three possible cases I was able to find in the code. Three commits pushed.

Copy link
Member

@yubiuser yubiuser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested it several times, no SQL errors occured.

@DL6ER DL6ER merged commit f2c82da into development Jan 9, 2025
19 checks passed
@DL6ER DL6ER deleted the fix/migration branch January 9, 2025 17:13
@PromoFaux PromoFaux mentioned this pull request Feb 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants