-
Notifications
You must be signed in to change notification settings - Fork 290
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
POC: fsck repair #345
POC: fsck repair #345
Conversation
Can one of the admins verify this patch?
|
What if we stored the remote we retrieved a commit from as a user xattr, and when we found a missing object, we look at the commits via which it's reachable and use those by default for re-pulls? |
I've been thinking about storing things like information about the TLS exchange as user xattrs too, so we can say in |
The idea is sounds alright to me, but this information might get stale (because remote was removed/renamed), so it might be a good idea to still be able to provide a flag like Also, are xattrs also checksummed? I guess they are, but only for file objects, so we should be fine here, because it won't alter commit's checksum, right? Also, what about corrupted commit object? Can we trust user xattrs in that case? Another thing needed for this PR would be exposing an API for fetching and writing a single object of a given type and checksum, something like:
That way, libsoup usage would be hidden again in the library and there wouldn't be any need for exposing a function like |
How about a new variant of the pull API which supported fetching multiple non-commit objects? Or just even extend the current API to support appending e.g. |
What I've hit in the past is But I'd like to know more about your situation - what case are you trying to fix? Does your use case e.g. include filesystem-level corruption? |
I agree though ultimately we'd need the ability to fetch from arbitrary (or all) remotes like your current code has, not just to handle the user removing them, but also because we should support current systems without xattr metadata. |
Something like "objects" option to
Ah, actually I'm not sure what was the source of the corruption… I'm running EndlessOS on a VM and there was a script changing the environment for development purposes, it breaks out of ostree chroot after a reboot IIUC. I also did some modifications in the system, but that was outside ostree. In the end, it was just 12 corrupted objects from 120k+. So I'd say that my use-case was "just do something to fix the thing, so I can make a copy of the ostree repository to a pendrive".
Right, that too. |
If this is for development, have you looked at |
Ah, no. I simply followed the instructions that turned the Endless OS VM into a developer machine. |
Probably the best is to use |
☔ The latest upstream changes (presumably 23049bb) made this pull request unmergeable. Please resolve the merge conflicts. |
Can one of the admins verify this patch?
|
If anyone is in this situation and wants to resync the repo, here's an example of how to do it manually. Let's delete the object for
|
I think a prereq here is for |
Any update on this? Is there anyway I can workaround this for now in some automatic way? |
So, i looked at this a bit and it is tricky to do with the current public ostree_repo_traverse_commit () code, because it assumes that the reachable hash table is a GVariant to GVariant map, but we need to add some back-pointer to it so that we can track which commits an object came from. But, besides that, the whole fsck operation seems to depend on the ostree_repo_traverse* methods successfully traversing all reachable objects. If one of the dirmeta objects are broken or missing, then ostree_repo_traverse_commit_union() will fail and we will never ever start fsck:ing the objects. |
This means we can later use various operations to heal the repository because ostree does not assume all objects are there. This the begining of a fix for ostreedev#345
This means we can later use various operations to heal the repository because ostree does not assume all objects are there. This the begining of a fix for ostreedev#345
add depcheck() to verify RPMs installed
@krnowak: PR needs rebase. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Not going to pick this up anytime soon, so closing it. |
This is something I quickly hacked up, since I haven't seen any possibility to tell ostree to fetch the missing objects after doing
ostree fsck --delete
.For now it only tries to download content objects, as doing that with metadata objects seems to be a bit more involved. And in my case all the corrupted objects were actually content objects.
WDYT?