Skip to content
This repository was archived by the owner on Aug 2, 2021. It is now read-only.

storage/localstore: fix for pinned chunks that are GC under certain conditions#2190

Closed
santicomp2014 wants to merge 1 commit intomasterfrom
local-pinning-fix
Closed

storage/localstore: fix for pinned chunks that are GC under certain conditions#2190
santicomp2014 wants to merge 1 commit intomasterfrom
local-pinning-fix

Conversation

@santicomp2014
Copy link
Copy Markdown
Contributor

@santicomp2014 santicomp2014 commented May 22, 2020

This PR is a fix, for pinned chunks that are being garbage collected when multiple GC is run.

This can be verified by

  • uploading a chunk/pinning it
  • downloading it (works ok)
  • upload random chunks multiple times the store capacity to flush all chunks with GC
  • try to download the chunk it can fail if the GC is run multiple times

This should be tested by pinning once the content, and also by pinning multiple times the same chunk and repeating the test.

After the first run of the Garbage collection, the pinned chunk is removed from gcExcludeIndex
So the next time the GC is run it will be deleted from the chunks.

I moved this delete from gcExcludeIndex from removeChunksInExcludeIndexFromGC (gc.go)
to setUnpin (mode_set.go)

Another issue was the metric localstore/gc/exclude/excluded-count this was only increasing, also decreased it in the setUnpin

closes #2195

@mortelli
Copy link
Copy Markdown
Contributor

awaiting comment from @jmozah before properly reviewing this

@mortelli mortelli added the global pinning experimental implementation of global pinning label May 26, 2020
@mortelli mortelli mentioned this pull request May 26, 2020
4 tasks
@mortelli
Copy link
Copy Markdown
Contributor

although not strictly a global-pinning issue, i have added the label since it impacts its implementation.

(keep in mind this merges to master)

@santicomp2014
Copy link
Copy Markdown
Contributor Author

santicomp2014 commented Jun 4, 2020

This PR was closed due to feedback with @jmozah
A new PR (#2200) is created with correct fix for local pinning bug.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

bug global pinning experimental implementation of global pinning ready for review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pinned content disappears when gc is triggered multiple times

2 participants