Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(ee): GetKeys should return an error (#7713) #7797

Merged
merged 1 commit into from
May 10, 2021
Merged

fix(ee): GetKeys should return an error (#7713) #7797

merged 1 commit into from
May 10, 2021

Conversation

ajeetdsouza
Copy link
Contributor

@ajeetdsouza ajeetdsouza commented May 9, 2021 •
edited by manishrjain
Loading

Cherry-picks 36139b4:

Currently, GetKeys exits immediately upon error. While this works for most uses, it ends up shutting down the Alpha when a restore request is made with bad input. This PR fixes that.

This change is Reviewable

@ajeetdsouza ajeetdsouza requested a review from jarifibrahim May 9, 2021 21:25
@github-actions github-actions bot added area/bulk-loader Issues related to bulk loading. area/enterprise Related to proprietary features area/live-loader Issues related to live loading. labels May 9, 2021
@ajeetdsouza ajeetdsouza merged commit 7cc134a into release/v21.03-slash May 10, 2021
@ajeetdsouza ajeetdsouza deleted the ajeet/keys branch May 10, 2021 08:40
ajeetdsouza added a commit that referenced this pull request May 27, 2021
@ajeetdsouza
fix(ee): GetKeys should return an error (#7713) (#7797)
c0b4493 
(cherry picked from commit 7cc134a)
mangalaman93 pushed a commit that referenced this pull request Dec 28, 2022
@ajeetdsouza @mangalaman93
fix(ee): GetKeys should return an error (#7713) (#7797)
b08aa50
mangalaman93 pushed a commit that referenced this pull request Dec 29, 2022
@ajeetdsouza @mangalaman93
fix(ee): GetKeys should return an error (#7713) (#7797)
79505f2
mangalaman93 pushed a commit that referenced this pull request Jan 2, 2023
@ahsanbarkati @mangalaman93
cherry-pick PR #7753
1335343 
This commit is a major rewrite of online restore code. It used
to use KVLoader in badger. Now it instead uses StreamWriter
that is much faster for writes in the case of restore.

following commits are cherry-picked (in reverse order):
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix: fixing graphql schema update when the data is restored + skipping /probe/graphql from audit (#7925)
 * Don't ban namespace in export_backup
mangalaman93 pushed a commit that referenced this pull request Jan 2, 2023
@ahsanbarkati @mangalaman93
cherry-pick PR #7753
65d6273 
This commit is a major rewrite of online restore code. It used
to use KVLoader in badger. Now it instead uses StreamWriter
that is much faster for writes in the case of restore.

following commits are cherry-picked (in reverse order):
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix: fixing graphql schema update when the data is restored + skipping /probe/graphql from audit (#7925)
 * Don't ban namespace in export_backup
mangalaman93 pushed a commit that referenced this pull request Jan 2, 2023
@ahsanbarkati @mangalaman93
cherry-pick PR #7753
b8b51d2 
This commit is a major rewrite of online restore code. It used
to use KVLoader in badger. Now it instead uses StreamWriter
that is much faster for writes in the case of restore.

following commits are cherry-picked (in reverse order):
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix: fixing graphql schema update when the data is restored + skipping /probe/graphql from audit (#7925)
 * Don't ban namespace in export_backup
mangalaman93 pushed a commit that referenced this pull request Jan 3, 2023
@ahsanbarkati @mangalaman93
use StreamWriter to write to badger during backup/restore
af7ed17 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * Don't ban namespace in export_backup
 * fix: fixing graphql schema update when the data is restored +
        skipping /probe/graphql from audit (#7925)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Jan 3, 2023
@ahsanbarkati @mangalaman93
fix(backup): use StreamWriter to write to badger during backup/restore
27861d7 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * Don't ban namespace in export_backup
 * fix: fixing graphql schema update when the data is restored +
        skipping /probe/graphql from audit (#7925)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Jan 3, 2023
@ahsanbarkati @mangalaman93
fix(backup): use StreamWriter to write to badger during backup/restore
c51a385 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * Don't ban namespace in export_backup
 * fix: fixing graphql schema update when the data is restored +
        skipping /probe/graphql from audit (#7925)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Jan 4, 2023
@ahsanbarkati @mangalaman93
fix(backup): use StreamWriter to write to badger during backup/restore
0c75dae 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * fix(backup): Fix full backup request (#7932) (#7933)
 * fix: fixing graphql schema update when the data is restored +
 * fix(restore): return nil if there is error (#7899)
        skipping /probe/graphql from audit (#7925)
 * Don't ban namespace in export_backup
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Jan 4, 2023
@ahsanbarkati @mangalaman93
fix(backup): use StreamWriter to write to badger during backup/restore
545677f 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * opt(restore): Sort the buffer before spinning the writeToDisk goroutine (#7984) (#7996)
 * fix(backup): Fix full backup request (#7932) (#7933)
 * fix: fixing graphql schema update when the data is restored +
 * fix(restore): return nil if there is error (#7899)
        skipping /probe/graphql from audit (#7925)
 * Don't ban namespace in export_backup
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Jan 6, 2023
@ahsanbarkati @mangalaman93
fix(backup): use StreamWriter to write to badger during backup/restore
d41203f 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * opt(restore): Sort the buffer before spinning the writeToDisk goroutine (#7984) (#7996)
 * fix(backup): Fix full backup request (#7932) (#7933)
 * fix: fixing graphql schema update when the data is restored +
        skipping /probe/graphql from audit (#7925)
 * fix(restore): return nil if there is error (#7899)
 * Don't ban namespace in export_backup
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Jan 17, 2023
@ahsanbarkati @mangalaman93
fix(backup): use StreamWriter to write to badger during backup/restore
9c59b2b 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * opt(restore): Sort the buffer before spinning the writeToDisk goroutine (#7984) (#7996)
 * fix(backup): Fix full backup request (#7932) (#7933)
 * fix: fixing graphql schema update when the data is restored +
        skipping /probe/graphql from audit (#7925)
 * fix(restore): return nil if there is error (#7899)
 * Don't ban namespace in export_backup
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Jan 17, 2023
@ahsanbarkati @mangalaman93
fix(backup): use StreamWriter to write to badger during backup/restore
2ac2673 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * opt(restore): Sort the buffer before spinning the writeToDisk goroutine (#7984) (#7996)
 * fix(backup): Fix full backup request (#7932) (#7933)
 * fix: fixing graphql schema update when the data is restored +
        skipping /probe/graphql from audit (#7925)
 * fix(restore): return nil if there is error (#7899)
 * Don't ban namespace in export_backup
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Jan 17, 2023
@ahsanbarkati @mangalaman93
fix(backup): use StreamWriter to write to badger during backup/restore
3307aab 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * opt(restore): Sort the buffer before spinning the writeToDisk goroutine (#7984) (#7996)
 * fix(backup): Fix full backup request (#7932) (#7933)
 * fix: fixing graphql schema update when the data is restored +
        skipping /probe/graphql from audit (#7925)
 * fix(restore): return nil if there is error (#7899)
 * Don't ban namespace in export_backup
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Jan 18, 2023
@ahsanbarkati @mangalaman93
fix(backup): use StreamWriter to write to badger during backup/restore
dd950f7 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * opt(restore): Sort the buffer before spinning the writeToDisk goroutine (#7984) (#7996)
 * fix(backup): Fix full backup request (#7932) (#7933)
 * fix: fixing graphql schema update when the data is restored +
        skipping /probe/graphql from audit (#7925)
 * fix(restore): return nil if there is error (#7899)
 * Don't ban namespace in export_backup
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
all-seeing-code pushed a commit that referenced this pull request Jan 23, 2023
@ahsanbarkati @all-seeing-code
fix(backup): use StreamWriter to write to badger during backup/restore
9b20a70 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * opt(restore): Sort the buffer before spinning the writeToDisk goroutine (#7984) (#7996)
 * fix(backup): Fix full backup request (#7932) (#7933)
 * fix: fixing graphql schema update when the data is restored +
 * fix(restore): return nil if there is error (#7899)
        skipping /probe/graphql from audit (#7925)
 * Don't ban namespace in export_backup
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
all-seeing-code pushed a commit that referenced this pull request Jan 23, 2023
@ahsanbarkati @all-seeing-code
fix(backup): use StreamWriter to write to badger during backup/restore
1a6744a 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * opt(restore): Sort the buffer before spinning the writeToDisk goroutine (#7984) (#7996)
 * fix(backup): Fix full backup request (#7932) (#7933)
 * fix: fixing graphql schema update when the data is restored +
        skipping /probe/graphql from audit (#7925)
 * fix(restore): return nil if there is error (#7899)
 * Don't ban namespace in export_backup
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NamanJain8 NamanJain8 NamanJain8 approved these changes

@jarifibrahim jarifibrahim Awaiting requested review from jarifibrahim

@manishrjain manishrjain Awaiting requested review from manishrjain

@vvbalaji-dgraph vvbalaji-dgraph Awaiting requested review from vvbalaji-dgraph

Assignees
No one assigned
Labels
area/bulk-loader Issues related to bulk loading. area/enterprise Related to proprietary features area/live-loader Issues related to live loading.
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ajeetdsouza @NamanJain8