Skip to content

Commit

Permalink
Feat(Backup): Add native google cloud storage backup support (#7829)
Browse files Browse the repository at this point in the history
Currently, we use MinIO to write backup files to CGP storage. To do this we need to start a minio server (with GCP credentials) and trigger a backup with the destination as the minio server. This requires the user to run the minio server every time (or always) when a backup needs to be triggered. Using native SDK will help get rid of the extra service.

Google cloud storage (GCS) requires the user to create Service account key and credentials file. When starting an alpha we can set env variable:

GOOGLE_APPLICATION_CREDENTIALS="/path/to/credentials.json" dgraph alpha

to allow access to GCS. To trigger a backup the user can use the admin endpoint to initiate the backup. The destination should be set as:

"gs://<bucket_name>/path/to/backup"
  • Loading branch information
rohanprasad authored May 26, 2021
1 parent 983debd commit 3dfa8df
Show file tree
Hide file tree
Showing 3 changed files with 570 additions and 38 deletions.
26 changes: 16 additions & 10 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ go 1.12
// replace github.com/dgraph-io/roaring => /home/mrjn/go/src/github.com/dgraph-io/roaring

require (
cloud.google.com/go/storage v1.15.0
contrib.go.opencensus.io/exporter/jaeger v0.1.0
contrib.go.opencensus.io/exporter/prometheus v0.1.0
github.com/Azure/azure-storage-blob-go v0.13.0
Expand Down Expand Up @@ -37,11 +38,12 @@ require (
github.com/gogo/protobuf v1.3.2
github.com/golang/geo v0.0.0-20170810003146-31fb0106dc4a
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
github.com/golang/protobuf v1.4.1
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.2
github.com/golang/snappy v0.0.2
github.com/google/codesearch v1.0.0
github.com/google/go-cmp v0.5.4
github.com/google/uuid v1.1.1
github.com/google/go-cmp v0.5.5
github.com/google/uuid v1.1.2
github.com/gorilla/websocket v1.4.2
github.com/graph-gophers/graphql-go v0.0.0-20200309224638-dae41bde9ef9
github.com/hashicorp/vault/api v1.0.4
Expand All @@ -65,15 +67,19 @@ require (
github.com/twpayne/go-geom v1.0.5
github.com/xdg/scram v0.0.0-20180814205039-7eeb5667e42c
go.etcd.io/etcd v0.0.0-20190228193606-a943ad0ee4c9
go.opencensus.io v0.22.5
go.opencensus.io v0.23.0
go.uber.org/zap v1.16.0
golang.org/x/crypto v0.0.0-20200820211705-5c72a883971a
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c
golang.org/x/text v0.3.3
golang.org/x/tools v0.0.0-20210106214847-113979e3529a
google.golang.org/grpc v1.27.0
golang.org/x/lint v0.0.0-20210508222113-6edffad5e616 // indirect
golang.org/x/net v0.0.0-20210510120150-4163338589ed
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c
golang.org/x/sys v0.0.0-20210511113859-b0526f3d8744
golang.org/x/text v0.3.6
golang.org/x/tools v0.1.1
google.golang.org/api v0.46.0
google.golang.org/genproto v0.0.0-20210510173355-fb37daa5cd7a // indirect
google.golang.org/grpc v1.37.1
google.golang.org/grpc/examples v0.0.0-20210518002758-2713b77e8526 // indirect
gopkg.in/DataDog/dd-trace-go.v1 v1.13.1 // indirect
gopkg.in/square/go-jose.v2 v2.3.1
gopkg.in/yaml.v2 v2.2.8
Expand Down
Loading

0 comments on commit 3dfa8df

Please sign in to comment.