Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(bigtable): Hot backups #11215

Merged
merged 9 commits into from
Jan 8, 2025
Merged

Conversation

bhshkh
Copy link
Contributor

@bhshkh bhshkh commented Dec 3, 2024

Fixes: b/365774730

More about hot backups: https://cloud.google.com/bigtable/docs/backups#hot-backups

Generated protos have 2 new fields in the Backup struct. These fields are being added to various methods in this PR:

// Indicates the backup type of the backup.
BackupType Backup_BackupType `protobuf:"varint,11,opt,name=backup_type,json=backupType,proto3,enum=google.bigtable.admin.v2.Backup_BackupType" json:"backup_type,omitempty"`
// The time at which the hot backup will be converted to a standard backup.
// Once the `hot_to_standard_time` has passed, Cloud Bigtable will convert the
// hot backup to a standard backup. This value must be greater than the backup
// creation time by:
// - At least 24 hours
//
// This field only applies for hot backups. When creating or updating a
// standard backup, attempting to set this field will fail the request.
HotToStandardTime *timestamppb.Timestamp `protobuf:"bytes,12,opt,name=hot_to_standard_time,json=hotToStandardTime,proto3" json:"hot_to_standard_time,omitempty"`


To create a hot backup that expires after 4 days:

hotToStdTime := 
adminClient.CreateBackupWithOptions(ctx, "TableID", "Cluster", "BkpName",
		time.Now().Add(24 * 4 * time.Hour), HotToStandardBackup(time.Now().Add(2 * 24 * time.Hour)))

or without specifying HotToStandardTime i.e. hot backup does not get converted to standard backup

hotToStdTime := time.Now()
adminClient.CreateBackupWithOptions(ctx, "TableID", "Cluster", "BkpName",
		time.Now().Add(24 * 4 * time.Hour), HotBackup())

To update hot_to_standard_time of hot backup:

adminClient.UpdateBackupHotToStandardTime(ctx, "Cluster", "BkpName", &hotToStdTime)

To remove hot_to_standard_time of hot backup:

adminClient.UpdateBackupHotToStandardTime(ctx, "Cluster", "BkpName", nil)

@bhshkh bhshkh requested review from a team as code owners December 3, 2024 20:22
@bhshkh bhshkh changed the title feat(bigtable): hot backups feat(bigtable): Hot backups Dec 3, 2024
@product-auto-label product-auto-label bot added the api: bigtable Issues related to the Bigtable API. label Dec 4, 2024
bigtable/admin.go Outdated Show resolved Hide resolved
bigtable/admin.go Outdated Show resolved Hide resolved
Copy link

@DerekLeeCS DerekLeeCS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall logic looks good, just have a few suggestions for the implementation

bigtable/admin.go Show resolved Hide resolved
bigtable/admin.go Outdated Show resolved Hide resolved
bigtable/admin.go Outdated Show resolved Hide resolved
@bhshkh bhshkh enabled auto-merge (squash) December 10, 2024 20:52
}

// BackupOption can be used to specify parameters for backup operations.
type BackupOption interface {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think my main question here for the option style here is how complex you expect the options to become. I see only a single option with two behavior fields, so struct-based options that satisfy the apply() contract may be more than you need.

By way of comparison, if the contract is simplified to the core backupOptions and some funcs that can modify it:

type BackupOption func(*backupOptions)

func WithHotBackupOption(t time.Time) BackupOption {
  return func(bo *backupOptions) {
     bo.backupType = blah
     bo.hotToStandardTime = blah
  }
}

func WithExpiration(t time.Time) BackupOption {
  return func(bo *backupOptions) {
     bo.expireTime = blah
  }
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review. I have updated the code.

bigtable/admin.go Show resolved Hide resolved
@bhshkh bhshkh requested a review from shollyman December 25, 2024 23:08
@bhshkh
Copy link
Contributor Author

bhshkh commented Jan 5, 2025

@DerekLeeCS , please review. I have updated the code as per review comments

@bhshkh
Copy link
Contributor Author

bhshkh commented Jan 6, 2025

@shollyman @gkevinzheng, please review. I have updated the code as per review comments

Copy link

@DerekLeeCS DerekLeeCS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the changes!

bigtable/admin.go Outdated Show resolved Hide resolved
bigtable/integration_test.go Outdated Show resolved Hide resolved
@bhshkh bhshkh merged commit 238ac1c into googleapis:main Jan 8, 2025
8 of 10 checks passed
@bhshkh bhshkh deleted the feature/cbt-hot-backup branch January 8, 2025 05:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigtable Issues related to the Bigtable API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants