Skip to content

HDDS-8189. [Snapshot] renamedKeyTable should only track keys in buckets that has at least one active snapshot. #4436

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
aswinshakil merged 7 commits into from
Mar 22, 2023
Merged

HDDS-8189. [Snapshot] renamedKeyTable should only track keys in buckets that has at least one active snapshot. #4436

aswinshakil merged 7 commits into from
Mar 22, 2023

Conversation

@aswinshakil
Copy link
Member

@aswinshakil aswinshakil commented Mar 20, 2023

What changes were proposed in this pull request?

To track the renames of keys for only those buckets that are snapshotted.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-8189

How was this patch tested?

Unit tests and existing tests.

@aswinshakil aswinshakil added the snapshot https://issues.apache.org/jira/browse/HDDS-6517 label Mar 20, 2023
@aswinshakil aswinshakil self-assigned this Mar 20, 2023
@aswinshakil aswinshakil mentioned this pull request Mar 20, 2023
Merged
hemantk-12
hemantk-12 reviewed Mar 21, 2023
View reviewed changes
Copy link
Contributor

@hemantk-12 hemantk-12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the patch @aswinshakil .

Left some comments. Please take a look.

...one/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/OMClientRequestUtils.java Outdated
+ OM_KEY_PREFIX;

iterator.seek(dbSnapshotBucketKey);
status = iterator.hasNext() && iterator.next().getKey()
Copy link
Contributor

@hemantk-12 hemantk-12 Mar 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious if we need to check the volume and bucket in case of legacy bucket type. Similar to https://github.com/apache/ozone/blob/master/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java#L1240

Copy link
Member Author

@aswinshakil aswinshakil Mar 21, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are also checking the snapshotDbKey which is /volumeName/bucketName/ along with the seek(), so it should be fine.

smengcl
smengcl reviewed Mar 21, 2023
View reviewed changes
@aswinshakil
Copy link
Member Author

aswinshakil commented Mar 21, 2023

Thanks for the review @hemantk-12 and @smengcl. I have updated the PR, please take a look at it again.

smengcl
smengcl approved these changes Mar 22, 2023
View reviewed changes
Copy link
Contributor

@smengcl smengcl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 pending CI

hemantk-12
hemantk-12 approved these changes Mar 22, 2023
View reviewed changes
Copy link
Contributor

@hemantk-12 hemantk-12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm.

@aswinshakil
Copy link
Member Author

aswinshakil commented Mar 22, 2023

The test passed here: https://github.com/aswinshakil/ozone/actions/runs/4492231188

@aswinshakil
Copy link
Member Author

aswinshakil commented Mar 22, 2023

Thanks, @smengcl and @hemantk-12 for the review.

@aswinshakil aswinshakil merged commit 15117d8 into apache:master Mar 22, 2023
errose28 added a commit to errose28/ozone that referenced this pull request Mar 23, 2023
@errose28
Merge branch 'master' into HDDS-6633
a24321d 
* master: (43 commits)
  HDDS-8148. Improve log for Pipeline creation failure (apache#4385)
  HDDS-7853. Add support for RemoveSCM in SCMRatisServer. (apache#4358)
  HDDS-8042. Display certificate issuer in cert list command. (apache#4429)
  HDDS-8189. [Snapshot] renamedKeyTable should only track keys in buckets that has at least one active snapshot. (apache#4436)
  HDDS-8154. Perf: Reuse Mac instances in S3 token validation (apache#4433)
  HDDS-8245. Info log for keyDeletingService when nonzero number of keys are deleted. (apache#4451)
  HDDS-8233. ReplicationManager: Throttle delete container commands from over-replication handlers (apache#4447)
  HDDS-8220. [Ozone-Streaming] Trigger volume check on IOException in StreamDataChannelBase (apache#4428)
  HDDS-8173. Fix to remove enrties from RocksDB after container gets deleted. (apache#4445)
  HDDS-7975. Rebalance acceptance tests (apache#4437)
  HDDS-8152. Reduce S3 acceptance test setup time (apache#4393)
  HDDS-8172. ECUnderReplicationHandler should consider commands already sent when processing the container (apache#4435)
  HDDS-7883. [Snapshot] Accommodate FSO, key renames and implement OMSnapshotPurgeRequest for SnapshotDeletingService (apache#4407)
  HDDS-8168. Make deadlines inside MoveManager for move commands configurable (apache#4415)
  HDDS-7918. EC: ECBlockReconstructedStripeInputStream should check for spare replicas before failing an index (apache#4441)
  HDDS-8222. EndpointBase#getBucket should handle BUCKET_NOT_FOUND (apache#4431)
  HDDS-8068. Fix Exception: JMXJsonServlet, getting attribute RatisRoles of Hadoop:service=OzoneManager. (apache#4352)
  HDDS-8139. Datanodes should not drop block delete transactions based on transaction ID (apache#4384)
  HDDS-8216. EC: OzoneClientConfig is overwritten in ECKeyOutputStream (apache#4425)
  HDDS-8054. Fix NPE in metrics for failed volume (apache#4340)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hemantk-12 hemantk-12 hemantk-12 approved these changes

@smengcl smengcl smengcl approved these changes

Assignees

@aswinshakil aswinshakil

Labels

snapshot https://issues.apache.org/jira/browse/HDDS-6517

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@aswinshakil @hemantk-12 @smengcl