[HUDI-9156] Add lock provider heartbeat manager #12958
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
danny0405
reviewed
Mar 12, 2025
| } | ||
|
|
||
| if (this.hasActiveHeartbeat()) { | ||
| logger.warn("Owner {}: Heartbeat is already running.", ownerId); |
Contributor
There was a problem hiding this comment.
Looks like this HeartbeatManager is hold by a client, and the conflict resolution is taken care by the server(heartbeat recipient).
Contributor
Author
There was a problem hiding this comment.
I'm not quite sure I understand what you are asking:
- heartbeat manager runs on a thread in same jvm as lock manager/writer client
- this allows us to abstract away maintaining the lock from the lock provider.
- from the perspective of lock provider, all it does is call
heartbeatManager.startHeartbeatForThread(Thread.currentThread())
I will update the PR description with these details.
alexr17
changed the title
alexr17
changed the title
Collaborator
danny0405
approved these changes
Apr 8, 2025
Contributor
danny0405
left a comment
danny0405
left a comment
There was a problem hiding this comment.
The API looks good to me.
4 tasks
voonhous
pushed a commit
to voonhous/hudi
that referenced
this pull request
Apr 8, 2025
(cherry picked from commit 802f5ae)
voonhous
pushed a commit
to voonhous/hudi
that referenced
this pull request
Apr 9, 2025
(cherry picked from commit 802f5ae)
voonhous
pushed a commit
to voonhous/hudi
that referenced
this pull request
Apr 11, 2025
(cherry picked from commit 802f5ae)
voonhous
pushed a commit
to voonhous/hudi
that referenced
this pull request
Apr 15, 2025
(cherry picked from commit 802f5ae)
voonhous
pushed a commit
to voonhous/hudi
that referenced
this pull request
Apr 16, 2025
(cherry picked from commit 802f5ae)
4 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Change Logs
Adds a heartbeat manager for the lock provider with conditional writes.
See RFC: #12927
Impact
This heartbeat manager is responsible for maintaining a thread which constantly updates the lock file in storage.
The heartbeat manager runs on a thread in same jvm as lock manager/writer client. This allows us to abstract away maintaining the lock from the lock provider. From the perspective of lock provider, all it does is call
heartbeatManager.startHeartbeatForThread(Thread.currentThread()).Risk level (write none, low medium or high below)
None
Documentation Update
Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none".
None
Contributor's checklist