Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion website/docs/concurrency_control.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,13 @@ When a transaction attempts to modify a resource that is currently locked by ano

In case of multi-writing in Hudi, the locks are acquired on the Hudi table for a very short duration during specific phases (such as just before committing the writes or before scheduling table services) instead of locking for the entire span of time. This approach allows multiple writers to work on the same table simultaneously, increasing concurrency and avoids conflicts.

There are 4 different lock providers that require different configurations to be set. Please refer to comprehensive locking configs [here](https://hudi.apache.org/docs/next/configurations#LOCK).
There are 4 different lock providers that require different configurations to be set. 1 lock provider requires no configurations. Please refer to comprehensive locking configs [here](https://hudi.apache.org/docs/next/configurations#LOCK).

#### Conditional writes based
```
hoodie.write.lock.provider=org.apache.hudi.client.transaction.lock.ConditionalWriteLockProvider
```
No configs are required for this lock provider, however as of 1.0.2 this lock provider is only supported for S3.

#### Zookeeper based
```
Expand Down
13 changes: 13 additions & 0 deletions website/docs/configurations.md
Original file line number Diff line number Diff line change
Expand Up @@ -1130,6 +1130,19 @@ Configs that control DynamoDB based locking mechanisms required for concurrency
| [hoodie.write.lock.wait_time_ms](#hoodiewritelockwait_time_ms) | 60000 | Lock Acquire Wait Timeout in milliseconds<br />`Config Param: LOCK_ACQUIRE_WAIT_TIMEOUT_MS_PROP_KEY`<br />`Since Version: 0.10.0` |
---

#### Storage based conditional write locks configurations {#ConditionalWrite-based-Locks-Configurations}
Configs that control conditional write based locking mechanisms required for concurrency control between writers to a Hudi table. Concurrency between Hudi's own table services are auto managed internally.

[**Advanced Configs**](#ConditionalWrite-based-Locks-Configurations-advanced-configs)


| Config Name | Default | Description |
| --------------------------------------------------------------------------------------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| [hoodie.write.lock.conditional_write.locks_location](#hoodiewritelockconditional_write_locks_location) | (N/A) | For conditional writes based lock provider, the centralized location to write all locks to. If not provided, locks are written to each table's base path: `.hoodie/.locks/table_lock.json`. <br />`Config Param: CONDITIONAL_WRITE_LOCKS_LOCATION`<br />`Since Version: 1.0.2.1` |
| [hoodie.write.lock.conditional_write.heartbeat_poll_ms](#hoodiewritelockconditional_write_heartbeat_poll_ms) | 30000 | For conditional writes based lock provider, How often to renew each held lock, default is 30 seconds.<br />`Config Param: CONDITIONAL_WRITE_HEARTBEAT_POLL_MS`<br />`Since Version: 1.0.2.0` |
| [hoodie.write.lock.conditional_write.lock_validity_timeout_ms](#hoodiewritelockconditional_write_lock_validity_timeout_ms) | 300000 | For conditional writes based lock provider, the amount of time each new lock is valid before expiring, default is 5 minutes. <br />`Config Param: CONDITIONAL_WRITE_LOCK_VALIDITY_TIMEOUT_MS`<br />`Since Version: 1.0.2` | |
---


### Key Generator Configs {#KEY_GENERATOR}
Hudi maintains keys (record key + partition path) for uniquely identifying a particular record. These configs allow developers to setup the Key generator class that extracts these out of incoming records.
Expand Down