You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
While testing splitting lots of tablets it was observed that seeding the split fate transactions was the bottleneck and not running the actual fate operation. Looking at the code there is a lot of back and forth to the seed a fate transaction
Do conditional mutation reserve
Do a scan to see if there is a top on the stack
Do a conditional mutation to set the transaction info
Do a conditional mutation to set the transaction info
Do a conditional mutation to set the status
All of this could likely be done in a single conditional mutation. This would avoid the scan and doing multiple conditional mutations. Observed this while looking into splits, but improving this would speed up compaction commit and split.
Expected behavior
If possible a single conditional mutation is run to seed a fate transaction.
Additional context
There may be other places in the code that are submitting multiple conditional mutations for fate operations when one would suffice. This is caused by the code evolving from initially having a zookeeper persistent store where data was stored in multiple zk nodes for a single fate operation. Multiple ZK nodes require multiple write. Implementing #4905 could allow zookeeper and accumuo table impls to have less writes for fate updates by improving the code for both to support doing more single writes.
The text was updated successfully, but these errors were encountered:
When experimenting with this was seeing that a single thread could seed around 1.3 fate operations per second. Making this a single conditional mutation should increase that rate significantly.
Describe the bug
While testing splitting lots of tablets it was observed that seeding the split fate transactions was the bottleneck and not running the actual fate operation. Looking at the code there is a lot of back and forth to the seed a fate transaction
All of this could likely be done in a single conditional mutation. This would avoid the scan and doing multiple conditional mutations. Observed this while looking into splits, but improving this would speed up compaction commit and split.
Expected behavior
If possible a single conditional mutation is run to seed a fate transaction.
Additional context
There may be other places in the code that are submitting multiple conditional mutations for fate operations when one would suffice. This is caused by the code evolving from initially having a zookeeper persistent store where data was stored in multiple zk nodes for a single fate operation. Multiple ZK nodes require multiple write. Implementing #4905 could allow zookeeper and accumuo table impls to have less writes for fate updates by improving the code for both to support doing more single writes.
The text was updated successfully, but these errors were encountered: