Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Seeding compaction commit and split fate transactions is slow #5097

Open
keith-turner opened this issue Nov 22, 2024 · 1 comment
Open

Seeding compaction commit and split fate transactions is slow #5097

keith-turner opened this issue Nov 22, 2024 · 1 comment
Labels
bug This issue has been verified to be a bug.
Milestone

Comments

@keith-turner
Copy link
Contributor

keith-turner commented Nov 22, 2024

Describe the bug

While testing splitting lots of tablets it was observed that seeding the split fate transactions was the bottleneck and not running the actual fate operation. Looking at the code there is a lot of back and forth to the seed a fate transaction

  1. Do conditional mutation reserve
  2. Do a scan to see if there is a top on the stack
  3. Do a conditional mutation to set the transaction info
  4. Do a conditional mutation to set the transaction info
  5. Do a conditional mutation to set the status

All of this could likely be done in a single conditional mutation. This would avoid the scan and doing multiple conditional mutations. Observed this while looking into splits, but improving this would speed up compaction commit and split.

Expected behavior
If possible a single conditional mutation is run to seed a fate transaction.

Additional context

There may be other places in the code that are submitting multiple conditional mutations for fate operations when one would suffice. This is caused by the code evolving from initially having a zookeeper persistent store where data was stored in multiple zk nodes for a single fate operation. Multiple ZK nodes require multiple write. Implementing #4905 could allow zookeeper and accumuo table impls to have less writes for fate updates by improving the code for both to support doing more single writes.

@keith-turner keith-turner added the bug This issue has been verified to be a bug. label Nov 22, 2024
@keith-turner
Copy link
Contributor Author

When experimenting with this was seeing that a single thread could seed around 1.3 fate operations per second. Making this a single conditional mutation should increase that rate significantly.

@keith-turner keith-turner added this to the 4.0.0 milestone Nov 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue has been verified to be a bug.
Projects
None yet
Development

No branches or pull requests

1 participant