Skip to content

Fix bug when DELETE ACID block is a DictionaryBlock#9354

Closed
djsagain wants to merge 2 commits intotrinodb:masterfrom
djsagain:david.stryker/fix-delete-all
Closed

Fix bug when DELETE ACID block is a DictionaryBlock#9354
djsagain wants to merge 2 commits intotrinodb:masterfrom
djsagain:david.stryker/fix-delete-all

Conversation

@djsagain
Copy link
Copy Markdown
Member

@djsagain djsagain commented Sep 23, 2021

This PR consists of two commits:

Before the first commit, Block.getChildren() was used to take apart the
DELETE ACID rowId block. That is always the wrong thing to do.
Fixed by using ColumnarRow to access the elements of the ACID rowId block.

This first commit also adds a comment warning developers not to use
Block.getChildren() so others don't make the same mistake of
calling the method.

The second commit replaces use of Block.getChildren in
HiveUpdateProcessor with ColumnarRow methods.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this check survive in the form of columnarRow.getFieldCount()==3 inside deleteRowsInternal?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, added.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

redundant

Suggested change
// Throw an exception if there are any null rows

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deleteRows is not a variable or anything else known

Suggested change
checkArgument(!columnarRow.isNull(position), "In the deleteRows block, found null position %s", position);
checkArgument(!columnarRow.isNull(position), "In the delete rowIds, found null row at position %s", position);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think the important aspect is "delete after delete".

assuming it's correct, let's reflect this in a test name: testDeleteAfterDelete

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea; renamed.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: lowercase count, true

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: lowercase true

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

verification of this stage is a bit redundant (we already have some tests for delete), especially if we run this on Hive too. remove, or keep only on Trino.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to test only on Trino.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was the bug triggered for DELETE without any predicate, or for a delete with a predicate too?

we should test both cases

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. I added a test that differs in this way:

            // A predicate sufficient to fool statistics-based optimization
            onTrino().executeQuery(format("DELETE FROM %s WHERE id != 2" + tableName));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All logging removed from this method.

@findepi
Copy link
Copy Markdown
Member

findepi commented Sep 24, 2021

cc @losipiuk

@djsagain djsagain force-pushed the david.stryker/fix-delete-all branch from 3fac599 to 6a9ca50 Compare September 24, 2021 14:57
@cla-bot cla-bot bot added the cla-signed label Sep 24, 2021
@djsagain
Copy link
Copy Markdown
Member Author

Thanks for the prompt, close review, @findepi. I force pushed an update making all your suggested changes.

BTW, separating the WHERE clause out as a separate argument to the TestHiveTransactionalTable.verifySelect* methods, which I introduced, was a dumb idea. I will submit a no-semantic-change PR to remove them.

Before this commit, Block.getChildren() was used to take apart the
DELETE ACID rowId block.  That is always the wrong thing to do.
Fixed by using ColumnarRow to access the elementa of the ACID rowId block.

This commit also adds a comment warning developers not to use
Block.getChildren() so others don't make the same mistake of
calling the method.
@djsagain djsagain force-pushed the david.stryker/fix-delete-all branch from 6a9ca50 to 4d85258 Compare September 25, 2021 14:11
Replace use of Block.getChildren in HiveUpdatablePageSource and
HiveUpdateProcessor with ColumnarRow methods.
@findepi
Copy link
Copy Markdown
Member

findepi commented Sep 28, 2021

CI #8432

@findepi
Copy link
Copy Markdown
Member

findepi commented Sep 28, 2021

Merged as 87785b4, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

3 participants