Skip to content

Conversation

@Yury-Fridlyand
Copy link

@Yury-Fridlyand Yury-Fridlyand commented Mar 21, 2023

Description

Rework PhysicalPlan tree serialization and deserialization. Make it scalable and extendable.
If a new plan should be serialized (a new feature is becoming supported by pagination) - it should properly overload SerializablePlan::writeExternal only.
The plan tree will be recovered in deserialization the same way it was before serialization.
New mechanism can also skip plans from serialization - for example, it skips ResourceMonitorPlan.

Issues Resolved

opensearch-project#1483

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Yury-Fridlyand <[email protected]>

Step 2.

Signed-off-by: Yury-Fridlyand <[email protected]>
@Yury-Fridlyand Yury-Fridlyand changed the base branch from dev-pagination-v2_revC to integ-pagination-p1 March 29, 2023 20:33
@Yury-Fridlyand Yury-Fridlyand force-pushed the dev-pagination-v2_revD branch from 7a9d285 to ef44600 Compare March 29, 2023 20:33
Signed-off-by: Yury-Fridlyand <[email protected]>
Signed-off-by: Yury-Fridlyand <[email protected]>
Signed-off-by: Yury-Fridlyand <[email protected]>
@codecov

This comment was marked as spam.

Signed-off-by: Yury-Fridlyand <[email protected]>
return new Cursor(CURSOR_PREFIX
+ serialize(((SerializablePlan) plan).getPlanForSerialization()));
// ClassCastException thrown when a plan in the tree doesn't implement SerializablePlan
} catch (ClassCastException | NoCursorException e) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When can ClassCastException be thrown?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are always trying to convert a plan tree to a cursor in


If tree contains a plan which doesn't implement SerializablePlan, there would be a ClassCastException somewhere, e.g. in
@Override
public SerializablePlan getPlanForSerialization() {
return (SerializablePlan) delegate;
}

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't serialize PaginateOperator (to reduce cursor size), so I can't check plan instanceof PaginateOperator here as it was before.

/** No need to serialize a PaginateOperator, it actually does nothing - it is a wrapper. */
@Override
public SerializablePlan getPlanForSerialization() {
return (SerializablePlan) input;
}

That means I have to do another protection here. The incoming tree could be ProjectOperator -> ResourceMonitorPlan -> PrometheusIndexScan or ProjectOperator -> ValuesOperator.

Signed-off-by: Yury-Fridlyand <[email protected]>
@Yury-Fridlyand
Copy link
Author

Superseded by opensearch-project#1498

@Yury-Fridlyand Yury-Fridlyand deleted the dev-pagination-v2_revD branch June 24, 2023 00:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants