Skip to content

Fix skiptoken paging edge case when combined with involving nullable datetime property - #872

Merged
gathogojr merged 1 commit into
OData:mainfrom
gathogojr:fix/1943-skiptoken-paging-edge-case-when-combined-with-orderby
Mar 31, 2023
Merged

Fix skiptoken paging edge case when combined with involving nullable datetime property#872
gathogojr merged 1 commit into
OData:mainfrom
gathogojr:fix/1943-skiptoken-paging-edge-case-when-combined-with-orderby

Conversation

@gathogojr

Copy link
Copy Markdown
Contributor

Related to #859. Fixes skiptoken paging edge case when combined with involving nullable datetime property

In the case of a DateTime property, it is mapped to DateTimeOffset since DateTime is mapped to Edm.DateTimeOffset in the Edm. This fix ensures proper handling of null when the skip token value of the DateTime property is null. The PR also adds tests for the edge case.

@gathogojr
gathogojr force-pushed the fix/1943-skiptoken-paging-edge-case-when-combined-with-orderby branch from 0d0065c to 2943c38 Compare March 27, 2023 08:31
binaryOperator: BinaryOperatorKind.Equal,
left: property,
right: parameterizeConstant ? LinqParameterContainer.Parameterize(propertyType, null) : Expression.Constant(null),
right: parameterizeConstant ? LinqParameterContainer.Parameterize(property.Type, null) : Expression.Constant(null),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You reset propertyType using propertyType = property.Type;

why do you use property.Type again?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xuzhg Since the Edm doesn't support DateTime, we map DateTime to DateTimeOffset. When later we're creating the binary expression to compare the DateTimeOffset value to a DateTime property, we convert the value back to a DateTime in the CreateBinaryExpression method at this point

Now, when the value is of type DateTimeOffset? and the value is null, everything breaks down here. The parameterizedConstantValue is null, so parameterizedConstantValue as DateTimeOffset? returns null and consequently the type conversion from DateTimeOffset? to DateTime? doesn't happen and an exception gets thrown later as a result of trying to compare DateTime? (the type for the property) with DateTimeOffset? (the type for the value). I found this to be the easiest way to address the issue - by creating the binary expression with the actual property type when the value is null.

habbes
habbes previously approved these changes Mar 28, 2023
}

IDictionary<string, Tuple<object, Type>> propertyValuePairs = PopulatePropertyValuePairs(skipTokenRawValue, context);
IDictionary<string, (object Value, Type Type)> propertyValuePairs = PopulatePropertyValuePairs(skipTokenRawValue, context);

@ElizabethOkerio ElizabethOkerio Mar 30, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the benefit of using this format(object Value, Type Type) in place of what was there?

@ElizabethOkerio ElizabethOkerio Mar 30, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure whether the Value and Type are meant to override the default field names item1, item2 etc. If that's the case then I'd suggest we have more descriptive names to tell someone which value is being referred to for example. etc

@gathogojr gathogojr Mar 30, 2023

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the benefit of using this format(object Value, Type Type) in place of what was there?

That is a ValueTuple. It's a struct and @habbes felt using it would help reduce allocations on the heap. Before I had Tuple<object, Type>.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure whether the Value and Type are meant to override the default field names item1, item2 etc. If that's the case then I'd suggest we have more descriptive names to tell someone which value is being referred to for example. etc

They are meant to override the default Item and Item2. I'll rename them to PropertyValue and PropertyType respectively.

@gathogojr
gathogojr force-pushed the fix/1943-skiptoken-paging-edge-case-when-combined-with-orderby branch from 999e380 to d6448be Compare March 30, 2023 08:22
left: property,
right: constant,
liftToNull: propertyIsNullable ? false : true,
liftToNull: !propertyIsNullable,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the meaning of this or why do we use the opposite of what the propertyIsNullable value is?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use in the BinaryExpression.IsLiftedToNull

A lifted operator allows an operator on a non-nullable type to be used with the nullable equivalent as well. See example here.

What we are doing here is setting liftToNull = true when the property in not nullable.

left: property,
right: constant,
liftToNull: propertyIsNullable ? false : true,
liftToNull: !propertyIsNullable,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use in the BinaryExpression.IsLiftedToNull

A lifted operator allows an operator on a non-nullable type to be used with the nullable equivalent as well. See example here.

What we are doing here is setting liftToNull = true when the property in not nullable.

Contract.Assert(context != null);

IDictionary<string, Tuple<object, Type>> propertyValuePairs = new Dictionary<string, Tuple<object, Type>>();
IDictionary<string, (object PropertyValue, Type PropertyType)> propertyValuePairs = new Dictionary<string, (object PropertyValue, Type PropertyType)>();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use a ValueTuple instead of an object?

Tuple - reference type
ValueTuple - value type
object - reference type

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@KenitoInc object is being used here because the property value can be of varying types. For that reason, we cannot swap object with ValueTuple

binaryOperator: BinaryOperatorKind.Equal,
left: property,
right: parameterizeConstant ? LinqParameterContainer.Parameterize(propertyType, null) : Expression.Constant(null),
right: parameterizeConstant ? LinqParameterContainer.Parameterize(property.Type, null) : Expression.Constant(null),

@ElizabethOkerio ElizabethOkerio Mar 30, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the propertyType variable can be used here in place of property.Type since in line 308 you assigned property.Type to propertyType. In any case the code here will be executed only if value is not null. So there is actually no relation between what is assigned in line 308 and what is here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ElizabethOkerio Use of property.Type here is deliberate due to the explanation provided here #872 (comment)

@gathogojr
gathogojr merged commit 230ca82 into OData:main Mar 31, 2023
@gathogojr
gathogojr deleted the fix/1943-skiptoken-paging-edge-case-when-combined-with-orderby branch March 31, 2023 09:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants