CSHARP-5442: Atlas Search doesn't respect the correct serializer #1583

papafe · 2024-12-31T14:56:55Z

papafe · 2024-12-31T15:45:03Z

@BorisDog I've added you as a reviewer since it seems you were the one that worked on search, mostly. This is still a work in progress, I've broke some things in the process.
A couple of notes:

Probably we need to change the in and range search definitions as well to use the correct serializers.
This is effectively a behaviour breaking change, so we need to decide if we want include it in next version or wait for next major

papafe · 2025-01-02T11:20:23Z

src/MongoDB.Driver/Search/OperatorSearchDefinitions.cs

+
+                var renderedField = _arrayField.Render(args);
+
+                var serializer =


Incredibly ugly, but was just testing if it worked

@rstam this is what I was referring to.
Do you think it would be worth adding a new property to RenderArgs so that we can get directly the serializer we need (similar to RenderForElemMatch for instance)?

I think this is on the right track but can use a little tweaking to how the right serializer is found and to reduce duplication between how scalar and array fields are handled.

Something like this:

private protected override BsonDocument RenderArguments(RenderArgs<TDocument> args) { IBsonSerializer<TValue> valueSerializer; if (_field != null) { var renderedField = _field.Render(args); valueSerializer = renderedField.FieldSerializer; } else { var renderedArrayField = _arrayField.Render(args); valueSerializer = (IBsonSerializer<TValue>)ArraySerializerHelper.GetItemSerializer(renderedArrayField.FieldSerializer); } var document = new BsonDocument(); using var bsonWriter = new BsonDocumentWriter(document); var context = BsonSerializationContext.CreateRoot(bsonWriter); bsonWriter.WriteStartDocument(); bsonWriter.WriteName("value"); valueSerializer.Serialize(context, _value); bsonWriter.WriteEndDocument(); return document; }

Perfect, this solves my doubts. That makes sense, I was mostly focusing on how to get the correct serializer before trying to refactor.

Regarding IBsonArraySerializer, the majority of times when we implement that interface we also implement IChildSerializerConfigurable, and that makes sense. It also seems that for this use case the result we'd get would be the same, so what would you think about using IChildSerializerConfigurable instead?
To be honest this is more of a generic question, I'm curious about if there's any preference on using one over the other, of course considering that TryGetItemSerializationInfo can potentially return more info (not in this case though).

IBsonArraySerializer and IChildSerializerConfigurable are orthogonal. We can't just implement IChildSerializerConfigurable instead of IBsonArraySerializer.

In this case we must implement IBsonArraySerializer.

It is true that in general whenever implementing IBsonArraySerializer we would likely also want to implement IChildSerializerConfigurable. However, that applies mostly to public serializers that a user might want to configure. This is an internal serializer, so I think implementing IBsonArraySerializer is sufficient.

rstam · 2025-01-03T20:35:42Z

src/MongoDB.Driver/Search/OperatorSearchDefinitions.cs

@@ -118,38 +119,88 @@ private protected override BsonDocument RenderArguments(RenderArgs<TDocument> ar

    internal sealed class EqualsSearchDefinition<TDocument, TField> : OperatorSearchDefinition<TDocument>
    {
-        private readonly BsonValue _value;
+        private readonly TField _value;


Rename TField to TValue because that is what it is.

rstam · 2025-01-03T20:36:55Z

src/MongoDB.Driver/Search/OperatorSearchDefinitions.cs

+        public EqualsSearchDefinition(FieldDefinition<TDocument, TField> path, TField value, SearchScoreDefinition<TDocument> score)
+            : base(OperatorType.Equals, new SingleSearchPathDefinition<TDocument>(path), score)
+        {
+            ValidateType();


I question whether we need to have ValidateType at all. Looks to me like this was just a client-side limitation related to what .NET types ToBsonValue knew how to convert to BSON, but now that we are using the proper serializer this limitation should no longer exist.

I checked with Boris and he confirmed that it is a server limitation which types are supported.

But even so, we probably shouldn't be checking this client side. We should just send the value to the server and let it return an error if it doesn't support the data type.

That way our driver will work well with all supported versions of the server, which might have different limitations. And if future server versions support new value types we won't have to make any changes.

I also like this idea. I tried to keep it consistent with what was there before, but I agree that moving this check server side would allow us to be more free in what we do and don't support over time.
I need to remember to do the same with the other operators too.

rstam · 2025-01-03T20:38:34Z

src/MongoDB.Driver/Search/OperatorSearchDefinitions.cs

+
+                var renderedField = _arrayField.Render(args);
+
+                var serializer =


I think this is on the right track but can use a little tweaking to how the right serializer is found and to reduce duplication between how scalar and array fields are handled.

Something like this:

private protected override BsonDocument RenderArguments(RenderArgs<TDocument> args) { IBsonSerializer<TValue> valueSerializer; if (_field != null) { var renderedField = _field.Render(args); valueSerializer = renderedField.FieldSerializer; } else { var renderedArrayField = _arrayField.Render(args); valueSerializer = (IBsonSerializer<TValue>)ArraySerializerHelper.GetItemSerializer(renderedArrayField.FieldSerializer); } var document = new BsonDocument(); using var bsonWriter = new BsonDocumentWriter(document); var context = BsonSerializationContext.CreateRoot(bsonWriter); bsonWriter.WriteStartDocument(); bsonWriter.WriteName("value"); valueSerializer.Serialize(context, _value); bsonWriter.WriteEndDocument(); return document; }

rstam · 2025-01-03T20:40:16Z

src/MongoDB.Driver/FieldValueSerializerHelper.cs

@@ -317,6 +317,8 @@ internal class IEnumerableSerializer<TItem> : SerializerBase<IEnumerable<TItem>>
        {
            private readonly IBsonSerializer<TItem> _itemSerializer;

+            public IBsonSerializer<TItem> ItemSerializer => _itemSerializer;
+


Instead of adding this new public property have the class implement IBsonArraySerializer and implement the TryGetItemSerializationIinfo method:

public bool TryGetItemSerializationInfo(out BsonSerializationInfo serialization { serializationInfo = new(elementName: null, _itemSerializer, typeof(TItem)); return true; }

All serializers for array-like values should implement IBsonArraySerializer. I don't know why this class did not already implement it.

rstam · 2025-01-03T20:43:15Z

src/MongoDB.Driver/Search/OperatorSearchDefinitions.cs

+                var renderedField = _arrayField.Render(args);
+
+                var serializer =
+                    (renderedField.ValueSerializer as FieldValueSerializerHelper.IEnumerableSerializer<TField>)


Instead of trying to cast to the very specific class FieldValueSerializerHelper.IEnumerableSerializer<TField> use the more general from (used everywhere else) to get the item serializer from an array-like serializer:

valueSerializer = (IBsonSerializer<TValue>)ArraySerializerHelper.GetItemSerializer(renderedArrayField.FieldSerializer);

This is why FieldValueSerializerHelper.IEnumerableSerializer<TItem> should implement IBsonArraySerializer (as all array-like serializers should). Not sure why it didn't already...

rstam · 2025-01-03T22:23:20Z

src/MongoDB.Driver/Search/OperatorSearchDefinitions.cs

+        public EqualsSearchDefinition(FieldDefinition<TDocument, TField> path, TField value, SearchScoreDefinition<TDocument> score)
+            : base(OperatorType.Equals, new SingleSearchPathDefinition<TDocument>(path), score)
+        {
+            ValidateType();


I checked with Boris and he confirmed that it is a server limitation which types are supported.

But even so, we probably shouldn't be checking this client side. We should just send the value to the server and let it return an error if it doesn't support the data type.

That way our driver will work well with all supported versions of the server, which might have different limitations. And if future server versions support new value types we won't have to make any changes.

papafe · 2025-01-06T09:29:32Z

tests/MongoDB.Driver.Tests/Search/SearchDefinitionBuilderTests.cs

@@ -290,6 +290,8 @@ public void Equals_should_render_supported_type<T>(
                subject.Equals("x", value),
                $"{{ equals: {{ path: 'x', value: {valueRendered} }} }}");

+            //For the Guid we get (correctly): GuidSerializer cannot serialize a Guid when GuidRepresentation is Unspecified.


@rstam This test here is now failing because now we're using the default serializer and that doesn't have the GuidRepresentation specified.
Do you have any suggestion on how to fix this? The exception makes sense, and normally we'd probably suggest statically registering a serializer, but this would impact all the other tests here.
Should we ignore the Guid field for this test and make a separable test here that can only be run manually?
This would need to be done also for other operators.

That's because on line 292 this testuses the string form of the field name and specified "x" and there is no field called 'x" to pick up the correctly configured GuidSerializer from.

I would simply skip this part of the test for Guid:

// in this test there is no property called "x" to pick up a properly configured GuidSerializer from if (typeof(T) != typeof(Guid)) { AssertRendered( subject.Equals("x", value), $"{{ equals: {{ path: 'x', value: {valueRendered} }} }}"); var scoreBuilder = new SearchScoreDefinitionBuilder<BsonDocument>(); AssertRendered( subject.Equals("x", value, scoreBuilder.Constant(1)), $"{{ equals: {{ path: 'x', value: {valueRendered}, score: {{ constant: {{ value: 1 }} }} }} }}"); }

Yeah, I would have loved to avoid making special cases, but it seems we need to. I'll make some additional tests that can be run manually.

I'm fine with just skipping this test for Guids, because:

This test is against IMongoCollection<BsonDocument> and that is probably not used often

Anyone using BsonDocument should either not be using Guid or should take control of the conversion themselves

A manual test would involve registering a properly configured global GuidSerializer, but we already know that works (in part because the strongly typed version uses a properly configured GuidSerializer, though it's true it doesn't get it from the global registry).

papafe added 5 commits December 31, 2024 10:42

Initial

40d8bd9

Various corrections

65567ee

Corrections

572ed5c

Small fix

1493813

Removed extra spaces

7f44c62

papafe requested a review from BorisDog December 31, 2024 15:29

papafe added 5 commits December 31, 2024 18:33

Small corrections

8dfa53f

Added tests and fixed

a5ba406

Missing statment

78490cd

Various corrections

845c7eb

Corrections

e283312

papafe commented Jan 2, 2025

View reviewed changes

papafe added 2 commits January 2, 2025 13:02

Small fix

dd89854

Small correction

7c7eafe

papafe requested a review from rstam January 2, 2025 16:36

rstam requested changes Jan 3, 2025

View reviewed changes

papafe commented Jan 6, 2025

View reviewed changes

papafe added 4 commits January 6, 2025 10:45

Moved testing to main file

2f1af5f

Corrected operator

f400017

Added missing parenthesis

5323c79

Removed unsupported test

b776914

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CSHARP-5442: Atlas Search doesn't respect the correct serializer #1583

CSHARP-5442: Atlas Search doesn't respect the correct serializer #1583

papafe commented Dec 31, 2024 •

edited

Loading

papafe commented Dec 31, 2024 •

edited

Loading

papafe Jan 2, 2025

papafe Jan 2, 2025 •

edited

Loading

rstam Jan 3, 2025

papafe Jan 6, 2025 •

edited

Loading

rstam Jan 6, 2025

rstam Jan 3, 2025

rstam Jan 3, 2025

rstam Jan 3, 2025

papafe Jan 6, 2025 •

edited

Loading

rstam Jan 3, 2025

rstam Jan 3, 2025

rstam Jan 3, 2025

rstam Jan 3, 2025

papafe Jan 6, 2025 •

edited

Loading

rstam Jan 6, 2025

papafe Jan 6, 2025

rstam Jan 6, 2025


		var renderedField = _arrayField.Render(args);

		var serializer =

CSHARP-5442: Atlas Search doesn't respect the correct serializer #1583

Are you sure you want to change the base?

CSHARP-5442: Atlas Search doesn't respect the correct serializer #1583

Conversation

papafe commented Dec 31, 2024 • edited Loading

papafe commented Dec 31, 2024 • edited Loading

Choose a reason for hiding this comment

papafe Jan 2, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

papafe Jan 6, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

papafe Jan 6, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

papafe Jan 6, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

papafe commented Dec 31, 2024 •

edited

Loading

papafe commented Dec 31, 2024 •

edited

Loading

papafe Jan 2, 2025 •

edited

Loading

papafe Jan 6, 2025 •

edited

Loading

papafe Jan 6, 2025 •

edited

Loading

papafe Jan 6, 2025 •

edited

Loading