Support "dynamic" and writable DOM

[justinushermawan]

Does JsonSerializer.Parse(String, Type, JsonSerializerOptions) support for dynamic ExpandoObject return type?

Something like this:
dynamic p = JsonSerializer.Parse(json, typeof(ExpandoObject));

[ahsonkhan]

No, this feature is not supported at this point but something we should consider for vNext. Do you have an example usage to motivate the feature request?

Marking as future.

System.NotSupportedException : The collection type 'System.Dynamic.ExpandoObject' is not supported.

[deinok]

@ahsonkhan GraphQL is a good example.

The spec recomends JSON but it is not tied to any specific serialization in the response.
This implys that the "data" field of the response is of the kind: dynamic. As it can't be infered.
Without ExpandoObject the deserialization makes dynamic a type of a JSON making. So accessing to that abstract dynamic "data" must be done knowing that in fact that dynamic is a JToken.

With ExpandoObject I think that we could enforce the access of the dynamic "data" like a common object

[tiksn]

@ahsonkhan another example is Configuration Service in our project. It exposes collection like REST endpoints, which create collections in MongoDB (it is not just a dummy REST wrapper, and rest collections and mongo collection do not have exact 1-1 mapping, it also asserts certain rules).

So in our project we need dynamic/ExpandoObject support.

We use it in other microservices as well.

[SidShetye]

We also ran into this limitation. Our use case is gradually building a dynamic object before json serialization. Went back to the more mature Json.NET serializer.

[MikeReznikov]

Hey guys

whats the walk around for now?
May i configure which one to use?
@SidShetye you said something about went back to more mature one, may you explain please?

[SidShetye]

@MickeyReznikov : See https://stackoverflow.com/questions/15455304/deserialize-a-property-as-an-expandoobject-using-json-net or google “json.net expandoobject”

[ahsonkhan]

@MickeyReznikov, was your question answered? I believe @SidShetye meant going back to using Newtonsoft.Json for serializing dynamic objects since the in-box library (System.Text.Json) doesn't have support for those yet. In the case of asp.net apps, you can configure it to AddNewtonsoftJson back.

See https://docs.microsoft.com/en-us/aspnet/core/migration/22-to-30?view=aspnetcore-3.0&tabs=visual-studio#jsonnet-support

[estiller]

I also have a use-case for this - I just want to call a REST API with HttpClient and retrieve a single property from the response. I do not want to create a dedicated class just to parse the response... With JSON.NET I could just use "dynamic" and access the property of choice.

[ahsonkhan]

From https://github.com/dotnet/corefx/issues/41472 by @ghost1372:

hi I hope this is a good place to ask this question
It seems we can't deserialize dynamic objects
i used this code but not worked is there any way to do this?

var objList = System.Text.Json.JsonSerializer.Deserialize<List<dynamic>>(json);

[sheryever]

In our many many applications we are controlling the fields of data from the stored procedures and dynamically render the List and Search List pages with jquery.jtable

Without JsonSerializer built-in support for ExpandoObject we cannot use dotnet core built-in Json Serialization

[daviddesmet]

We do share the same use case as already mentioned by @estiller and @SidShetye
We had to switch back to Json.NET in the meantime.

[lfr]

Is it possible to have a milestone closer to now than Future? 🤔

ExpandoObject's been in the BCL for a looooong time

[fatihyildizhan]

Is there any alternative for ExpandoObject?

[sheryever]

@fatihyildizhan No there is no alternate.

but we have written our own ExpandoObject converter, you can take the hint from this article ASP.NET Core 3.0: Custom JsonConverter for the new System.Text.Json

We need only the serialization, so we just create the serialization

[corbinmunce]

I was surprised this isn't supported yet.

[ahsonkhan]

Moving this to 5.0.

[RobbyDeLaet]

Moving to 5.0? That means we have to wait for a year at least? Back to JSON.Net it is.

[Laiteux]

5.0? wow, that definitely sucks.

[AlexeiScherbakov]

For temporal ugly workaround I can use JsonDocument with custom converter for it, but it is IDisposable.

public sealed class EventObject
	{
		[JsonPropertyName("id")]
		public long Id
		{
			get; set;
		}

		[JsonPropertyName("eventData")]
		[JsonConverter(typeof(JsonDocumentConverter))]
		public System.Text.Json.JsonDocument EventData
		{
			get; set;
		}
	}

	internal sealed class JsonDocumentConverter
		: JsonConverter<System.Text.Json.JsonDocument>
	{
		public override JsonDocument Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
		{
			return JsonDocument.ParseValue(ref reader);
		}

		public override void Write(Utf8JsonWriter writer, JsonDocument value, JsonSerializerOptions options)
		{
			value.WriteTo(writer);
		}
	}

[Arash-Sabet]

We are no longer able to use the following POST action in the presence of System.Text.Json:

[HttpPost]
public async Task<IActionResult> SubmitAsync(dynamic model)

Instead, we had to use the following method but there was no straight forward to use 'model' in the downstream backend code:

[HttpPost]
public async Task<IActionResult> SubmitAsync(JsonElement model)

'model' is a complex object and it may contain a collection of objects and/or other nested complex objects. To be able to conclude a dynamic object out of JsonElement, we had to call model.GetRawText() and have Newtonsoft decode it into a dynamic object. This way is not the desired way because the main purpose of this exercise was to decommission Newtonsoft.json from the project.

Can I assume that addressing this ticket/issue implies a fix for our issue that we've been experiencing? It seems to be a bit urgent issue to address, so can it be addressed sooner than later?

/cc @ahsonkhan @terrajobst

[tchivs]

.NET Core 3.0 JsonSerializer.Deserialize to dynamic object

JsonSerializer support for ExpandoObject(Interim measures)
I am newbie, many places are not perfect, welcome everyone to modify
.net Core3 no support

Add the Json Converter

add using:

• using System.Text.Json;
• using System.Text.Json.Serialization;

    /// <summary>
    /// Temp Dynamic Converter
    /// by:[email protected]
    /// </summary>
    public class DynamicJsonConverter : JsonConverter<dynamic>
    {
        public override dynamic Read(ref Utf8JsonReader reader,
            Type typeToConvert,
            JsonSerializerOptions options)
        {

            if (reader.TokenType == JsonTokenType.True)
            {
                return true;
            }

            if (reader.TokenType == JsonTokenType.False)
            {
                return false;
            }

            if (reader.TokenType == JsonTokenType.Number)
            {
                if (reader.TryGetInt64(out long l))
                {
                    return l;
                }

                return reader.GetDouble();
            }

            if (reader.TokenType == JsonTokenType.String)
            {
                if (reader.TryGetDateTime(out DateTime datetime))
                {
                    return datetime;
                }

                return reader.GetString();
            }

            if (reader.TokenType == JsonTokenType.StartObject)
            {
                using JsonDocument documentV = JsonDocument.ParseValue(ref reader);
                return ReadObject(documentV.RootElement);
            }
            // Use JsonElement as fallback.
            // Newtonsoft uses JArray or JObject.
            JsonDocument document = JsonDocument.ParseValue(ref reader);
            return document.RootElement.Clone();
        }

        private object ReadObject(JsonElement jsonElement)
        {
            IDictionary<string, object> expandoObject = new ExpandoObject();
            foreach (var obj in jsonElement.EnumerateObject())
            {
                var k = obj.Name;
                var value = ReadValue(obj.Value);
                expandoObject[k] = value;
            }
            return expandoObject;
        }
        private object? ReadValue(JsonElement jsonElement)
        {
            object? result = null;
            switch (jsonElement.ValueKind)
            {
                case JsonValueKind.Object:
                    result = ReadObject(jsonElement);
                    break;
                case JsonValueKind.Array:
                    result = ReadList(jsonElement);
                    break;
                case JsonValueKind.String:
                    //TODO: Missing Datetime&Bytes Convert
                    result = jsonElement.GetString();
                    break;
                case JsonValueKind.Number:
                    //TODO: more num type
                    result = 0;
                    if (jsonElement.TryGetInt64(out long l))
                    {
                        result = l;
                    }
                    break;
                case JsonValueKind.True:
                    result = true;
                    break;
                case JsonValueKind.False:
                    result = false;
                    break;
                case JsonValueKind.Undefined:
                case JsonValueKind.Null:
                    result = null;
                    break;
                default:
                    throw new ArgumentOutOfRangeException();
            }
            return result;
        }

        private object? ReadList(JsonElement jsonElement)
        {
            IList<object?> list = new List<object?>();
            foreach (var item in jsonElement.EnumerateArray())
            {
                list.Add(ReadValue(item));
            }
            return list.Count == 0 ? null : list;
        }
        public override void Write(Utf8JsonWriter writer,
            object value,
            JsonSerializerOptions options)
        {
           // writer.WriteStringValue(value.ToString());
        }
    }

How to Use?

var serializerOptions = new JsonSerializerOptions
{
    Converters = { new DynamicJsonConverter() }
};
return JsonSerializer.Deserialize<dynamic>("{OK:"200"}", serializerOptions);

[denmitchell]

@tchivs, your solution worked for me; but since the Converters property is read-only, I had to do something like this:

var serializerOptions = new JsonSerializerOptions();
serializerOptions.Converters.Add(new DynamicJsonConverter());
return JsonSerializer.Deserialize<dynamic>("{OK:"200"}", serializerOptions);

[zehavibarak]

Try using the JsonElement type:

public JsonElement MyProperty {get; set;}

[denmitchell]

@tchivs, I made some modifications to your code -- reworking it so that it uses dynamically generated "projection" types (having a subset of properties from a base type), rather than ExpandoObject. I posted the code (in a sample console project) here: EDennis.DynamicDeserialization.

I will be testing this approach with more complicated objects and under various scenarios (e.g., deserialization of dynamic objects used to patch existing, fully typed EF Core entities; deserialization of json objects and arrays for test cases). Let me know if you find this useful or if you see anything problematic.

[GuerrillaCoder]

Thanks for the community workaround. It's surprising Microsoft are unable to put out this feature in a sensible timeframe. Working with GraphQL responses without dynamic object of some sort leads to lots of verbose and ugly code. Or even just checking for existence of deeply nested properties.

[mauroservienti]

I read through the thread of comments and most are focused on deserialization, I'm facing an issue where also serialization of dynamic objects apparently silently "fails". In an attempt to reproduce my scenario I came up with the following minimal repro:

using System;
using System.Collections.Generic;
using System.Dynamic;
using System.Text.Json;

namespace SampleConsoleApp
{
    class Program
    {
        static void Main(string[] args)
        {
            dynamic d = new CustomDynamicObject();
            d.AProperty = 10;
            var s = JsonSerializer.Serialize(d);

            Console.WriteLine(s);
            Console.Read();
        }
    }

    class CustomDynamicObject : DynamicObject 
    {
        private readonly IDictionary<string, object> properties = new Dictionary<string, object>();

        public override bool TryGetMember(GetMemberBinder binder, out object result) => properties.TryGetValue(binder.Name, out result);

        public override bool TrySetMember(SetMemberBinder binder, object value)
        {
            properties[binder.Name] = value;
            return true;
        }

        public override IEnumerable<string> GetDynamicMemberNames()
        {
            foreach (var item in properties.Keys)
            {
                yield return item;
            }
        }
    }
}

When run s is {}, so serialization doesn't fail but yields an empty json object.

Is this the right issue? or should I raise/follow a different one?

[ahsonkhan]

Is this the right issue? or should I raise/follow a different one?

This is the right issue. No need to open one for the two halves of the same feature. This issue is about adding support for expando object (which means for both sides, serialize and deserialize).

[John0King]

I saw many people in this issue talk about the dynamic but what about the Writable JSON DOM ?
As a Newtonsoft.Json user, I won't use dynamic, instead I use JToken , because in many case the json is data: T | T[] ,
and to handle those data, you can't use just dynamic because it hard to know what is the type of the object.
the JToken solve 99% of the serialize/deserialize problem , and we can "corrent" the JToken to right form (or add necessary other properties ), and then send to other place (eg. return to client or write to db or some message handler)

[steveharter]

I saw many people in this issue talk about the dynamic but what about the Writable JSON DOM ?

The proposal for 6.0 is to support for dynamic and non-dynamic cases with an overlapping but consistent programming model. Newtsoft does this as well since JToken implements IDynamicMetaObjectProvider which supports dynamic. This means if you don't want to use dynamic, you don't have to. In general, I would expect most consumers to not use dynamic since it is slower.

and we can "corrent" the JToken to right form

Can you elaborate on "correcting" to make sure we have the scenario. I assume you mean changing the DOM to modify a value of a property\element or by adding\removing properties\elements.

[John0King]

@steveharter

Can you elaborate on "correcting" to make sure we have the scenario. I assume you mean changing the DOM to modify a value of a property\element or by adding\removing properties\elements.

yes, for example change the data:T|T[] to data:T[] . I can't do it (or hard) with dynamic because I need to know the type of the property/object , and JToken can do this perfectly because every property/object have a TokenType property.

but today the JsonDocument JsonElement is just a StringReader or SpanReader (compare to newtonsoft.Json is new JsonTextReader(new StringReader(ref jsonString)) , just that today we can't pass by ref) ,
and it seems that this issue became to use dynamic to serialize and deserialize the object again and again, it will be way slower than JToken like "C# object" , and hard to detect it's type .

we don't need a string reader we need the "C# object" !

[keithn]

Seems weird C# doesn't have a standard library to manipulate JSON data, only serialize, deserialize from static types, or read only an arbitrary json document. Really need to be able to read, query and to manipulate JsonDocument, add elements, remove elements, etc and then turn it back to a string. The simple pain of getting any arbitrary piece of json and adding an extra element to it is astonishing. Think you guys need to implement a library that allows full manipulation of JSON data, not just a means to and end for deserializing/serializing static types.

[rs38]

in .NET there are 2 standards: 3rd party Nuget package Json.NET and .NET Core’s built-in classes System.Text.JSON which are very fast but a bit limited compared to Json.NET.

here is an example of manipulating a JSON file without deserializing with the built in Json classes:

using var json = File.OpenRead (jsonPath);
using JsonDocument document = JsonDocument.Parse (json);
var options = new JsonWriterOptions { Indented = true };
using (var outputStream = File.Create ("NewFile.json"))
using (var writer = new Utf8JsonWriter (outputStream, options))
{
 writer.WriteStartArray();
 foreach (var person in document.RootElement.EnumerateArray())
 {
 int friendCount = person.GetProperty ("Friends").GetArrayLength();
 if (friendCount >= 2)
 person.WriteTo (writer);
 }
}

(from: C# 8.0 in a Nutshell)

[udlose]

As promised, the PR providing a dynamic implementation sample is at #42097.

@steveharter - I'm unable to get your PR Code sample for supporting dynamic objects #42097 to load consistently - most of the time it won't load.

I also noticed that the example is missing a ToString() implementation override on JsonDynamicString - was this intentional? I'm planning on adding the override but don't want to rely on it if the final implementation in 6.0 won't use it.

[John0King]

Am I the only one who want a JSON POCO Object in C# (eg. JToken) , so we can manipulate the "object" easily and also other Json library can directly handle it too.

and @rs38 you solution is about write, and a mutable JSON DOM is about change and read (or write)

//  do some change to the original JSON
var jobj = xxxx.Parse(Request.Body) as JObject;
if(ShouldInjectClaim == true)
{
  
   jobj.Add("Claims", JArray.Parse(new []{  "claim1". "claim2" }));
}

//  handle the "data"
DoSomething(jobj);  // mostly read again

[ericstj]

@John0King I don't think you're the only person, that's part of what this issue is about. Have you seen dotnet/designs#163?

@udlose not sure why GitHub is having trouble loading the PR. You can always check out the commit: 095539f or even just browse to these tests that are now part of the product.

I also noticed that the example is missing a ToString() implementation override on JsonDynamicString - was this intentional?

I think that's just because the sample was minimal. I see @steveharter mentions this in the design doc:
https://github.com/dotnet/designs/pull/163/files#diff-34fc37119c1d4491b844207aeed3a9c846c3e2d835fd712e164b1e47b89687c0R77

[steveharter]

Closing; the JsonNode feature is in main and will be in Preview 4.

Preview 5 will also add some new capabilities for JsonObject including deterministic property ordering and being able to use JsonObject as an "extension property" via [JsonExtensionDataAttribute]. These are mentioned in the PR description.

[udlose]

@steveharter , @ericstj I pulled Steve's code sample for supporting dynamic from 095539f. It worked fine when I was running under .NET Core 3.1. I'm upgrading the app to .NET 5 and getting an error:

JsonSerializerExtensions.JsonDynamicType.Value is inaccessible due to its protection level

I can easily change these from internal to public but I'm concerned about why it broke when migrating to .NET 5. Is there a new version of the code sample that works for .NET 5?

[ericstj]

It looks to me like a compiler error around a identifier resolution, where something in another assembly is resolving to that Value member when it didn't before. It could be something unrelated to the update. If you have an issue to report (even against sample code) it's better to open a new issue and provide us with full details and a repro, that way we can better reproduce and diagnose what you are seeing.

Also, if you're willing to grab a nightly build you can try the official feature which is in as of last week: https://dev.azure.com/dnceng/public/_packaging?_a=package&feed=dotnet6&package=System.Text.Json&protocolType=NuGet&version=6.0.0-preview.5.21229.9

Thanks!

[techhowdy]

You can also try this

var obj = JsonDocument.Parse(Convert.ToString(your_dynamic_object));

 var propertyVal = obj.RootElement.GetProperty("Your_Property_Name").GetString();

[steveharter]

For those still interested in dynamic how valuable is that now that we have a writable DOM with JsonNode?

Please add your comments to this issue: #53195