Ser deser for discriminated union and polymorphic base

[qiaozha]

fixes #2139
fixes #1982

Summary

In this PR, we will support serialize/deserialize for discriminated union and polymorphic base, as we previously agree that we should not handle other unions because it's hard to predict the type as of the additional properties.

As such we will not generate predict function for unions, but we still need to distinguish which union variants are special, we only generate special handling logic for those special variants.

We also address the circular reference issue in this PR, as circular reference in inheritance and in combination could be handled differently

Special Union Variants / Subtypes

In general, we have two kinds of basic special union variants.

1. datetime type
2. binary array type

For model type, we have the following basic three types of special union variants.
3. model with property of datetime type.
4. model with property of binary array type.
5. model that has different property name between rest layer and modular layer.

To extend the combination, we get:
6. nested model i.e. model with property that is a model with one of the above 4-6 conditions.
7. model with property of special union.
8. An array type, with all the above 7 types as the element types.
9. A record type, with all the above 7 types as the value type. (TBD...)

Example for Discriminated Union

The typespec would be like

    @discriminator("kind")
    union WidgetData {
      kind0: WidgetData0;
      kind1: WidgetData1;
    }

    model WidgetData0 {
      kind: "kind0";
      fooProp: string;
    }
    
    model WidgetData1 {
      kind: "kind1";
      data: bytes;
    }

In this case, WidgetData0 is not a special union, but WidgetData1 is, we will not generate special serialize or deserialize logic for it. Our deserialize util would just be like

      /** deserialize function for WidgetData1 */
      function deserializeWidgetData1(obj: WidgetData1Output): WidgetData1 {
        return {
          kind: obj["kind"],
          data:
            typeof obj["data"] === "string"
              ? stringToUint8Array(obj["data"], "base64")
              : obj["data"],
        };
      }
      
      /** deserialize function for WidgetDataOutput */
      export function deserializeWidgetData(obj: WidgetDataOutput): WidgetData {
        switch (obj.kind) {
          case "kind1":
            return deserializeWidgetData1(obj);
          default:
            return obj;
        }
      }

Example for Polymorphic Base

For simplicity, in OpenAI's case, we have typespec like

@doc("An abstract representation of a structured content item within a chat message.")
@discriminator("type")
model ChatMessageContentItem {
  type: string;
}

model ChatMessageTextContentItem extends ChatMessageContentItem {
  type: "text";
  text: string;
}

model ChatMessageImageContentItem extends ChatMessageContentItem {
  type: "image_url";

  @projectedName("json", "image_url")
  imageUrl: ChatMessageImageUrl;
}

model ChatMessageImageUrl {
  @doc("The URL of the image.")
  url: url;

  detail?: ChatMessageImageDetailLevel; // ChatMessageImageDetailLevel is an normal enum
}

In this case, ChatMessageTextContentItem is not a special union variant, but ChatMessageImageContentItem is, our serialize util would be like

/** serialize function for ChatMessageImageContentItem */
function serializeChatMessageImageContentItem(
  obj: ChatMessageImageContentItem
): ChatMessageImageContentItemRest {
  return {
    type: obj["type"],
    image_url: { url: obj.imageUrl["url"], detail: obj.imageUrl["detail"] },
  };
}

/** serialize function for ChatMessageContentItemUnion */
export function serializeChatMessageContentItemUnion(
  obj: ChatMessageContentItemUnion
): ChatMessageContentItemRest {
  switch (obj.type) {
    case "image_url":
      return serializeChatMessageImageContentItem(
        obj as ChatMessageImageContentItem
      );
    default:
      return obj;
  }
}

Circular Reference Issue

1. Circular reference from inheritance

This is for typespec like

    @discriminator("kind")
    model Pet {
      kind: string;
      name: string;
      weight?: float32;
    }
    model Cat extends Pet {
      kind: "cat";
      meow: int32;
    }
    @discriminator("type")
    model Dog extends Pet {
      kind: "dog";
      type: string;
      bark: string;
    }
    model Gold extends Dog {
      type: "gold";
      friends: Pet[];
    }

In this case, Gold has a property which is type of his ancestors, but the overall model doesn't have anything special subtype, we will not generate the serialize and deserialize utils, if Gold has a birthday property of utcDateTime type, then we should generate the serialize utils for it.

It is possible that a discriminated union can also have circular reference, the logic should be the same, but ut TBD...

2. Circular reference from combination

This is for typespec like

      export interface Foo {
        name: string;
        weight?: number;
        bar: Bar;
      }
      
      export interface Bar {
        foo: Foo;
      }

In this case, Foo has a property point to Bar and Bar also has a property point to Foo, as this is out of the scope of discriminated union and polymorphic base, the current logic is, we will just detect if these models has anything special, if it's not, we will just give whatever we get to the Modular layer if it's for deserialize and to the Rest layer if it's for serialize. otherwse, we will generate as any. a case in point is ChatMessage in OpenAI's case.

LRO and Paging dependency issue

While integrate the latest OpenAI's typespec, I found they have a LRO operation that is defined in their typespec, but is removed in their typespec, which cause us need to generate LRO helper in RLC, but doesn't need to generate the LRO in Modular, as we currently don't really split RLC from Modular, we need to take both into consideration about whether we have a core-lro or core-paging dependency.

[qiaozha]

initial reviews

[qiaozha]

I will merge this PR and use #2253 to track other non-problematic union serialize and deserialize work. Let me know if anyone has any concerns.