Serialize Enum to underscored String by default

[straight-shoota]

This picks up #9905 and introduces a stricter default serialization plus a converter for optionally using the current, number-based serialization.

The new behaviour for Enum JSON/YAML serialization:

• Non-flag enums are strictly serialized to/from a string. Number is not accepted.
• Flag enums are strictly serialized to/from a list of strings. Number or list of numbers are not accepted.

The new type Enum::NumberOrStringConverter implements the previous behaviour:

• Both flag and non-flag enums are serialized to a number and serialize from a number or a string.

This type can be used with *::Serializable as a *::Field converter. It also provides a convenient API to use it directly (this could probably be generalized in a followup).

require "json"

class EnumStringMessage
  include JSON::Serializable

  enum Hint
    Up
    Down
  end

  @[Flags]
  enum Flag
    Left
    Right
  end

  property hint : Hint

  @[JSON::Field(converter: Enum::NumberOrStringConverter(EnumStringMessage::Hint))]
  property hint_number : Hint

  property flag : Flag

  @[JSON::Field(converter: Enum::NumberOrStringConverter(EnumStringMessage::Flag))]
  property flag_number : Flag
end

message = EnumStringMessage.from_json(%({"hint":"up", "hint_number": 1, "flag": ["left", "right"], "flag_number": "right"}))
message.hint        # => EnumStringMessage::Hint::Up
message.hint_number # => EnumStringMessage::Hint::Down
message.flag        # => EnumStringMessage::Flag::Left | EnumStringMessage::Flag::Right
message.flag_number # => EnumStringMessage::Flag::Right
message.to_json     # => %({"hint":"up","hint_number":1,"flag":["left","right"],"flag_number":2}})

# Use converter directly without JSON::Serializable:
Enum::NumberOrStringConverter(EnumStringMessage::Hint).from_json(%("down")) # => EnumStringMessage::Hint::Down
Enum::NumberOrStringConverter(EnumStringMessage::Hint).from_json(%(1))      # => EnumStringMessage::Hint::Down
Enum::NumberOrStringConverter(EnumStringMessage::Hint).to_json(:down)       # => %(1)

The example also applies to YAML serialization, using the appropriate methods and annotations.

Documentation is currently still missing, I'll add that later.

---

This PR also introduces two new public helper methods that I found convenient for the implementation:

• JSON::PullParser.raise is the equivalent to YAML::Nodes::Node#raise and offers a simple interface for raising JSON::ParseException at the current location
• YAML::Nodes::Node#type returns the name of the node type which is used in error messages to describe the actual token type (vs. the expected one).

Resolves #9905
Resolves #9881

I chose to open a new PR instead of amending the previous one in order to start a fresh review. Most of the discussion was about fleshing out the intended behaviour, which should have better been discussed in #9881.

Thanks to @caspiano for the first implementation 🙏

[asterite]

To me it's a bit strange that it's number or string for reading, but just number for writing. Maybe we should just let it work with numbers? I can't imagine why one would like to use sometimes strings and sometimes numbers for reading, but just numbers for writing. It should always be either always numbers or always strings.

If it's because of backwards compatibility, maybe that could be in a shard.

[straight-shoota]

Yes, the idea was to have a simple converter for an easy migration from the current behaviour. You can just plug in Enum::NumberOrStringConverter and it works.

I agree that a more strict number converter would probably be better. But we didn't have that until now, so I would see that as an additional feature.

If we accept a reduced backward compatibility, we could leave NumberOrStringConverter out and add a strict NumberConverter directly for the main use case.

[bcardiff]

I like the path to be more strict. I though that number or string was introduced as an option in the former PR.

Let's introduce Enum::NumberConverter and leave the default as proposed to string / List of string.

[straight-shoota]

Deserialization from number or string has been there since Enum.from_json was implemented. I wouldn't expect the string variant to be used much, because serialization was only as number.

So I'll change the converter to a strict NumberConverter.

If there's some actual demand, we could consider publishing NumberOrStringConverter as a shard or whatever, but it's really trivial to implement. And as I said, I don't expect much use anyways.

[straight-shoota]

I've changed the name to more generalized ValueConverter. I think that's better for describing the purpose.

[straight-shoota]

The two specs failures are probably caused by libyaml writing the document end marker only after some scalars. We really shouldn't be bothered by this, I'll just add an lchop("...\n") to always ignore the document end marker in the actual result.

[kostya]

Updating big project, quite hard to write every where converters like: Enum::ValueConverter(MyEnum) (because many of classes autogenerated by macros). Not understand why bad NumberOrStringConverter decoder, and String encoder?. Such thing Enum::ValueConverter(MyEnum) quite not good for autogenerators, because you should know type and is this type enum or not. Suggestion: add global struct/class Option:

 @[JSON::Serializable::Options(enums_as_value: true)]
 class A
    include JSON::Serializable
    @a : MyEnum?
 end

[straight-shoota]

Instead of using a converter on the serializable field, you could also change the implementation of the enum types themselves. Maybe that would be easier for your use case?
You'll just need to override .new(pull : JSON::PullParser) and #to_json(json : JSON::Builder) to the value implementation.

I don't think adding a global option to JSON::Serializable would be a good idea. It would add a huge lot of complexity for a single use case.