fix!: deserialization of former required field

[shumkov]

Issue being fixed or feature implemented

Data Contract Update functionality allows developers to make previously required fields optional. This is required for application model evolution in case a developer wants to deprecate an old field so clients don't need to provide a value for this field. For example, a developer needs to change the field type. He/she needs to create a new field with a new type and mark the old field as optional (not required).

Currently, document serialization logic doesn't use the marker of presence for fields that are marked as required. Since the required flag can be removed with the data contract update, such serialized data becomes invalid.

What was done?

• Serialize all document fields with the marker of presence.
• Expect a marker of presence during deserialization.

How Has This Been Tested?

• Implemented a test when we serialize a document with the old schema, then update the schema and successfully deserialize the document.
• Implemented an integration test when we create a contract with required system fields, create a document, then update a contract and the previously created document is correctly deserialized. Then we update the document created with previous contract with updated contract and it works correctly as well. Creation of a new document with updated contract works as expected.

Breaking Changes

Previously serialized documents with required fields won't be valid.

Checklist:

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have added or updated relevant unit/integration/functional/e2e tests
• I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
• I have made corresponding changes to the documentation if needed

For repository code-owners and collaborators only

• I have assigned this pull request to a milestone

[QuantumExplorer]

We decided to postpone this till after release, for now required fields can not change to optional.