[server][common][vpj] Introduce ComplexVenicePartitioner to materialized view

[xunyin8]

[server][common][vpj] Introduce ComplexVenicePartitioner to materialized view

The change will not work if record is actually large and chunked. Proper chunking support is needed and will be addressed in a separate PR.

1. Introduced ComplexVenicePartitioner which extends VenicePartitioner and offer a new API to partition by value and provide possible one-to-many partition mapping.

2. Added value provider of type Lazy to VeniceViewWriter's processRecord API to access deserialized value if needed. e.g. when a ComplexVenicePartitioner is involved.

3. MergeConflictResultWrapper and WriteComputeResultWrapper will now provide deserialized value in a best effort manner. This is useful when we already deserialized the value for a partial update operation so that the deserialized value can be provided directly to the materialized view writer.

4. Refactored VeniceWriter to expose some APIs to child class. Introduced ComplexVeniceWriter which extends VeniceWriter. Reasoning here is that the ComplexVeniceWriter will have different APIs to be used in MaterializedViewWriter and CompositeVeniceWriter to write to materialized view partition(s) and potentially involving a ComplexVenicePartitioner. Alternatively we could push common logic from VeniceWriter to AbstractVeniceWriter. However, ComplexVeniceWriter needs/shares too much common logic with VeniceWriter (chunking, DIV support, pubSubAdapter, etc...) it will make AbstractVeniceWriter too specialized and unable to offer the flexibility it needs to support something like the CompositeVeniceWriter.

5. Override putLargeValue in ComplexVeniceWriter to skip chunking and writing large messages. Once we have proper chunking support we need to be careful to not re-chunk when writing the same value to different partition in ComplexVeniceWriter.

How was this PR tested?

Added new integration test with A/A, W/C and a new test value based partitioner.
Will add new unit tests once we have some consensus on the API changes

Does this PR introduce any user-facing changes?

• No. You can skip the rest of this section.
• Yes. Make sure to explain your proposed changes and call out the behavior change.

[FelixGV]

Just some early thoughts... did not read the whole PR yet. But hopefully useful in terms of discussing the API changes.

[gaojieliu]

Code change looks good overall.
I do think we need to take care of the comment I just left, which is very tricky as it is a race condition.

[gaojieliu]

readStoredValueRecord method would read the most-recent data, which means it will try to read the transient record cache first and then RocksDB.
For WC enabled store, delete will update the transient record to be null for the key and I think this method will always return null.

Even we perform the lookup before updating transient record cache, it will be wrong as it is a lazy function, so when ViewWriter tries to produce to view topics, it will still read the most recent value, which is null, and the situation will become worse when parallel compute for AA/WC workload is enabled as all the updates to the same key in the same batch will be executed (updating transient cache) before producing to version/view topics, which means for the delete operation, the lazy function can read the most-recent value, which can be populated by a later put in the same batch.

Can we do a non-lazy lookup always before finding out a more optimized solution?