Skip to content

Support protobuf build-in Value within c#15384

Closed
mx123 wants to merge 25 commits intotrinodb:masterfrom
mx123:kafka_protobuf_buildin_value
Closed

Support protobuf build-in Value within c#15384
mx123 wants to merge 25 commits intotrinodb:masterfrom
mx123:kafka_protobuf_buildin_value

Conversation

@mx123
Copy link
Copy Markdown
Contributor

@mx123 mx123 commented Dec 13, 2022

Description

Protobuf Google build-in com.google.protobuf.Value should be supported within Kafka Schema Registry.

Additional context and related issues

see Prevent Kafka protobuf schema registry message parsing dead loop recursion PR for details.

dain and others added 17 commits December 11, 2022 17:32
When refresh token is retrieved for UI, currently we were sending
HTTP Status 303, assuming that all the request will just repeat the
call on the Location header. When this works for GET/PUT verbs, it does
not for non-idempotent ones like POST, as every js http client should
do a GET on LOCATION after 303 on POST. Due to that I change it to 307, that
should force every client to repeat exactly the same request,
no matter the verb.

Co-authored-by: s2lomon <s2lomon@gmail.com>
Actual work is done in `pageProjectWork.process()` call while
`projection.project` only performs setup of projection.
So both `expressionProfiler` and `metrics.recordProjectionTime`
needed to be around that method.
Removes outdated comments and unnecessary methods in local exchange
PartitioningExchanger since the operator is no longer implemented
in a way that attempts to be thread-safe.
- Change ColumnHandle to BigQueryColumnHandle in BigQueryTableHandle
- Extract buildColumnHandles in BigQueryClient
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Json object convert is doing by very naive way to open discussion how to better implement it.
cc: @kokosing, @Praveen2112, @findepi

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally would prefer to IF on the source type ( whether it's https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#value )

but i don't know Kafka connector conventions

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it initiates to recognise as Value object here . After that registry's schema parsed there are it has only defined type and some value encapsulated to DynamicMessage. To know what's the exact type passing it's need to be converted anyway by json formatter.

findinpath and others added 6 commits December 14, 2022 14:31
Kafka protobuf registry schema cannot be translated to plain
Trino SQL structure in common cases when included data types use
referencing to same objects recursively.
Message sending to Kafka thows NPE if proto file has `import` derective.
Kafka protobuf schema registry parsing falls to dead loop if has included references to the same object.
It's need to abort the recursion loop with appropriated parsing error.
@mx123
Copy link
Copy Markdown
Contributor Author

mx123 commented Feb 10, 2023

closed after offline discussion with @hashhar

@mx123 mx123 closed this Feb 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.