Skip to content

Commit d6f3759

Browse files
committed
Document Protobuf oneof support in Kafka connector
1 parent 02cc332 commit d6f3759

File tree

1 file changed

+42
-0
lines changed

1 file changed

+42
-0
lines changed

docs/src/main/sphinx/connector/kafka.rst

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -418,6 +418,7 @@ table description supplier are:
418418
* New tables can be defined without a cluster restart.
419419
* Schema updates are detected automatically.
420420
* There is no need to define tables manually.
421+
* Some Protobuf specific types like ``oneof`` are supported and mapped to JSON.
421422

422423
Set ``kafka.table-description-supplier`` to ``CONFLUENT`` to use the
423424
schema registry. You must also configure the additional properties in the following table:
@@ -479,6 +480,45 @@ used to resolve the subject name via the topic name. Note that a case
479480
insensitive match must be done, as identifiers cannot contain upper case
480481
characters.
481482

483+
Protobuf-specific type handling in Confluent table description supplier
484+
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
485+
486+
When using the Confluent table description supplier, the following Protobuf
487+
specific types are supported in addition to the :ref:`normally supported types
488+
<kafka-protobuf-decoding>`:
489+
490+
oneof
491+
+++++
492+
493+
Protobuf schemas containing ``oneof`` fields are mapped to a ``JSON`` field in
494+
Trino.
495+
496+
For example, given the following Protobuf schema:
497+
498+
.. code-block:: text
499+
500+
syntax = "proto3";
501+
502+
message schema {
503+
oneof test_oneof_column {
504+
string string_column = 1;
505+
uint32 integer_column = 2;
506+
uint64 long_column = 3;
507+
double double_column = 4;
508+
float float_column = 5;
509+
bool boolean_column = 6;
510+
}
511+
}
512+
513+
The corresponding Trino row is a ``JSON`` field ``test_oneof_column``
514+
containing a JSON object with a single key. The value of the key matches
515+
the name of the ``oneof`` type that is present.
516+
517+
In the above example, if the Protobuf message has the
518+
``test_oneof_column`` containing ``string_column`` set to a value ``Trino``
519+
then the corresponding Trino row includes a column named
520+
``test_oneof_column`` with the value ``JSON '{"string_column": "Trino"}'``.
521+
482522
.. _kafka-sql-inserts:
483523

484524
Kafka inserts
@@ -1350,6 +1390,8 @@ The schema evolution behavior is as follows:
13501390
If the type coercion is supported by Avro, then the conversion happens. An
13511391
error is thrown for incompatible types.
13521392

1393+
.. _kafka-protobuf-decoding:
1394+
13531395
Protobuf decoder
13541396
""""""""""""""""
13551397

0 commit comments

Comments
 (0)