arrow_json: support binary deserialization

[Folyd]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

I have a bunch of parquet files that contain a binary column, here is the schema:

CREATE EXTERNAL TABLE `buildings`(
  `id` string,
  `updateTime` string,
  `version` int,
  `names` string,
  `level` int,
  `height` double,
  `geometry` binary
)

The geometry column is binary. Here is my struct:

#[derive(Deserialize)]
struct Building {
    // omit for brevity ...

    #[serde(default)]
    geometry: Vec<u8>,
}

However, it failed to deserialize the parquet data with arrow::json::writer::record_batches_to_json_rows(), here is the error:

Error: Json error: data type Binary not supported in nested map for json writer

Describe the solution you'd like

arrow_json should support deserialize binary to Vec<u8>.

[tustvold]

So if I understand correctly you are reading binary data from a parquet file and are trying to write it to JSON?

Unfortunately JSON does not define a mechanism to transport binary data, you have two common options:

• If the binary data is actually UTF-8 you could cast the column to a StringArray
• You could use a crate like base64 to base64 encode the binary data prior to writing it

[Folyd]

In my example, the geometry is actually a wkb (well-known-binary) type. Does arrow support wkb?

Here is an example of wkb type, it can be decoded to GeoJson:

[tustvold]

Arrow supports WKB, but JSON would only support WKT

[tustvold]

label_issue.py automatically added labels {'arrow'} from #4967