Implement native ESRI reader

[ljw9111]

Description

This PR implements the native ESRI reader for reading Esri JSON which can be used for geospatial queries. (NOTE: we only support UTC timezone in this port)

Customer can now submit geospatial query on a table using ESRI serde.

DDL example

CREATE external TABLE earthquakes
(
 earthquake_date string,
 latitude double,
 longitude double,
 depth double,
 magnitude double,
 magtype string,
 mbstations string,
 gap string,
 distance string,
 rms string,
 source string,
 eventid string
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
STORED AS TEXTFILE LOCATION 's3://amzn-s3-demo-bucket/my-query-log/csv/';

CREATE external TABLE IF NOT EXISTS counties
 (
 Name string,
 BoundaryShape binary
 )
ROW FORMAT SERDE 'com.esri.hadoop.hive.serde.EsriJsonSerDe'
STORED AS INPUTFORMAT 'com.esri.json.hadoop.EnclosedEsriJsonInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION 's3://amzn-s3-demo-bucket/my-query-log/json/';

Example data is from https://docs.aws.amazon.com/athena/latest/ug/geospatial-example-queries.html

DML example

trino:esri> SELECT c.name,
         ->         COUNT(*) cnt
         -> FROM esri.counties as c
         -> CROSS JOIN esri.earthquakes
         -> WHERE ST_CONTAINS (geometry_from_hadoop_shape(c.boundaryshape), ST_POINT(earthquakes.longitude, earthquakes.latitude))
         -> GROUP BY  c.name
         -> ORDER BY  cnt DESC;
      name       | cnt 
-----------------+-----
 Kern            | 288 
 San Bernardino  | 280 
 Imperial        | 224 
 Inyo            | 160 
 Los Angeles     | 144 
 Monterey        | 112 
 Riverside       | 112 
 Santa Clara     |  96 
 Fresno          |  88 
 San Benito      |  88 
 San Diego       |  56 
 Santa Cruz      |  40 
 San Luis Obispo |  24 
 Ventura         |  24 
 Orange          |  16 
 San Mateo       |   8 
(16 rows)

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

## Hive connector
* Add support for reading tables using the ESRI JSON format ({issue}`25241`)

[findinpath]

After skimming the code and trying TestEsri I definitely understand the purpose of this contribution.

The referenced library geometry-api-java seems not lively anymore.

It would be useful to have a test reading all types.

However before adding any other changes, I think it is worth asking the maintainers @wendigo , @dain whether this contribution is basically fit from a functional perspective to be inclued in the Trino project code.

[dain]

I spent a good while reviewing this. Overall I think the approach is sound, but the code is missing defenses against bad data files. In this code we should strive to be bug-for-bug compatible with hive, and this includes handling of "bad" files, because users often rely on these undocumented behaviors.

Additionally, Jackson has some unexpected behaviors when recursing into nested structures, and this code falls into that trap. Specifically, the code isn't properly skipping nexted data which can result in processing inside of objects that is not expected (I had to learn this the hard way a couple of years back). In general, I used (copied) the framework laid out in the Json reader, which handles these issues.

Instead of adding a lot of mundane comments, I just applied them to the code which you can find in this commit dain@8b95a73

Finally, the tests seem to be missing cases for some of the supported attribute types... I see then when running the tests with coverage.

[github-actions]

This pull request has gone a while without any activity. Ask for help on #core-dev on Trino slack.

[github-actions]

Closing this pull request, as it has been stale for six weeks. Feel free to re-open at any time.

[ljw9111]

@dain Thank you so much for the review and update! I just pulled your commit and made new revision with additional unit test called testSupportedAttributeTypes and fixing assertGeometry in TestEsriDeserializer.java a little bit. Could you review it when you get chance?

My comments have been integrated. James will take over the for the final review.

[pettyjamesm]

Approved, thanks @ljw9111!