Update CQL2 to the finalized specification

[aaime]

If we want to have CQL2 filtering as part of the extension we'll have to update the GeoTools implementation of cql2-text and cql2-json to support the finalized specification.

I cannot find a CITE test, but the ATS for the specification assumes a specific version of Natural Earth 110 million that is delivered as a 1MB GeoPackage, and provides many sample filters with the expected amount of matching features.

A few notes:

• It's a ton of tests
• The examples are given only as cql2-text (not cql2-json)
• We don't support all the conformance classes (e.g., don't support accent removal for example, nor array comparison
• Adding a 1MB payload in our git repositories is a bit big ... maybe it could be an online test downloading the test data on demand (1MB download is not that much).
• JSON schema for tests is here
• Supporting CASEI/ACCENTI/array comparisons, and more in general, anything that is not in our existing filtering model, is considered a stretch goal, only to be worked on at the end.

[pmauduit]

I created a working branch for the test, which adds 330 tests on each modules (cql2-text & cql2-json), I also renamed the former ENVELOPE keyword to the expected BBOX one, current diff against main is here:
geotools/geotools@main...pmauduit:geotools:cql2-text-conformance-testsuite

As for now,

• 205 out of the 330 tests pass on cql2-text
• 184 out of 330 on cql2-json

in its current implementation, geotools is not able to parse some of the provided filters, as a result we cannot translate them into json version of the filter. For these ones I am currently using an empty json object as input for the test, which will make the test fail.

I am confused by some of the expected results provided by the spec though, the following filter is expected to return 2 features:

"date"<>DATE('2022-04-16')

Our tests is returning more than that because on the NaturalEarth dataset, a lot of dates are left empty (NULL). Considering that NULL is different from the '2022-04-16' date, we also should return the elements having a NULL date, contrary to what the specifications currently state in table 7 of https://docs.ogc.org/is/21-065r2/21-065r2.html#_conformance_test_8. I will open an issue onto the dedicated OGC repository to clarify this.

[pmauduit]

Setting the TZ=GMT environment variable (honoured by the libsqlite-C implementation used by xerial's sqlite-jdbc java library, used by geotools to access the geopackage), it gives:

• gt-cql2-text: 231 tests / 330
• gt-cql2-json: 210 / 330

[groldan]

@pmauduit @aaime moved this ticket to the done column, as merged on geotools/geotools@e58d016
Let me know if there's something else to do for it

[aaime]

See the draft graduation e-mail for a list of functionality that could be addressed by expanding the javacc parser (and eventually implementing functions to support missing filters):
#18 (comment)

However, IMHO, it's time to focus on shipping rather than continue to implement new stuff.