Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for vector tiles API #1652

Open
iverase opened this issue Jan 13, 2023 · 6 comments
Open

Add support for vector tiles API #1652

iverase opened this issue Jan 13, 2023 · 6 comments
Labels
enhancement Improves the status quo tech debt

Comments

@iverase
Copy link

iverase commented Jan 13, 2023

It would be great if we could add operations in rally that points to the vector tiles API. It currently fails because the result of the query is a protbuffer that the system does not understand.

@craigtaverner craigtaverner self-assigned this Jan 18, 2023
@pquentin
Copy link
Member

The main blocker for this issue is migrating to version 8.x of the Python client in #1350 as version 7.14 of the Elasticsearch Python client does not support that API. Support was added in 7.15.0: https://www.elastic.co/guide/en/elasticsearch/client/python-api/7.17/release-notes.html#rn-7-15-0. This API is quite specific in that it returns binary data, so it could make sense to wait for the upgrade, which hopefully should be done in March.

@craigtaverner craigtaverner removed their assignment Jan 24, 2023
@pquentin
Copy link
Member

pquentin commented Jul 7, 2023

@craigtaverner @iverase We released Rally 2.8.0 last month with the 8.x client. Are you still interested in doing that?

@iverase
Copy link
Author

iverase commented Jul 7, 2023

Yes we are super interested, need to find the time (and in my case, some python skills :))

@pquentin
Copy link
Member

pquentin commented Jul 7, 2023

Can you maybe write how such an operation would look in a Rally track, so that we can see how difficult it would be to implement? Some runners are only a few lines of code.

@iverase
Copy link
Author

iverase commented Jul 11, 2023

A mvt request is very similar to a search request except it requires 4 mandatory parameters. One is the field where the geo data is, and the other three is the actual tile to be processed by providing the z/x/y values.

Therefore a simple example would look like:

 {
      "name": "mvt-search",
      "operation-type": "mvt",
       "z" : 11,
       "x" : 23,
       "y" : 145,
       "field": "my-geo-field",
      "body": {
           -- extra configuration here 
      }
    }

@iverase
Copy link
Author

iverase commented Jul 12, 2023

An example of how this would look like in the geoshape track is:

{
      "name": "mvt-search-hits",
      "operation-type": "mvt",
      "index": "osm*",
       "field": "shape",
       "z" : 2,
       "x" : 2,
       "y" : 1,
       "field": "my-geo-field",
      "body": {
           "grid_precision" : 0,
           "sort": [
               {
                 "size": {
                  "order": "desc"
                }
              }
           ]
      }
    }

or another example:

{
      "name": "mvt-search-grid",
      "operation-type": "mvt",
      "index": "osm*",
       "field": "shape",
       "z" : 2,
       "x" : 2,
       "y" : 1,
      "body": {
           "size" : 0,
            "grid_agg" : "geo_hex"
      }
    }

@ebadyano ebadyano added enhancement Improves the status quo tech debt labels May 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Improves the status quo tech debt
Projects
None yet
Development

No branches or pull requests

4 participants