Skip to content

Comments

feat: add simpler etag handling to all non-critical endpoints#1836

Merged
CommanderStorm merged 32 commits intomaplibre:mainfrom
CommanderStorm:more-etag-handling
May 27, 2025
Merged

feat: add simpler etag handling to all non-critical endpoints#1836
CommanderStorm merged 32 commits intomaplibre:mainfrom
CommanderStorm:more-etag-handling

Conversation

@CommanderStorm
Copy link
Member

@CommanderStorm CommanderStorm commented May 19, 2025

This PR adds etag handling via an actix wrap instruction to all "non-tile" endpoints.
I ommitted some endpoints which I suspect are not called more than once such as / if the webui is not compiled in.

The reason for not being smarter for some is that honestly, the impact of this is less than I expected based on benchmarks in #1834 and that actix-middleware-etag uses the same hash algorithm.

I have not addded tests for this as frankly, this does not seem like a feature that needs deep tests. By request, this PR now includes all stemi-static headers.

This comment was marked as resolved.

@CommanderStorm CommanderStorm requested a review from nyurik May 19, 2025 12:34
@CommanderStorm
Copy link
Member Author

I don't super like the importing part as this makes headers::ETag and actix_middleware_etag::Etag indistinguisable, but am ok with doing it like this.

@CommanderStorm CommanderStorm changed the title feat: add etag handling to all non-critical endpoints feat: add simpler etag handling to all non-critical endpoints May 20, 2025
Copy link
Member

@nyurik nyurik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is a minor nit, and feel free to merge after addressing it

@sharkAndshark
Copy link
Collaborator

sharkAndshark commented May 21, 2025

I have not addded tests for this as frankly, this does not seem like a feature that needs deep tests

I think a really simple test for this is still necessary. Maybe next PR.

@CommanderStorm
Copy link
Member Author

I think a really simple test for this is still necessary.

I added a testcase for this. Whas what I added what you thought of?

@CommanderStorm CommanderStorm requested a review from nyurik May 21, 2025 10:06
@CommanderStorm CommanderStorm mentioned this pull request May 21, 2025
6 tasks
echo "Testing $(basename "$FILENAME") from $URL"
# jq before 1.6 had a different float->int behavior, so trying to make it consistent in all
$CURL "$URL" | jq --sort-keys -e 'walk(if type == "number" then .+0.0 else . end)' > "$FILENAME"
$CURL --dump-header "$FILENAME.headers" "$URL" | jq --sort-keys -e 'walk(if type == "number" then .+0.0 else . end)' > "$FILENAME"
Copy link
Collaborator

@sharkAndshark sharkAndshark May 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe do a test like we did under the martin/tests, and sure, it's good to test it in test.sh either.

But dump the header for each request it's not a good idea I guess...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at first I also thought headers might be too flaky, but than it seems headers only have the relatively stable things... with the possible exception of the length? i.e. in theory we could have a slightly different compression between different brotli implementation versions? So lets add it, and if we ever run into issues with the test flakiness, we could filter the header files to remove unstable headers?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

responding to this explicitely:

  • If length changes, then the etag changes as well.
  • If encoding changes, then the etag and length changes as well.

We cannot have both tests that

  • ensure that the etag does not change and
  • not check that the etag changes.

But since we have PRs for this kind of dependency changes, this should be fine.

This is not like the build in main being broken by new cippy lints every few months.

Copy link
Member

@nyurik nyurik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think this is good to go, thanks!

@CommanderStorm CommanderStorm merged commit 63e257b into maplibre:main May 27, 2025
20 checks passed
@CommanderStorm
Copy link
Member Author

Nice

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants