🚀 Feature: Add idempotency middleware #2253

leonklingele · 2022-12-03T04:46:47Z

Description

Add idempotency middleware.
Implements #2163.

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

Checklist:

For new functionalities I follow the inspiration of the express js framework and built them similar in usage
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation - https://github.com/gofiber/docs for https://docs.gofiber.io/
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes
If new dependencies exist, I have checked that they are really necessary and agreed with the maintainers/community (we want to have as few dependencies as possible)
I tried to make my code as fast as possible with as few allocations as possible
For new code I have written benchmarks so that they can be analyzed and improved

middleware/idempotency/storage.go

GalvinGao

I would argue the same as in the conversation https://github.com/gofiber/fiber/pull/2253/files#r1038919642 to have consistency with fiber.Storage. Otherwise wonderful work!

efectn · 2022-12-04T09:15:20Z

Why did you target v3-beta branch? I think this can be a part of v2.

Also apply PR template into your PRs please.

leonklingele · 2022-12-04T16:46:33Z

@efectn I'll backport this to v2 once merged.

leonklingele · 2022-12-04T23:53:52Z

Soo, finally this is now ready for review. I removed the Storage interface, switched to msgp for marshalling, and improved some other things.

leonklingele · 2022-12-15T17:02:05Z

Let's make this go in v3 first and I'll open up another PR targeting v2 later on.

leonklingele · 2023-01-04T18:47:33Z

Anyone gotten the chance to review this already? I'm working on a project which might benefit from having this.

ReneWerner87 · 2023-01-04T19:38:43Z

Will do it tomorrow

ReneWerner87

implementation looks very good

a few small comments/improvements

benchmark for the middleware usage would be good, so we can improve it later on
instead of the clousure i would maybe try to use a function that doesn't need to be created on-the-fly every time
with the locals thing i'm not quite sure, what happens e.g. if you define the idempotency middleware multiple times, normally there shouldn't be any collisions as they should be used on different routes then ... currently quite not sure with that
in some places we can optimize and save allocations, like in the ToLower( there exists a fn which does not cause allocations) and other places where only comparisons are done
-> for comparisons no copies need to be made, only if the data should be kept

ReneWerner87 · 2023-01-05T12:34:40Z

@leonklingele will wait for the others for the feedback and then i merge it

@efectn @GalvinGao do you have any other points, otherwise i would merge the pull request

leonklingele · 2023-01-05T15:00:55Z

Thanks for the review! I’ll push the updates later today. Which calls can be optimized? I’ve only found the single call to strings.ToLower used when doing the map lookup which we could replace by utils.ToLower Also, did you see my question regarding using utils.CopyBytes when storing the request body? I think we need to use it.

ReneWerner87 · 2023-01-05T15:08:56Z

Yes use the copy there, i was not sure about this

GalvinGao

lgtm!

…ng unused methods

The real "strings.ToLower" still needs to be used when storing the data.

leonklingele · 2023-01-06T00:34:19Z

I've rebased & addressed @ReneWerner87's points. ptal

* middleware: add idempotency middleware * middleware/idempotency: use fiber.Storage instead of custom storage * middleware/idempotency: only allocate data if really required * middleware/idempotency: marshal response using msgp * middleware/idempotency: add msgp tests * middleware/idempotency: do not export response * middleware/idempotency: disable msgp's -io option to disable generating unused methods * middleware/idempotency: switch to time.Duration based app.Test * middleware/idempotency: only create closure once * middleware/idempotency: add benchmarks * middleware/idempotency: optimize strings.ToLower when making comparison The real "strings.ToLower" still needs to be used when storing the data. * middleware/idempotency: safe-copy body

* 🚀 Feature: Add idempotency middleware (#2253) * middleware: add idempotency middleware * middleware/idempotency: use fiber.Storage instead of custom storage * middleware/idempotency: only allocate data if really required * middleware/idempotency: marshal response using msgp * middleware/idempotency: add msgp tests * middleware/idempotency: do not export response * middleware/idempotency: disable msgp's -io option to disable generating unused methods * middleware/idempotency: switch to time.Duration based app.Test * middleware/idempotency: only create closure once * middleware/idempotency: add benchmarks * middleware/idempotency: optimize strings.ToLower when making comparison The real "strings.ToLower" still needs to be used when storing the data. * middleware/idempotency: safe-copy body * middleware/idempotency: backport to v2

leonklingele force-pushed the add-idempotency-middleware branch 6 times, most recently from 5593988 to 979bbe3 Compare December 3, 2022 05:16

efectn reviewed Dec 3, 2022

View reviewed changes

middleware/idempotency/storage.go Outdated Show resolved Hide resolved

leonklingele force-pushed the add-idempotency-middleware branch 8 times, most recently from 5dd8b22 to 4372dd9 Compare December 3, 2022 15:33

leonklingele marked this pull request as ready for review December 3, 2022 15:41

leonklingele mentioned this pull request Dec 3, 2022

🚀 [Feature]: Idempotency Middleware #2163

Closed

3 tasks

leonklingele force-pushed the add-idempotency-middleware branch from 4372dd9 to f242f13 Compare December 3, 2022 16:54

GalvinGao suggested changes Dec 4, 2022

View reviewed changes

efectn added the ✏️ Feature label Dec 4, 2022

leonklingele changed the title ~~middleware: add idempotency middleware~~ 🚀 Feature: Add idempotency middleware Dec 4, 2022

leonklingele force-pushed the add-idempotency-middleware branch 6 times, most recently from 7dbc78a to 7912335 Compare December 4, 2022 23:45

ReneWerner87 approved these changes Jan 5, 2023

View reviewed changes

GalvinGao approved these changes Jan 5, 2023

View reviewed changes

leonklingele added 12 commits January 6, 2023 01:26

middleware: add idempotency middleware

1536c98

middleware/idempotency: use fiber.Storage instead of custom storage

2dd0ca2

middleware/idempotency: only allocate data if really required

35fe663

middleware/idempotency: marshal response using msgp

e334f81

middleware/idempotency: add msgp tests

71a6b76

middleware/idempotency: do not export response

f718470

middleware/idempotency: disable msgp's -io option to disable generati…

4efb162

…ng unused methods

middleware/idempotency: switch to time.Duration based app.Test

3c1ca89

middleware/idempotency: only create closure once

18f7d8b

middleware/idempotency: add benchmarks

25650c0

middleware/idempotency: optimize strings.ToLower when making comparison

a109827

The real "strings.ToLower" still needs to be used when storing the data.

middleware/idempotency: safe-copy body

dc6cfb8

leonklingele force-pushed the add-idempotency-middleware branch from f334ff0 to dc6cfb8 Compare January 6, 2023 00:33

efectn approved these changes Jan 6, 2023

View reviewed changes

ReneWerner87 approved these changes Jan 6, 2023

View reviewed changes

ReneWerner87 merged commit 0b5a7d0 into gofiber:v3-beta Jan 6, 2023

ReneWerner87 linked an issue Jan 6, 2023 that may be closed by this pull request

🚀 [Feature]: Idempotency Middleware #2163

Closed

3 tasks

leonklingele mentioned this pull request Jan 6, 2023

🚀 Feature: Add idempotency middleware (v2 backport) #2288

Merged

13 tasks

leonklingele mentioned this pull request Apr 2, 2023

🚀 [Feature]: Split fiber.Storage to multiple interfaces #2399

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🚀 Feature: Add idempotency middleware #2253

🚀 Feature: Add idempotency middleware #2253

leonklingele commented Dec 3, 2022 •

edited

Loading

GalvinGao left a comment •

edited

Loading

efectn commented Dec 4, 2022

leonklingele commented Dec 4, 2022

leonklingele commented Dec 4, 2022

leonklingele commented Dec 15, 2022

leonklingele commented Jan 4, 2023

ReneWerner87 commented Jan 4, 2023

ReneWerner87 left a comment

ReneWerner87 commented Jan 5, 2023

leonklingele commented Jan 5, 2023 via email

ReneWerner87 commented Jan 5, 2023

GalvinGao left a comment

leonklingele commented Jan 6, 2023

🚀 Feature: Add idempotency middleware #2253

🚀 Feature: Add idempotency middleware #2253

Conversation

leonklingele commented Dec 3, 2022 • edited Loading

Description

Type of change

Checklist:

GalvinGao left a comment • edited Loading

Choose a reason for hiding this comment

efectn commented Dec 4, 2022

leonklingele commented Dec 4, 2022

leonklingele commented Dec 4, 2022

leonklingele commented Dec 15, 2022

leonklingele commented Jan 4, 2023

ReneWerner87 commented Jan 4, 2023

ReneWerner87 left a comment

Choose a reason for hiding this comment

ReneWerner87 commented Jan 5, 2023

leonklingele commented Jan 5, 2023 via email

ReneWerner87 commented Jan 5, 2023

GalvinGao left a comment

Choose a reason for hiding this comment

leonklingele commented Jan 6, 2023

leonklingele commented Dec 3, 2022 •

edited

Loading

GalvinGao left a comment •

edited

Loading