Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Named Fields and Align Structures to Reduce Memory Usage #1814

Merged
merged 3 commits into from
Aug 2, 2024

Conversation

gaby
Copy link
Contributor

@gaby gaby commented Jul 28, 2024

@erikdubbelboer Not sure your thoughts on this since it changes the order of fields in some of the public structs.

Affected Public Structs:

  • Client
  • HostClient
  • Cookie
  • URI
  • ErrDialWithUpstream
  • TCPDialer
  • RequestCtx
  • Response
  • Request
  • Server
  • Prefork
  • LBClient

This is something we ran into in Fiber gofiber/fiber#3079, we had a lot of places in the code base using positional instead of named fields. This can cause major bugs in the future when the underlying struct changes. I wrote an analyzer to detect and fix these issues. After fixing those, I ran betteralign against the whole code base.

Summary of savings:

/fasthttp/fasthttputil/inmemory_listener.go:16:23: 16 bytes saved: struct with 40 pointer bytes could be 24
/fasthttp/fasthttputil/pipeconns.go:44:16: 24 bytes saved: struct with 312 pointer bytes could be 288
/fasthttp/fasthttputil/pipeconns.go:95:15: 24 bytes saved: struct with 128 pointer bytes could be 104
/fasthttp/stackless/writer.go:41:13: 8 bytes saved: struct with 72 pointer bytes could be 64
/fasthttp/client.go:175:13: 144 bytes saved: struct with 288 pointer bytes could be 144
/fasthttp/client.go:684:17: 144 bytes saved: struct with 368 pointer bytes could be 224
/fasthttp/client.go:951:24: 24 bytes saved: struct with 48 pointer bytes could be 24
/fasthttp/client.go:2034:15: 8 bytes saved: struct with 40 pointer bytes could be 32
/fasthttp/client.go:2089:20: 8 bytes saved: struct with 40 pointer bytes could be 32
/fasthttp/client.go:2167:21: 72 bytes saved: struct with 144 pointer bytes could be 72
/fasthttp/client.go:2281:25: 88 bytes saved: struct with 208 pointer bytes could be 120
/fasthttp/client.go:2312:19: 24 bytes saved: struct with 1288 pointer bytes could be 1264
/fasthttp/cookie.go:67:13: 32 bytes saved: struct with 208 pointer bytes could be 176
/fasthttp/fs.go:253:9: 48 bytes saved: struct with 160 pointer bytes could be 112
/fasthttp/fs.go:504:16: 24 bytes saved: struct with 160 pointer bytes could be 136
/fasthttp/fs.go:522:13: 24 bytes saved: struct with 184 pointer bytes could be 160
/fasthttp/fs.go:847:27: 8 bytes saved: struct with 32 pointer bytes could be 24
/fasthttp/header.go:26:21: 32 bytes saved: struct with 304 pointer bytes could be 272
/fasthttp/header.go:61:20: 24 bytes saved: struct with 344 pointer bytes could be 320
/fasthttp/header.go:3233:20: 16 bytes saved: struct with 88 pointer bytes could be 72
/fasthttp/http.go:38:14: 8 bytes saved: struct with 776 pointer bytes could be 768
/fasthttp/http.go:89:15: 24 bytes saved: struct with 416 pointer bytes could be 392
/fasthttp/lbclient.go:27:15: 8 bytes saved: struct with 48 pointer bytes could be 40
/fasthttp/peripconn.go:9:23: 8 bytes saved: struct with 96 pointer bytes could be 88
/fasthttp/peripconn.go:41:16: 8 bytes saved: struct with 32 pointer bytes could be 24
/fasthttp/peripconn.go:48:19: 8 bytes saved: struct with 24 pointer bytes could be 16
/fasthttp/server.go:148:13: 8 bytes saved: struct of size 560 could be 552
/fasthttp/server.go:585:17: 48 bytes saved: struct with 1440 pointer bytes could be 1392
/fasthttp/tcpdialer.go:128:16: 24 bytes saved: struct with 80 pointer bytes could be 56
/fasthttp/tcpdialer.go:374:26: 8 bytes saved: struct with 32 pointer bytes could be 24
/fasthttp/tcpdialer.go:398:19: 24 bytes saved: struct with 56 pointer bytes could be 32
/fasthttp/uri.go:42:10: 8 bytes saved: struct with 280 pointer bytes could be 272
/fasthttp/workerpool.go:17:17: 56 bytes saved: struct with 144 pointer bytes could be 88
/fasthttp/examples/client/client.go:20:13: 8 bytes saved: struct with 16 pointer bytes could be 8
/fasthttp/fasthttpadaptor/adaptor.go:90:28: 8 bytes saved: struct with 40 pointer bytes could be 32
/fasthttp/prefork/prefork.go:44:14: 16 bytes saved: struct with 96 pointer bytes could be 80
/fasthttp/prefork/prefork.go:166:15: 8 bytes saved: struct with 24 pointer bytes could be 16
/fasthttp/client_test.go:1948:22: 8 bytes saved: struct with 40 pointer bytes could be 32
/fasthttp/client_test.go:3213:13: 8 bytes saved: struct with 48 pointer bytes could be 40
/fasthttp/client_test.go:3346:13: 16 bytes saved: struct with 72 pointer bytes could be 56
/fasthttp/client_test.go:3404:13: 8 bytes saved: struct with 32 pointer bytes could be 24
/fasthttp/client_timing_test.go:19:21: 24 bytes saved: struct with 56 pointer bytes could be 32
/fasthttp/server_test.go:2653:20: 8 bytes saved: struct with 16 pointer bytes could be 8
/fasthttp/server_test.go:4351:17: 8 bytes saved: struct with 16 pointer bytes could be 8
/fasthttp/server_timing_test.go:252:19: 40 bytes saved: struct with 64 pointer bytes could be 24

Total savings: 1336 bytes

@gaby
Copy link
Contributor Author

gaby commented Jul 28, 2024

I'm not sure why the CI golangci-lint is failing, it runs fine locally.

image

@gaby
Copy link
Contributor Author

gaby commented Jul 28, 2024

@valyala Thoughts on these changes?

@byte0o
Copy link
Contributor

byte0o commented Jul 30, 2024

我不确定为什么 CI golangci-lint 失败了,它在本地运行良好。

图像

@gaby ci error message :
tcpdialer.go:129: File is not gofumpt-ed (gofumpt)
prefork/prefork.go:45: File is not gofumpt-ed (gofumpt)

This is due to the use of https://github.com/mvdan/gofumpt to implement a stricter fmt. Delete the blank lines of tcpdialer.go:129 and prefork/prefork.go:45 should be able to solve the problem.

@gaby
Copy link
Contributor Author

gaby commented Jul 30, 2024

我不确定为什么 CI golangci-lint 失败了,它在本地运行良好。
图像

@gaby ci error message : tcpdialer.go:129: File is not gofumpt-ed (gofumpt) prefork/prefork.go:45: File is not gofumpt-ed (gofumpt)

This is due to the use of mvdan/gofumpt to implement a stricter fmt. Delete the blank lines of tcpdialer.go:129 and prefork/prefork.go:45 should be able to solve the problem.

Yeah, I had ran it and it wasn't removing them. I removed them manually. Should work now. Thanks

@erikdubbelboer erikdubbelboer merged commit 1fb3453 into valyala:master Aug 2, 2024
19 checks passed
@erikdubbelboer
Copy link
Collaborator

Thanks!

@gaby gaby deleted the mem-usage branch August 2, 2024 21:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants