-
Notifications
You must be signed in to change notification settings - Fork 43
[Backend] refactor: 增加json模块 #1152
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Labels
backlog
需求初始状态,等待产品进行评估
Layer: Backend
bk-iam backend project
Priority: High
Type: Enhancement
New feature or request
Comments
Closed
jsoniter的MarshalToString 封装的是 所以, 如果想要透明, 需要实现相应的 |
压测代码: var table = []struct {
jsonExpr string
}{
{jsonExpr: `{"StringEquals":{"bk_cmdb.biz.id":["100605"]}}`},
{
jsonExpr: `{"AND":{"content":[{"StringEquals":{"bk_bcs_app.namespace.id":["15171:d7a4aa53ie"]}},{"StringPrefix":{"bk_bcs_app.namespace._bk_iam_path_":["/project,b37778ec757544868a01e1f01f07037f/cluster,BCS-K8S-15171/"]}}]}}`,
},
{
jsonExpr: `{"OR":{"content":[{"AND":{"content":[{"StringEquals":{"bk_bcs_app.namespace.id":["15091:772cdb3dhi","15091:11c90639hi","15091:8f07ea69hi","15091:549be245hi","15091:36da8568te","15091:97b04a4chi"]}},{"StringPrefix":{"bk_bcs_app.namespace._bk_iam_path_":["/project,b37778ec757544868a01e1f01f07037f/cluster,BCS-K8S-15091/"]}}]}},{"AND":{"content":[{"StringEquals":{"bk_bcs_app.namespace.id":["15113:772cdb3dhi","15113:e08c977fhi","15113:9700d8bfhi","15113:d849d60cte","15113:522ffd1dte","15113:2678b8fehi"]}},{"StringPrefix":{"bk_bcs_app.namespace._bk_iam_path_":["/project,b37778ec757544868a01e1f01f07037f/cluster,BCS-K8S-15113/"]}}]}},{"AND":{"content":[{"StringEquals":{"bk_bcs_app.namespace.id":["15171:b5725905ie","15171:fe14bf80ie","15171:f6d3cfafie"]}},{"StringPrefix":{"bk_bcs_app.namespace._bk_iam_path_":["/project,b37778ec757544868a01e1f01f07037f/cluster,BCS-K8S-15171/"]}}]}}]}}`,
},
{
jsonExpr: `{"OR":{"content":[{"StringEquals":{"bk_job.cron.id":["1001800","1001801","1001802","1001803","1001804","1001805","1001806","1001807","1001808","1001809","1001810","1001811","1001812","1001813","1001814","1001815","1001816","1001817","1001818","1001819","1001820","1001821","1001822","1001823","1001824","1001825","1001826","1001827","1001828","1001829","1001830","1001831","1001832","1001833","1001834","1001835","1001836","1001837","1001838","1001839","1001840","1001841","1001842","1001847","1001848","1001855","1001856","1001858","1001868","1001869","1001871","1001872","1001873","1001876","1001877","1001878","1001879","1001880","1001881","1001882","1001883","1001884","1001885","1001886","1001887","1001888","1001889","1001890","1001891","1001892","1001893","1001894","1001895","1001896","1001897","1001898","1001899","1001900","1001901","1001902","1001903","1001904","1001905","1001906","1001907","1001908","1001909","1001910","1001911","1001912","1001913","1001914","1001915","1001916","1001917","1001918","1001919","1001920","1001921","1001922","1001923","1001924","1001925","1001926","1001927","1001928","1001929","1001930","1001931","1001932","1001933","1001936","1001937","1001938","1001939","1001940","1001941","1001942","1001943","1001944","1001945","1001946","1001947","1001948","1001949","1001950","1001951","1001952","1001953","1001954","1001955","1001956","1001957","1001958","1001959","1001960","1001961","1001962","1001963","1001964","1001965","1001966","1001967","1001968","1001969","1001970","1001971","1001972","1001973","1001974","1001975","1001976","1001977","1001978","1001979","1001980","1001981","1001983","1001984","1001986","1001987","1001988","1001989","1001990","1001991","1001992","1001993","1001994","1001995","1001996","1001997","1001998","1001999","1002006","1002007","1002008","1002009","1002010","1002011","1002015","1002016","1002017","1002018","1002019","1002020","1002022","1002023","1002024","1002025","1002026","1002027","1002028","1002029","1002030","1002031","1002032","1002033"]}},{"StringPrefix":{"bk_job.cron._bk_iam_path_":["/business_set,9991001/"]}}]}}`,
},
{
jsonExpr: `{"StringEquals":{"bk_cmdb.biz_custom_query.id":["8aa702a2-5ee5-11ec-a562-525400a7e3df","bd6e11ee-5ee6-11ec-a562-525400a7e3df","0ad936e8-5ee7-11ec-a562-525400a7e3df","d82133e5-5ee7-11ec-a562-525400a7e3df","cceed653-5eed-11ec-a562-525400a7e3df","368dc825-5eee-11ec-a562-525400a7e3df","b19e9ea5-6133-11ec-a562-525400a7e3df","ddeb5882-61fc-11ec-a562-525400a7e3df","06877010-62c6-11ec-a562-525400a7e3df","30dc4ecc-638f-11ec-a562-525400a7e3df","5c7f6b07-6458-11ec-a562-525400a7e3df","77a83df8-649c-11ec-a562-525400a7e3df","89c8d804-66b2-11ec-82f6-525400a7e3df","daff1f6f-66b3-11ec-82f6-525400a7e3df","065b8a6f-677d-11ec-82f6-525400a7e3df","f9f3447f-67a8-11ec-ad80-525400a7e3df","32c9a39d-6846-11ec-9af8-525400a7e3df","5a179e3c-690f-11ec-9af8-525400a7e3df","87a9ae66-69d8-11ec-9af8-525400a7e3df","03ecf84c-6c34-11ec-9af8-525400a7e3df","30efe7d6-6cfd-11ec-9af8-525400a7e3df","5afbdd0c-6dc6-11ec-9af8-525400a7e3df","841edefd-6e8f-11ec-9af8-525400a7e3df","aecd9dee-6f58-11ec-9af8-525400a7e3df","2de27248-71b4-11ec-9af8-525400a7e3df","58a8c2e4-727d-11ec-9af8-525400a7e3df","0bd80573-72b4-11ec-896a-525400a7e3df","86115da2-72b4-11ec-896a-525400a7e3df","88705055-72b7-11ec-896a-525400a7e3df","b6346f7f-72b7-11ec-896a-525400a7e3df","dce9929b-72b7-11ec-896a-525400a7e3df","83b27dbc-7346-11ec-896a-525400a7e3df","3f0cd20d-7389-11ec-896a-525400a7e3df","a5767973-7389-11ec-896a-525400a7e3df","ace9ee4f-740f-11ec-896a-525400a7e3df","d7576759-74d8-11ec-896a-525400a7e3df","584d94fb-7734-11ec-896a-525400a7e3df","83b94408-77fd-11ec-896a-525400a7e3df","abacbdbf-78c6-11ec-896a-525400a7e3df","1866d1ad-7a62-11ec-896a-525400a7e3df","51dd49d0-7a62-11ec-896a-525400a7e3df","e57aa13e-7a62-11ec-896a-525400a7e3df","1d549b92-7a63-11ec-896a-525400a7e3df","4d25f9a8-7a63-11ec-896a-525400a7e3df","a539f5e0-7a63-11ec-896a-525400a7e3df","763c94ff-7a64-11ec-896a-525400a7e3df","2cd49119-7a65-11ec-896a-525400a7e3df","2ed538e7-7a6d-11ec-896a-525400a7e3df","86862290-7bfd-11ec-896a-525400a7e3df","805f8dda-7cb4-11ec-896a-525400a7e3df","523b491c-7cc0-11ec-896a-525400a7e3df","8d5cd5a1-7cc0-11ec-896a-525400a7e3df","eee76596-7cc0-11ec-896a-525400a7e3df","2cbcf274-7cc9-11ec-896a-525400a7e3df","d30df9ab-7cc9-11ec-896a-525400a7e3df","a825f605-7d7d-11ec-896a-525400a7e3df","d49be3ab-7e46-11ec-896a-525400a7e3df","ff044b71-7f0f-11ec-896a-525400a7e3df","2a2e7fbf-7fd9-11ec-896a-525400a7e3df","a7572b90-8234-11ec-896a-525400a7e3df","d30a6f38-82fd-11ec-896a-525400a7e3df","fe01e096-83c6-11ec-896a-525400a7e3df","298caf10-8490-11ec-896a-525400a7e3df","50bbfffd-8559-11ec-896a-525400a7e3df","d0b1fe08-87b4-11ec-896a-525400a7e3df","fca20fcd-887d-11ec-896a-525400a7e3df","26823ed8-8947-11ec-896a-525400a7e3df","5069f02a-8a10-11ec-896a-525400a7e3df","7bfe72b6-8ad9-11ec-896a-525400a7e3df","f90da93e-8d34-11ec-896a-525400a7e3df","52d3ecd7-8e10-11ec-896a-525400a7e3df","c1160905-8e10-11ec-896a-525400a7e3df","5fec003a-8e11-11ec-896a-525400a7e3df","9076342a-8e11-11ec-896a-525400a7e3df","0b6431d3-8e12-11ec-896a-525400a7e3df","97ac234a-8e12-11ec-896a-525400a7e3df","8ce20eef-8e14-11ec-896a-525400a7e3df","a7b245df-8e26-11ec-896a-525400a7e3df","3831ea58-8e29-11ec-896a-525400a7e3df","51db9b85-8e29-11ec-896a-525400a7e3df","88fdb823-8e29-11ec-896a-525400a7e3df","11dfd741-8e2a-11ec-896a-525400a7e3df","72e2407c-8e2b-11ec-896a-525400a7e3df","d7351709-8e2c-11ec-896a-525400a7e3df","8f86f87d-8e2f-11ec-896a-525400a7e3df","b19dc306-8e30-11ec-896a-525400a7e3df","436a4498-8e31-11ec-896a-525400a7e3df","d7b0187c-8e31-11ec-896a-525400a7e3df","28f395f0-8e32-11ec-896a-525400a7e3df","4ff7ac98-8ec7-11ec-896a-525400a7e3df","730d53e7-8eda-11ec-896a-525400a7e3df","89838078-8f90-11ec-84dd-525400a7e3df","85ed0eeb-8f98-11ec-84dd-525400a7e3df","b082c357-8fc5-11ec-84dd-525400a7e3df","f2bfb97b-8fc6-11ec-84dd-525400a7e3df","e110bd7c-8fd3-11ec-84dd-525400a7e3df","e843084b-8fd4-11ec-84dd-525400a7e3df","b1fe9ce8-9059-11ec-84dd-525400a7e3df","457ce7a6-9060-11ec-84dd-525400a7e3df","31d5180d-92b5-11ec-84dd-525400a7e3df","59b00266-937e-11ec-883d-525400a7e3df","860e5623-9447-11ec-883d-525400a7e3df","1538e809-9450-11ec-945f-525400a7e3df","ef73ef30-9459-11ec-9e25-525400a7e3df","80e3207a-945a-11ec-9e25-525400a7e3df","b22d2001-9510-11ec-9e25-525400a7e3df","1b0f7b1d-953a-11ec-9595-525400a7e3df","de8323eb-95d9-11ec-9595-525400a7e3df","5d6b06ec-9835-11ec-9595-525400a7e3df","849281b1-98fe-11ec-9595-525400a7e3df","b211914e-99c7-11ec-9595-525400a7e3df","d9ba05cc-9a90-11ec-9595-525400a7e3df","01efd19e-9b5a-11ec-9595-525400a7e3df","83c60a58-9db5-11ec-9595-525400a7e3df","f0b02502-9dbc-11ec-9595-525400a7e3df","2d281daa-9dbe-11ec-9595-525400a7e3df","60b7fb04-9dbf-11ec-9595-525400a7e3df","8392dcbc-9dc0-11ec-9595-525400a7e3df","eff9b4ba-9dc0-11ec-9595-525400a7e3df","ac6638ad-9e7e-11ec-9595-525400a7e3df","d89c0ce7-9f47-11ec-9595-525400a7e3df","0525cd22-a011-11ec-9595-525400a7e3df","2b6acf18-a0da-11ec-9595-525400a7e3df","ae961815-a335-11ec-9595-525400a7e3df","d4e2442e-a3fe-11ec-9595-525400a7e3df","0a918d8a-a4c8-11ec-9595-525400a7e3df","2ca89423-a591-11ec-9595-525400a7e3df","51effd8d-a65a-11ec-9595-525400a7e3df","d763494f-a8b5-11ec-9595-525400a7e3df","ff78e69a-a97e-11ec-9595-525400a7e3df","280a0252-aa48-11ec-9595-525400a7e3df","56d87b6c-ab11-11ec-9595-525400a7e3df","8407433b-abda-11ec-9595-525400a7e3df","002fbeca-ae36-11ec-9595-525400a7e3df","2af3aedf-aeff-11ec-9595-525400a7e3df","5227417f-afc8-11ec-9595-525400a7e3df","7d6f0a57-b091-11ec-9595-525400a7e3df","a793c539-b15a-11ec-9595-525400a7e3df","27332842-b3b6-11ec-9595-525400a7e3df","51d7d4fb-b47f-11ec-9595-525400a7e3df","7ea8593a-b548-11ec-9595-525400a7e3df","a8eab5f3-b611-11ec-9595-525400a7e3df","d1a45d9b-b6da-11ec-9595-525400a7e3df","50c02c97-b936-11ec-9595-525400a7e3df","ca69119c-b96e-11ec-9595-525400a7e3df","7fcf96b4-b9ff-11ec-9595-525400a7e3df","a9a73d16-bac8-11ec-9595-525400a7e3df","d14aa84e-bb91-11ec-9595-525400a7e3df","fcaa254f-bc5a-11ec-9595-525400a7e3df","b6147708-bc5e-11ec-9595-525400a7e3df","788ce1b1-beb6-11ec-9595-525400a7e3df","a2928a9e-bf7f-11ec-9595-525400a7e3df","d1791432-bfb6-11ec-b2b0-525400a7e3df","cc94940a-c048-11ec-b2b0-525400a7e3df","f558d2c9-c111-11ec-b2b0-525400a7e3df","1ff67d7b-c1db-11ec-b2b0-525400a7e3df","a10684e1-c436-11ec-b2b0-525400a7e3df","c8e67a5c-c4ff-11ec-b2b0-525400a7e3df","f1db1027-c5c8-11ec-b2b0-525400a7e3df","224a3abb-c692-11ec-b2b0-525400a7e3df","4ca51070-c75b-11ec-b2b0-525400a7e3df","ca949f4f-c9b6-11ec-b2b0-525400a7e3df","eefb2490-ca7f-11ec-b2b0-525400a7e3df","213ca755-cb49-11ec-b2b0-525400a7e3df","4d61a555-cc12-11ec-b2b0-525400a7e3df","3ac3f76c-cc25-11ec-b2b0-525400a7e3df","75193bef-ccdb-11ec-b2b0-525400a7e3df","f061e921-cf36-11ec-b2b0-525400a7e3df","195f6232-d000-11ec-b2b0-525400a7e3df","4407c5a1-d0c9-11ec-b2b0-525400a7e3df","6e9c0ed3-d192-11ec-b2b0-525400a7e3df","98876c15-d25b-11ec-b2b0-525400a7e3df","1b7d1c5f-d4b7-11ec-b2b0-525400a7e3df","4334e3dc-d580-11ec-b2b0-525400a7e3df","6eb7cad4-d649-11ec-b2b0-525400a7e3df","996c80ef-d712-11ec-b2b0-525400a7e3df","c5591ebb-d7db-11ec-b2b0-525400a7e3df","437cde7e-da37-11ec-b2b0-525400a7e3df","6b3b9ab0-db00-11ec-9df7-525400a7e3df","9261881e-dbc9-11ec-bb28-525400a7e3df","19ee7d40-dbcc-11ec-bb28-525400a7e3df","f3a5000f-dbce-11ec-bb28-525400a7e3df","3bdf4f7a-dbcf-11ec-bb28-525400a7e3df","6a4d258d-dbd0-11ec-bb28-525400a7e3df","fd3b3a9a-dbd1-11ec-bb28-525400a7e3df","7c71be5d-dbd3-11ec-bb28-525400a7e3df","180ff695-dbd4-11ec-bb28-525400a7e3df","b80ddb98-dc92-11ec-bf35-525400a7e3df","e4a9741d-dd5b-11ec-bf35-525400a7e3df","6218c1d5-dfb7-11ec-bf35-525400a7e3df","aec960c5-dfc7-11ec-bf35-525400a7e3df","c5c9cf38-dfc8-11ec-bf35-525400a7e3df","8d2072bf-e080-11ec-bf35-525400a7e3df","b68b1020-e149-11ec-bf35-525400a7e3df","e1c986e3-e212-11ec-bf35-525400a7e3df","0c7df664-e2dc-11ec-b8c1-525400a7e3df","8b8d8c5d-e537-11ec-b8c1-525400a7e3df","b58b4d89-e600-11ec-b8c1-525400a7e3df","df6246a6-e6c9-11ec-b8c1-525400a7e3df","0a749adb-e793-11ec-b8c1-525400a7e3df","35a8fdc6-e85c-11ec-b8c1-525400a7e3df","b442b3a1-eab7-11ec-b8c1-525400a7e3df","dd861d2b-eb80-11ec-b8c1-525400a7e3df","07a3d097-ec4a-11ec-b8c1-525400a7e3df","3373e2e2-ed13-11ec-b8c1-525400a7e3df","5fe840de-eddc-11ec-bdeb-c6e85b52860d"]}}`,
},
}
func BenchmarkJsoniterUnmarshal(b *testing.B) {
for _, v := range table {
cond := pdptypes.PolicyCondition{}
expr := []byte(v.jsonExpr)
b.Run(fmt.Sprintf("expr_length_%d", len(v.jsonExpr)), func(b *testing.B) {
for i := 0; i < b.N; i++ {
jsoniter.Unmarshal(expr, &cond)
}
})
}
}
func BenchmarkGoJsonUnmarshal(b *testing.B) {
for _, v := range table {
cond := pdptypes.PolicyCondition{}
expr := []byte(v.jsonExpr)
b.Run(fmt.Sprintf("expr_length_%d", len(v.jsonExpr)), func(b *testing.B) {
for i := 0; i < b.N; i++ {
gojson.Unmarshal(expr, &cond)
}
})
}
}
func BenchmarkJsoniterMarshal(b *testing.B) {
for _, v := range table {
cond := pdptypes.PolicyCondition{}
expr := []byte(v.jsonExpr)
jsoniter.Unmarshal(expr, &cond)
b.Run(fmt.Sprintf("expr_length_%d", len(v.jsonExpr)), func(b *testing.B) {
for i := 0; i < b.N; i++ {
jsoniter.Marshal(cond)
}
})
}
}
func BenchmarkGoJsonMarshal(b *testing.B) {
for _, v := range table {
cond := pdptypes.PolicyCondition{}
expr := []byte(v.jsonExpr)
gojson.Unmarshal(expr, &cond)
b.Run(fmt.Sprintf("expr_length_%d", len(v.jsonExpr)), func(b *testing.B) {
for i := 0; i < b.N; i++ {
gojson.Marshal(cond)
}
})
}
} |
BenchmarkJsoniterUnmarshal/expr_length_46-8 854223 1246 ns/op 544 B/op 12 allocs/op
BenchmarkJsoniterUnmarshal/expr_length_212-8 302418 3832 ns/op 2272 B/op 37 allocs/op
BenchmarkJsoniterUnmarshal/expr_length_887-8 65258 17590 ns/op 9403 B/op 173 allocs/op
BenchmarkJsoniterUnmarshal/expr_length_2102-8 25634 46273 ns/op 18198 B/op 633 allocs/op
BenchmarkJsoniterUnmarshal/expr_length_8084-8 34735 33629 ns/op 21887 B/op 430 allocs/op
BenchmarkGoJsonUnmarshal/expr_length_46-8 1326015 905.7 ns/op 560 B/op 10 allocs/op
BenchmarkGoJsonUnmarshal/expr_length_212-8 349120 3537 ns/op 2418 B/op 36 allocs/op
BenchmarkGoJsonUnmarshal/expr_length_887-8 82766 14516 ns/op 9580 B/op 152 allocs/op
BenchmarkGoJsonUnmarshal/expr_length_2102-8 36567 33287 ns/op 13978 B/op 428 allocs/op
BenchmarkGoJsonUnmarshal/expr_length_8084-8 32104 38614 ns/op 18726 B/op 420 allocs/op
BenchmarkJsoniterMarshal/expr_length_46-8 1561269 737.5 ns/op 296 B/op 6 allocs/op
BenchmarkJsoniterMarshal/expr_length_212-8 493842 2485 ns/op 984 B/op 18 allocs/op
BenchmarkJsoniterMarshal/expr_length_887-8 115850 9291 ns/op 3449 B/op 60 allocs/op
BenchmarkJsoniterMarshal/expr_length_2102-8 77986 15296 ns/op 3065 B/op 18 allocs/op
BenchmarkJsoniterMarshal/expr_length_8084-8 53676 22854 ns/op 8445 B/op 6 allocs/op
BenchmarkGoJsonMarshal/expr_length_46-8 2074903 561.2 ns/op 48 B/op 1 allocs/op
BenchmarkGoJsonMarshal/expr_length_212-8 729400 1696 ns/op 224 B/op 1 allocs/op
BenchmarkGoJsonMarshal/expr_length_887-8 170414 6372 ns/op 896 B/op 1 allocs/op
BenchmarkGoJsonMarshal/expr_length_2102-8 93480 12716 ns/op 2307 B/op 1 allocs/op
BenchmarkGoJsonMarshal/expr_length_8084-8 87489 14424 ns/op 8210 B/op 1 allocs/op |
影响较大, 一不小心就是生产事故, 需要谨慎 先分类, 评估, 再确定 => 例如gin的可以先升级变异切换, 业务代码的先不动(评估影响) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
backlog
需求初始状态,等待产品进行评估
Layer: Backend
bk-iam backend project
Priority: High
Type: Enhancement
New feature or request
提取代码中的jsoniter处理逻辑, 封装
使用tag编译期决定使用哪种实现
参考实现: https://github.com/gin-gonic/gin/blob/master/internal/json/go_json.go
The text was updated successfully, but these errors were encountered: