Skip to content

fix(database/gdb): performance improvement in fields grouping when in…#4440

Merged
hailaz merged 6 commits intogogf:masterfrom
wanghaolong613:fix/3906
Sep 26, 2025
Merged

fix(database/gdb): performance improvement in fields grouping when in…#4440
hailaz merged 6 commits intogogf:masterfrom
wanghaolong613:fix/3906

Conversation

@wanghaolong613
Copy link
Contributor

@wanghaolong613 wanghaolong613 commented Sep 22, 2025

fix #3906

image

…serting batch data

fix(database/gdb): performance improvement in fields grouping when inserting batch data
@hailaz hailaz requested a review from Copilot September 23, 2025 03:56
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR improves performance in the database fields grouping logic during batch insert operations by reducing redundant operations. The optimization specifically addresses issue #3906 by caching field combinations and minimizing calls to the expensive fieldsToSequence method.

Key changes:

  • Optimizes field grouping logic by pre-sorting keys and caching field combinations
  • Reduces redundant fieldsToSequence calls by grouping items with identical field sets
  • Adds benchmark test to measure batch insert performance improvements

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
database/gdb/gdb_core.go Refactors DoInsert method to cache field combinations and reduce redundant operations
contrib/drivers/mysql/mysql_z_unit_core_bench_test.go Adds benchmark test for batch insert operations to validate performance improvements

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@hailaz
Copy link
Contributor

hailaz commented Sep 25, 2025

@wanghaolonggit 做了一点点修改,复查一下

@wanghaolong613
Copy link
Contributor Author

@wanghaolonggit 做了一点点修改,复查一下

@wanghaolong613
Copy link
Contributor Author

@wanghaolonggit 做了一点点修改,复查一下

@hailaz 修改的特别好,我另外加了一个测试用例

@wanghaolong613
Copy link
Contributor Author

image 我还有个小问题,是否要保证最后执行插入的一条,是数据列表的最后一条数据。 LastInsertId() 查询到的 ID是最后一条数据的ID

@hailaz
Copy link
Contributor

hailaz commented Sep 26, 2025

image 我还有个小问题,是否要保证最后执行插入的一条,是数据列表的最后一条数据。 LastInsertId() 查询到的 ID是最后一条数据的ID

底层分组之后的map是随机顺序的,看起来目前没法保证哪个是最后一条。

@wanghaolong613
Copy link
Contributor Author

image 我还有个小问题,是否要保证最后执行插入的一条,是数据列表的最后一条数据。 LastInsertId() 查询到的 ID是最后一条数据的ID

底层分组之后的map是随机顺序的,看起来目前没法保证哪个是最后一条。

是的 确实目前保证不了

@hailaz hailaz merged commit b60b04e into gogf:master Sep 26, 2025
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

database/gdb: performance improvement in fields grouping when inserting batch data

3 participants