Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(cmd/gf): refactor command gf gen service with AST #3488

Merged
merged 39 commits into from
May 27, 2024

Conversation

oldme-git
Copy link
Member

@oldme-git oldme-git commented Apr 14, 2024

Review Dear:
本次 PR 对 gf gen service 进行了部分重构,即使用 AST 替换 Regex 来解析 logic 文件。因为gf gen是一个比较重要功能,反馈的问题也比较多,所以此次提交比较重要

性能方面

相较于多次正则不同,AST一次循环解析出所有需要的语法,例如 import, func, param, comment,所以性能得到了略微的提升。
以下是对1000个文件,若干方法的测试结果,单位秒,测试环境:go1.22.1 windows11/AMD Ryzen 5500U/16G memory
AST 三次:

4.0956484
3.862473
3.8385975

Regex 三次:

4.1110877
4.0835742
4.2559844

测试文件和方法click me

改动简述

fixed #3067
fixed #3492
fixed #2310
fixed #3104
fixed #3434
fixed #3474

@oldme-git oldme-git added the wip label Apr 14, 2024
@oldme-git oldme-git changed the title enhance: improve gf gen ctrl using AST enhance: improve gf gen service using AST Apr 15, 2024
@gqcn
Copy link
Member

gqcn commented Apr 17, 2024

这块逻辑比较复杂,需要花点时间review

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


This logic is relatively complex and will take some time to review.

@gqcn
Copy link
Member

gqcn commented Apr 29, 2024

@oldme-git 我在项目中执行了一下,遇到生成方法重复的问题。我把文件作为附件上传,你可以尝试一下,建议加上一个类似的单例在提交代码中。
image

打包的logic最小文件:logic.zip

@oldme-git
Copy link
Member Author

oldme-git commented May 2, 2024

@oldme-git 我在项目中执行了一下,遇到生成方法重复的问题。我把文件作为附件上传,你可以尝试一下,建议加上一个类似的单例在提交代码中。 image

打包的logic最小文件:logic.zip

@gqcn

单测已经加上了。生成文件的那一块也整理了一下,去掉了因为正则不准确而又多添加的矫正代码。另外,在生成文件的地方进行了两个优化:

  • 使用协程
  • 使用 bytes.Buffer 替代字符串拼接,以提高性能。

最终性能得到了约 30% 的提升,以下是对 3000 个文件,若干方法测试的结果,单位秒,测试环境:go1.22.1 windows11/i7-13700k/64G memory

正则
38.7244941
33.9636121
45.8428419

AST
26.774159
21.3870076
31.7287746

在测试中,大部分的时间都用来 gofmt 生成好的文件。如果想获得更好的性能,可以对 gofmt 这一步进行优化,不过当前,也足够使用了

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


@oldme-git I executed it in the project and encountered the problem of duplicate generation methods. I uploaded the file as an attachment. You can try it. It is recommended to add a similar singleton in the submission code. image

Packaged logic minimum file: logic.zip

Single test has been added. The generated file has also been tidied up, and the correction code added due to inaccurate regular expressions has been removed. In addition, two optimizations have been made where the file is generated:

  • Use coroutines
  • Use bytes.Buffer instead of string concatenation to improve performance.

The final performance was improved by about 30%. The following are the results of testing 3000 files with several methods, unit seconds, test environment: go1.22.1 windows11/i7-13700k/64G memory

regular
38.7244941
33.9636121
45.8428419

AST
26.774159
21.3870076
31.7287746

In testing, most of the time is spent generating good files with gofmt. If you want to get better performance, you can optimize the gofmt step, but currently, it is enough.

@oldme-git oldme-git changed the title enhance: improve gf gen service using AST cmd/gf: refactor gf gen service with AST May 21, 2024
@gqcn
Copy link
Member

gqcn commented May 23, 2024

@oldme-git 有点小细节需要特殊处理一下,就是在logic中涉及到import service的调用中,需要做下替换。参考以下报错。这个是logic模块代码,通用分布式锁,不包含业务逻辑:
locker.zip

image

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


@oldme-git There are some small details that need special treatment, that is, the calls to import service in logic need to be replaced. Refer to the following error report. This is the logic module code, a universal distributed lock, which does not contain business logic:
locker.zip

image

@gqcn gqcn added the slow reviewing It might be complicated or too many changes, which needs more time reviewing. label May 23, 2024
@oldme-git
Copy link
Member Author

@oldme-git 有点小细节需要特殊处理一下,就是在logic中涉及到import service的调用中,需要做下替换。参考以下报错。这个是logic模块代码,通用分布式锁,不包含业务逻辑: locker.zip

image

可以了,强哥。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


@oldme-git There are some small details that need special treatment, that is, the calls to import service in logic need to be replaced. Refer to the following error report. This is the logic module code, a universal distributed lock, which does not contain business logic: locker.zip

image

That’s it, Brother Qiang.

@gqcn gqcn merged commit 8f4d5f7 into gogf:master May 27, 2024
23 checks passed
@oldme-git oldme-git deleted the AST-logic/logic branch May 28, 2024 07:51
@gqcn gqcn changed the title cmd/gf: refactor gf gen service with AST refactor(cmd/gf): refactor command gf gen service with AST Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
slow reviewing It might be complicated or too many changes, which needs more time reviewing.
Projects
None yet
3 participants