Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(test): add test command to e2e test the review command #24

Merged
merged 8 commits into from
Jul 24, 2023

Conversation

fabienzucchet
Copy link
Collaborator

@fabienzucchet fabienzucchet commented Jul 19, 2023

Fixes #19

Behaviour of this feature:

  • new command test running test
  • tests are defined in natural language (test case) and a snippet is generated (i.e. a code snippet corresponding to the code description). An example of the expected answer is also provided (snapshot).
  • The models runs a review of the tests code snippets, then compare the result to the snapshots
  • if the two reviews are close enough (threshold defined in constants.ts), the test PASS
  • if they are relatively close, the tests is WARN and the two reviews are displayed for manual review
  • else, the test fails and the two reviews are also displayed for debugging purpose
  • ci integration
image

@github-actions
Copy link
Contributor

github-actions bot commented Jul 19, 2023

LOGAF Level 3 - /home/runner/work/code-review-gpt/code-review-gpt/src/review/prompt/constructPrompt.ts

Consider using a more efficient algorithm for splitting files into batches. The current implementation has a time complexity of O(n^2), which could be improved to O(n) by keeping track of the current batch size.


LOGAF Level 3 - /home/runner/work/code-review-gpt/code-review-gpt/src/test/run/generateTestReport.ts

  1. The generateTestReport function has a lot of repeated code in the if-else statements. This could be simplified by creating a helper function that takes the test result and message as parameters and returns the formatted string. For example:
function formatTestReport(result: testResult, message: string) {
  switch (result) {
    case testResult.PASS:
      return chalk.green(message);
    case testResult.WARN:
      return chalk.yellow(message);
    case testResult.FAIL:
      return chalk.red(message);
  }
}
  1. The generateTestResultsSummary function could be simplified by using the reduce function to calculate the counts of each test result. For example:
const counts = Object.values(testResults).reduce((counts, result) => {
  counts[result]++;
  return counts;
}, { PASS: 0, WARN: 0, FAIL: 0 });

LOGAF Level 3 - /home/runner/work/code-review-gpt/code-review-gpt/src/test/load/loadTestCodeSnippets.ts

Consider handling errors when generating code snippets. Currently, if there is an error when generating a code snippet, the entire process will fail. It would be better to catch the error, log it, and continue with the remaining test cases.


🔄🧹🔍


Powered by Code Review GPT

@fabienzucchet fabienzucchet force-pushed the feat/19/prompt-testing-tool branch 21 times, most recently from 3640de3 to 4233618 Compare July 24, 2023 09:27
@fabienzucchet fabienzucchet marked this pull request as ready for review July 24, 2023 09:34
@fabienzucchet fabienzucchet force-pushed the feat/19/prompt-testing-tool branch 3 times, most recently from 4392d66 to ee5b71b Compare July 24, 2023 11:09
@fabienzucchet fabienzucchet force-pushed the feat/19/prompt-testing-tool branch from 17d3c3f to 6a996b3 Compare July 24, 2023 13:10
@fabienzucchet fabienzucchet force-pushed the feat/19/prompt-testing-tool branch from 5f394e3 to 7c1b09d Compare July 24, 2023 13:23
@fabienzucchet fabienzucchet force-pushed the feat/19/prompt-testing-tool branch 5 times, most recently from dd4b521 to f53b2df Compare July 24, 2023 14:12
@github-actions
Copy link
Contributor

github-actions bot commented Jul 24, 2023

Test results summary:

[PASS] - Test case: Bad variable name
[PASS] - Test case: Exposed secret
[PASS] - Test case: Too many nested loops
[PASS] - Test case: Unawaited Promise

SUMMARY: ✅ PASS: 4, ⚠️ WARN: 0, ❌ FAIL: 0


Tests Powered by Code Review GPT

@fabienzucchet fabienzucchet force-pushed the feat/19/prompt-testing-tool branch 3 times, most recently from d921ff5 to 25eba3f Compare July 24, 2023 14:25
@fabienzucchet fabienzucchet force-pushed the feat/19/prompt-testing-tool branch from 25eba3f to f4512ac Compare July 24, 2023 14:36
@fabienzucchet fabienzucchet changed the title feat(test): add test command and implement code snippet generation feat(test): add test command to unit test the review command Jul 24, 2023
@fabienzucchet fabienzucchet changed the title feat(test): add test command to unit test the review command feat(test): add test command to e2e test the review command Jul 24, 2023
@fabienzucchet fabienzucchet merged commit 8d07257 into main Jul 24, 2023
@StanGirard
Copy link

WTF ?! that is a fantastic feature ! This is a killer feature !!!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Prompt testing tool
3 participants