Skip to content

Conversation

@jsibbison-square
Copy link
Contributor

The logging::tests::test_log_file_name often fail flakily. Run cargo test logging::tests::test_log_file_name to see.

I found 3 issues:

  • The combination of [tokio::test] and [test_case] seemed to be causing each test to be running twice with the same parameters increase the chances of folder/file collisions
  • Two of the test cases used the same session name
  • The test directory suffix was based on nano seconds which is less likely than random numbers to be unique

This PR changes each of those issues so now tests only run once, use unique session names and a suffix based on random numbers. I'm not sure I got all the sources of flakiness but it seems to be much more reliable.

@jsibbison-square jsibbison-square merged commit 09b06c5 into main Jun 23, 2025
7 checks passed
@jsibbison-square jsibbison-square deleted the jsibbison-20250623-flaky-log-test branch June 23, 2025 03:57
lifeizhou-ap added a commit that referenced this pull request Jun 23, 2025
* main:
  fix: Improves reliability of flaky log tests (#3029)
  Add xAI Test Coverage (#3020)
lifeizhou-ap added a commit that referenced this pull request Jun 23, 2025
* main:
  fix: Improves reliability of flaky log tests (#3029)
  Add xAI Test Coverage (#3020)
  Reorganizing tutorials (#3028)
  feat(providers): update Google Gemini models to latest available models (#2989)
  fix(docker): install protoc to fix lance-encoding build (#2995)
  fix: updated openrouter known models (#3021)
  Mnovich/temporal foreground tasks (#2895)
  add 'install in goose' asset (#3016)
  Added useDarkMode hook for detecting dark mode setting dynamically (#3019)
  docs: add sagemaker provider (#2980)
  Docs: Add Goose Recipes Cookbook Page  (#2998)
  feat (cli): add an alias --id for --name param for goose session --resume command (#3018)
  fix: disable extension  (#3000)
  Add xAI Provider Support for Grok Models (#2976)
  removing dlls from core bin (#3014)
  docs: Update Leaderboard in New Community Landing Page (#2954)
  Douwe/fix include for recipes (#2914)
jessejacksonafterpay added a commit to jessejacksonafterpay/goose that referenced this pull request Jun 24, 2025
…text-addition

* upstream/main:
  fix temporal build for windows (block#3045)
  fix cron parsing for windows (block#3044)
  feat: list Databricks-supported models and enable fuzzy search during model configuration (block#3039)
  fix: update index when tool selection strategy changes (block#2991)
  docs: Add "Add Recipe button" To Recipe Cookbook (block#3038)
  fix: issue when parsing recipe parameters (block#3031)
  fix: Improves reliability of flaky log tests (block#3029)
btdeviant pushed a commit to btdeviant/goose that referenced this pull request Jun 25, 2025
laanak08 added a commit that referenced this pull request Jun 26, 2025
# By Alice Hau (6) and others
# Via Alice Hau
* ahau/tool-strategy: (27 commits)
  fix merge
  merged
  updated computercontroller and developer descriptions
  add additional strats to test
  fmt
  update to get vector db path from env var
  feat: list Databricks-supported models and enable fuzzy search during model configuration (#3039)
  fix: update index when tool selection strategy changes (#2991)
  docs: Add "Add Recipe button" To Recipe Cookbook (#3038)
  fix: issue when parsing recipe parameters (#3031)
  fix: Improves reliability of flaky log tests (#3029)
  Add xAI Test Coverage (#3020)
  Reorganizing tutorials (#3028)
  feat(providers): update Google Gemini models to latest available models (#2989)
  fix(docker): install protoc to fix lance-encoding build (#2995)
  fix: updated openrouter known models (#3021)
  Mnovich/temporal foreground tasks (#2895)
  add 'install in goose' asset (#3016)
  Added useDarkMode hook for detecting dark mode setting dynamically (#3019)
  docs: add sagemaker provider (#2980)
  ...

# Conflicts:
#	.gitignore
#	crates/goose/src/agents/router_tools.rs
s-soroosh pushed a commit to s-soroosh/goose that referenced this pull request Jul 18, 2025
cbruyndoncx pushed a commit to cbruyndoncx/goose that referenced this pull request Jul 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants