Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unsupported Darwin Platform Causes Test Failures on macOS #287

Open
GabLeRoux opened this issue Nov 8, 2024 · 2 comments
Open

Unsupported Darwin Platform Causes Test Failures on macOS #287

GabLeRoux opened this issue Nov 8, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@GabLeRoux
Copy link
Member

GabLeRoux commented Nov 8, 2024

Bug Description:

When committing changes on macOS (Darwin), certain tests fail due to Darwin not being recognized as a supported platform. Specifically, the ImageTag tests return errors, indicating that Darwin is unsupported, which blocks successful test execution on macOS systems.

How to Reproduce:

  1. Commit code changes on a macOS machine.
  2. Observe that the test suite runs, but fails on specific tests due to unsupported platform errors.

Expected Behavior:

Tests should either support the Darwin platform or handle unsupported platforms gracefully without causing test failures.

Failing Test Details:

FAIL  src/model/image-tag.test.ts
  ImageTag
    constructor
      ✕ can be called (12 ms)
      ✕ accepts parameters and sets the right properties (1 ms)
      ✕ accepts "2000.0.0f0" version format (1 ms)
      ✕ accepts "2011.1.11f1" version format (1 ms)
      ✕ accepts "6000.0.0f1" version format (2 ms)
      ✓ throws for incorrect versions "some version"
      ✓ throws for incorrect versions "" (1 ms)
      ✓ throws for incorrect versions 1
    toString
      ✕ returns the correct version
      ✕ returns customImage if given
      ✕ returns the specific build platform (1 ms)
      ✕ returns no specific build platform for generic targetPlatforms

Error example:

ImageTag › constructor › can be called

expect(received).not.toThrow()

Error name:    "Error"
Error message: "The Operating System of this runner, \"darwin\", is not yet supported."

Additional Details:

The tests should ideally support Darwin, or if that is not feasible, an alternative handling method should be implemented to prevent these failures on macOS systems.


Complete output of the `yarn test` command executed on macOS

yarn test
yarn run v1.22.21
warning ../../../package.json: No license field
$ jest
watchman warning:  Recrawled this watch 7 times, most recently because:
MustScanSubDirs UserDroppedTo resolve, please review the information on
https://facebook.github.io/watchman/docs/troubleshooting.html#recrawl
To clear this warning, run:
`watchman watch-del '/Users/gableroux/repos/gameci/unity-test-runner' ; watchman watch-project '/Users/gableroux/repos/gameci/unity-test-runner'`

 FAIL  src/model/image-tag.test.ts
  ImageTag
    constructor
      ✕ can be called (11 ms)
      ✕ accepts parameters and sets the right properties
      ✕ accepts "2000.0.0f0" version format (3 ms)
      ✕ accepts "2011.1.11f1" version format (1 ms)
      ✕ accepts "6000.0.0f1" version format (1 ms)
      ✓ throws for incorrect versions "some version"
      ✓ throws for incorrect versions ""
      ✓ throws for incorrect versions 1
    toString
      ✕ returns the correct version
      ✕ returns customImage if given
      ✕ returns the specific build platform
      ✕ returns no specific build platform for generic targetPlatforms

  ● ImageTag › constructor › can be called

    expect(received).not.toThrow()

    Error name:    "Error"
    Error message: "The Operating System of this runner, \"darwin\", is not yet supported."

          73 |         return Platform.types.StandaloneWindows;
          74 |       default:
        > 75 |         throw new Error(
             |               ^
          76 |           `The Operating System of this runner, "${platform}", is not yet supported.`,
          77 |         );
          78 |     }

          at Function.getImagePlatformType (src/model/image-tag.ts:75:15)
          at new ImageTag (src/model/image-tag.ts:15:33)
          at src/model/image-tag.test.ts:19:20
          at Object.<anonymous> (node_modules/expect/build/toThrowMatchers.js:83:11)
          at Object.throwingMatcher [as toThrow] (node_modules/expect/build/index.js:338:21)
          at Object.<anonymous> (src/model/image-tag.test.ts:19:68)

      17 |     it('can be called', () => {
      18 |       const { targetPlatform } = some;
    > 19 |       expect(() => new ImageTag({ platform: targetPlatform })).not.toThrow();
         |                                                                    ^
      20 |     });
      21 |
      22 |     it('accepts parameters and sets the right properties', () => {

      at Object.<anonymous> (src/model/image-tag.test.ts:19:68)

  ● ImageTag › constructor › accepts parameters and sets the right properties

    The Operating System of this runner, "darwin", is not yet supported.

      60 |         return 'windows';
      61 |       default:
    > 62 |         throw new Error(
         |               ^
      63 |           `The Operating System of this runner, "${platform}", is not yet supported.`,
      64 |         );
      65 |     }

      at Function.getImagePlatformPrefix (src/model/image-tag.ts:62:15)
      at new ImageTag (src/model/image-tag.ts:33:41)
      at Object.<anonymous> (src/model/image-tag.test.ts:23:21)

  ● ImageTag › constructor › accepts "2000.0.0f0" version format

    expect(received).not.toThrow()

    Error name:    "Error"
    Error message: "The Operating System of this runner, \"darwin\", is not yet supported."

          60 |         return 'windows';
          61 |       default:
        > 62 |         throw new Error(
             |               ^
          63 |           `The Operating System of this runner, "${platform}", is not yet supported.`,
          64 |         );
          65 |     }

          at Function.getImagePlatformPrefix (src/model/image-tag.ts:62:15)
          at new ImageTag (src/model/image-tag.ts:33:41)
          at src/model/image-tag.test.ts:35:17
          at Object.<anonymous> (node_modules/expect/build/toThrowMatchers.js:83:11)
          at Object.throwingMatcher [as toThrow] (node_modules/expect/build/index.js:338:21)
          at src/model/image-tag.test.ts:36:15

      34 |         expect(
      35 |           () => new ImageTag({ editorVersion, targetPlatform: some.targetPlatform }),
    > 36 |         ).not.toThrow();
         |               ^
      37 |       },
      38 |     );
      39 |

      at src/model/image-tag.test.ts:36:15

  ● ImageTag › constructor › accepts "2011.1.11f1" version format

    expect(received).not.toThrow()

    Error name:    "Error"
    Error message: "The Operating System of this runner, \"darwin\", is not yet supported."

          60 |         return 'windows';
          61 |       default:
        > 62 |         throw new Error(
             |               ^
          63 |           `The Operating System of this runner, "${platform}", is not yet supported.`,
          64 |         );
          65 |     }

          at Function.getImagePlatformPrefix (src/model/image-tag.ts:62:15)
          at new ImageTag (src/model/image-tag.ts:33:41)
          at src/model/image-tag.test.ts:35:17
          at Object.<anonymous> (node_modules/expect/build/toThrowMatchers.js:83:11)
          at Object.throwingMatcher [as toThrow] (node_modules/expect/build/index.js:338:21)
          at src/model/image-tag.test.ts:36:15

      34 |         expect(
      35 |           () => new ImageTag({ editorVersion, targetPlatform: some.targetPlatform }),
    > 36 |         ).not.toThrow();
         |               ^
      37 |       },
      38 |     );
      39 |

      at src/model/image-tag.test.ts:36:15

  ● ImageTag › constructor › accepts "6000.0.0f1" version format

    expect(received).not.toThrow()

    Error name:    "Error"
    Error message: "The Operating System of this runner, \"darwin\", is not yet supported."

          60 |         return 'windows';
          61 |       default:
        > 62 |         throw new Error(
             |               ^
          63 |           `The Operating System of this runner, "${platform}", is not yet supported.`,
          64 |         );
          65 |     }

          at Function.getImagePlatformPrefix (src/model/image-tag.ts:62:15)
          at new ImageTag (src/model/image-tag.ts:33:41)
          at src/model/image-tag.test.ts:35:17
          at Object.<anonymous> (node_modules/expect/build/toThrowMatchers.js:83:11)
          at Object.throwingMatcher [as toThrow] (node_modules/expect/build/index.js:338:21)
          at src/model/image-tag.test.ts:36:15

      34 |         expect(
      35 |           () => new ImageTag({ editorVersion, targetPlatform: some.targetPlatform }),
    > 36 |         ).not.toThrow();
         |               ^
      37 |       },
      38 |     );
      39 |

      at src/model/image-tag.test.ts:36:15

  ● ImageTag › toString › returns the correct version

    The Operating System of this runner, "darwin", is not yet supported.

      60 |         return 'windows';
      61 |       default:
    > 62 |         throw new Error(
         |               ^
      63 |           `The Operating System of this runner, "${platform}", is not yet supported.`,
      64 |         );
      65 |     }

      at Function.getImagePlatformPrefix (src/model/image-tag.ts:62:15)
      at new ImageTag (src/model/image-tag.ts:33:41)
      at Object.<anonymous> (src/model/image-tag.test.ts:48:21)

  ● ImageTag › toString › returns customImage if given

    The Operating System of this runner, "darwin", is not yet supported.

      60 |         return 'windows';
      61 |       default:
    > 62 |         throw new Error(
         |               ^
      63 |           `The Operating System of this runner, "${platform}", is not yet supported.`,
      64 |         );
      65 |     }

      at Function.getImagePlatformPrefix (src/model/image-tag.ts:62:15)
      at new ImageTag (src/model/image-tag.ts:33:41)
      at Object.<anonymous> (src/model/image-tag.test.ts:65:21)

  ● ImageTag › toString › returns the specific build platform

    The Operating System of this runner, "darwin", is not yet supported.

      60 |         return 'windows';
      61 |       default:
    > 62 |         throw new Error(
         |               ^
      63 |           `The Operating System of this runner, "${platform}", is not yet supported.`,
      64 |         );
      65 |     }

      at Function.getImagePlatformPrefix (src/model/image-tag.ts:62:15)
      at new ImageTag (src/model/image-tag.ts:33:41)
      at Object.<anonymous> (src/model/image-tag.test.ts:77:21)

  ● ImageTag › toString › returns no specific build platform for generic targetPlatforms

    The Operating System of this runner, "darwin", is not yet supported.

      60 |         return 'windows';
      61 |       default:
    > 62 |         throw new Error(
         |               ^
      63 |           `The Operating System of this runner, "${platform}", is not yet supported.`,
      64 |         );
      65 |     }

      at Function.getImagePlatformPrefix (src/model/image-tag.ts:62:15)
      at new ImageTag (src/model/image-tag.ts:33:41)
      at Object.<anonymous> (src/model/image-tag.test.ts:95:21)

(node:32114) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
 PASS  src/model/results-check.test.ts
  ResultsCheck
    createCheck
      ✓ throws for missing input

 PASS  src/model/index.test.ts
  Index
    ✓ exports Action (1 ms)
    ✓ exports Docker
    ✓ exports ImageTag
    ✓ exports Input
    ✓ exports Output
    ✓ exports ResultsCheck

 PASS  src/model/results-parser.test.ts
  ResultsParser
    parseResults
      ✓ throws for missing file (1 ms)
      ✓ parses editmode-results.xml
      ✓ parses playmode-results.xml
    convertResults
      ✓ converts editmode-results.xml (4 ms)
      ✓ converts playmode-results.xml (2 ms)
    convertSuite
      ✓ convert single (1 ms)
    convertTests
      ✓ convert array
      ✓ convert single
    convertTestCase
      ✓ not failed
      ✓ no stack trace (1 ms)
      ✓ no cdata in stack trace
      ✓ no annotation path
      ✓ prepare annotation without console output
      ✓ prepare annotation with console output
    findAnnotationPoint
      ✓ keep working if not matching
      ✓ simple annotation point
      ✓ first entry with non-zero line number annotation point (1 ms)
      ✓ setup annotation point (2 ms)
      ✓ Debug.LogError annotation point (1 ms)

 PASS  src/model/input.test.ts
  Input
    getFromUser
      ✓ does not throw
      ✓ returns an object (1 ms)
    isValidFolderName
      ✓ returns true for .
      ✓ returns true for ./
      ✓ returns true for folder
      ✓ returns true for trailing/
      ✓ returns true for .hidden (1 ms)
      ✓ returns true for .hidden/sub
      ✓ returns true for .hidden/trailing/
      ✓ returns true for ./.hidden-sub
      ✓ returns true for hyphen-folder
      ✓ returns true for under_score
      ✓ returns false for .. (1 ms)
      ✓ returns false for ../
    getPackageNameFromPackageJson
      ✓ throws error if package.json cannot be found at the given project path (5 ms)
      ✓ throws error if package.json contents cannot be parsed
      ✓ throws error if name field in package.json is not present (1 ms)
      ✓ throws error if name field in package.json is present but not a string
      ✓ throws error if name field in package.json is present but empty
      ✓ returns the name field in package.json if it is present as a non-empty string
    verifyTestsFolderIsPresent
      ✓ throws error if tests folder is not present (1 ms)
      ✓ does not throw if tests folder is present

 PASS  src/model/output.test.ts
  Output
    setArtifactsPath
      ✓ does not throw
    setCoveragePath
      ✓ does not throw (1 ms)

 PASS  src/model/action.test.ts
  Action
    ✓ returns the root folder of the action
    ✓ returns the action folder
    compatibility check
      ✓ throws for anything other than linux or windows (2 ms)

Trying to open ./artifacts/editmode-results.xml
Trying to open ./artifacts/playmode-results.xml
Start analyzing results: editmode-results.xml
::debug::Convert data for test Editor.EditorModeTest.FailedTest
::debug::No console output for test case: Editor.EditorModeTest.FailedTest
- /github/workspace/unity-project/Assets/Editor/EditorModeTest.cs:21 - Editor.EditorModeTest.FailedTest
::debug::Convert data for test Editor.EditorModeTest.FailedUnityTest
::debug::No console output for test case: Editor.EditorModeTest.FailedUnityTest
- /github/workspace/unity-project/Assets/Editor/EditorModeTest.cs:40 - Editor.EditorModeTest.FailedUnityTest
::debug::Skip test Editor.EditorModeTest.IgnoredTest without failure data
::debug::Skip test Editor.EditorModeTest.IgnoredUnityTest without failure data
::debug::Skip test Editor.EditorModeTest.PassedTest without failure data
::debug::Skip test Editor.EditorModeTest.PassedUnityTest without failure data
::debug::[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Start analyzing results: playmode-results.xml
::debug::Convert data for test Tests.PlayModeTest.FailedTest
::debug::No console output for test case: Tests.PlayModeTest.FailedTest
- /github/workspace/unity-project/Assets/Tests/PlayModeTest.cs:20 - Tests.PlayModeTest.FailedTest
::debug::Convert data for test Tests.PlayModeTest.FailedUnityTest
::debug::No console output for test case: Tests.PlayModeTest.FailedUnityTest
- /github/workspace/unity-project/Assets/Tests/PlayModeTest.cs:39 - Tests.PlayModeTest.FailedUnityTest
::debug::Skip test Tests.PlayModeTest.IgnoredTest without failure data
::debug::Skip test Tests.PlayModeTest.IgnoredUnityTest without failure data
::debug::Skip test Tests.PlayModeTest.PassedTest without failure data
::debug::Skip test Tests.PlayModeTest.PassedUnityTest without failure data
::debug::Convert data for test Tests.SetupFailedTest.PassedTest
::debug::No console output for test case: Tests.SetupFailedTest.PassedTest
- /github/workspace/unity-project/Assets/Tests/SetupFailedTest.cs:10 - Tests.SetupFailedTest.PassedTest
::debug::Convert data for test Tests.TearDownFailedTest.PassedTest
::debug::No console output for test case: Tests.TearDownFailedTest.PassedTest
- /github/workspace/unity-project/Assets/Tests/TearDownFailedTest.cs:10 - Tests.TearDownFailedTest.PassedTest
::debug::[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
::debug::Skip test undefined without failure data
::debug::Skip test undefined without failure data
::debug::Skip test undefined without failure data
::debug::Skip test undefined without failure data
::debug::Skip test undefined without failure data
::debug::Skip test undefined without failure data
::debug::Skip test undefined without failure data
::debug::Convert data for test undefined
::warning::No stack trace for test case: undefined
::debug::Convert data for test undefined
::warning::No cdata in stack trace for test case: undefined
::debug::Convert data for test undefined
::warning::Not able to find annotation point for failed test! Test trace: Test CDATA
::debug::Convert data for test Test Full Name
::debug::No console output for test case: Test Full Name
- /github/workspace/unity-project/Assets/Tests/SetupFailedTest.cs:10 - Test Full Name
::debug::Convert data for test Test Full Name
- /github/workspace/unity-project/Assets/Tests/SetupFailedTest.cs:10 - Test Full Name

::set-output name=artifactsPath::

::set-output name=coveragePath::

::set-output name=coveragePath::artifacts

::set-output name=coveragePath::coverage

::set-output name=coveragePath::CodeCoverage

::set-output name=coveragePath::./artifacts

::set-output name=coveragePath::./coverage

::set-output name=coveragePath::./CodeCoverage

::set-output name=coveragePath::./artifacts/coverage

::set-output name=coveragePath::./coverage/

::set-output name=coveragePath::./CodeCoverage/

::set-output name=coveragePath::./artifacts/coverage/
File ./artifacts/playmode-results.xml parsed...
Start analyzing results: playmode-results.xml
::debug::Convert data for test Tests.PlayModeTest.FailedTest
::debug::No console output for test case: Tests.PlayModeTest.FailedTest
- /github/workspace/unity-project/Assets/Tests/PlayModeTest.cs:20 - Tests.PlayModeTest.FailedTest
::debug::Convert data for test Tests.PlayModeTest.FailedUnityTest
::debug::No console output for test case: Tests.PlayModeTest.FailedUnityTest
- /github/workspace/unity-project/Assets/Tests/PlayModeTest.cs:39 - Tests.PlayModeTest.FailedUnityTest
::debug::Skip test Tests.PlayModeTest.IgnoredTest without failure data
::debug::Skip test Tests.PlayModeTest.IgnoredUnityTest without failure data
::debug::Skip test Tests.PlayModeTest.PassedTest without failure data
::debug::Skip test Tests.PlayModeTest.PassedUnityTest without failure data
::debug::Convert data for test Tests.SetupFailedTest.PassedTest
::debug::No console output for test case: Tests.SetupFailedTest.PassedTest
- /github/workspace/unity-project/Assets/Tests/SetupFailedTest.cs:10 - Tests.SetupFailedTest.PassedTest
::debug::Convert data for test Tests.TearDownFailedTest.PassedTest
::debug::No console output for test case: Tests.TearDownFailedTest.PassedTest
- /github/workspace/unity-project/Assets/Tests/TearDownFailedTest.cs:10 - Tests.TearDownFailedTest.PassedTest
::debug::[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
File ./artifacts/editmode-results.xml parsed...
Start analyzing results: editmode-results.xml
::debug::Convert data for test Editor.EditorModeTest.FailedTest
::debug::No console output for test case: Editor.EditorModeTest.FailedTest
- /github/workspace/unity-project/Assets/Editor/EditorModeTest.cs:21 - Editor.EditorModeTest.FailedTest
::debug::Convert data for test Editor.EditorModeTest.FailedUnityTest
::debug::No console output for test case: Editor.EditorModeTest.FailedUnityTest
- /github/workspace/unity-project/Assets/Editor/EditorModeTest.cs:40 - Editor.EditorModeTest.FailedUnityTest
::debug::Skip test Editor.EditorModeTest.IgnoredTest without failure data
::debug::Skip test Editor.EditorModeTest.IgnoredUnityTest without failure data
::debug::Skip test Editor.EditorModeTest.PassedTest without failure data
::debug::Skip test Editor.EditorModeTest.PassedUnityTest without failure data
::debug::[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
 PASS  src/model/unity-version-parser.test.ts
  UnityVersionParser
    parse
      ✓ throws for empty string (1 ms)
      ✓ parses from ProjectVersion.txt
      ✓ parses Unity 6000 and newer from ProjectVersion.txt
    read
      ✓ throws for invalid path
      ✓ reads from unity-project-with-correct-tests

 PASS  src/model/platform.test.ts
  Platform
    default
      ✓ does not throw
      ✓ returns a string
      ✓ returns a platform
    isWindows
      ✓ returns true for windows
      ✓ returns false for MacOS
    isAndroid
      ✓ returns true for Android (1 ms)
      ✓ returns false for Windows

Test Suites: 1 failed, 1 skipped, 8 passed, 9 of 10 total
Tests:       9 failed, 1 skipped, 68 passed, 78 total
Snapshots:   0 total
Time:        1.218 s
Ran all test suites.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.


This works fine on Windows or Linux, but we cannot commit easily on macOS at this time because husky runs the tests when we commit. Workaround is to use --no-verify parameter when we commit on this project, which is not ideal.

Example:

# Fix some code
git add .
git commit -m "Fix tests" --no-verify
# build the project by hand
yarn build
git add dist
git commit -m "yarn build" --no-verify
@GabLeRoux GabLeRoux added the bug Something isn't working label Nov 8, 2024
@GabLeRoux
Copy link
Member Author

@coderabbitai based on these observations, what do you recommend? Considering the CI runs on Linux and will catch failing tests, should we skip the tests when we detect we're on Darwin? If so, please suggest a fix.

Copy link

coderabbitai bot commented Nov 8, 2024

Tip

For best results, initiate chat on the files or code changes.

@GabLeRoux, based on your observations, I recommend adjusting the tests to ensure they pass on macOS without skipping them. Since the CI runs on Linux and will catch any platform-specific issues, making the tests platform-independent where possible will improve the developer experience on macOS.

One effective approach is to mock the process.platform value during the tests so that it returns a supported platform like 'linux'. This way, when running tests on macOS, the platform-dependent code won't throw errors due to the unsupported 'darwin' platform.

Here's how you can modify your image-tag.test.ts file:

// At the top of your image-tag.test.ts file
const originalPlatform = process.platform;

beforeAll(() => {
  Object.defineProperty(process, 'platform', {
    value: 'linux',
  });
});

afterAll(() => {
  Object.defineProperty(process, 'platform', {
    value: originalPlatform,
  });
});

This code temporarily overrides process.platform to 'linux' before the tests run and restores the original value afterward. This allows the tests to execute successfully on macOS without affecting the actual platform detection in your production code.

Alternatively, if you prefer to conditionally skip the platform-specific tests on unsupported platforms, you can adjust your test suites as follows:

const isSupportedPlatform = ['linux', 'win32'].includes(process.platform);

if (isSupportedPlatform) {
  describe('ImageTag', () => {
    // Place your tests here
  });
} else {
  describe.skip('ImageTag', () => {
    // Tests are skipped on unsupported platforms
  });
}

However, mocking process.platform is generally preferable because it ensures that your tests are run and validated consistently across different development environments.

Let me know if this solution works for you or if you need further assistance!

🔗 Analysis chain
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Find and output the contents of 'image-tag.ts' and 'platform.ts' to analyze platform detection logic
fd 'image-tag.ts' --type f --exec cat {}
fd 'platform.ts' --type f --exec cat {}

Length of output: 5643

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant