-
-
Notifications
You must be signed in to change notification settings - Fork 8.7k
Add guidance for testing conventions in each language #16734
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: trunk
Are you sure you want to change the base?
Changes from all commits
f54b076
844b9ce
00a62a3
3bc48df
58a693b
09277b3
392f4cd
69be16d
1a02aa1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,134 @@ | ||||||||||||||||
| # .NET Testing Guide | ||||||||||||||||
|
|
||||||||||||||||
| This guide helps contributors write tests in the Selenium .NET codebase. | ||||||||||||||||
|
|
||||||||||||||||
| ## Test Framework | ||||||||||||||||
|
|
||||||||||||||||
| * Tests use NUnit. | ||||||||||||||||
| * All tests inherit from `DriverTestFixture`. | ||||||||||||||||
| * Test HTML pages accessed via properties like `simpleTestPage`, `javascriptPage`. | ||||||||||||||||
| * `WaitFor<T>()` provides waiting with 5-second default timeout. | ||||||||||||||||
|
|
||||||||||||||||
| ```csharp | ||||||||||||||||
| [TestFixture] | ||||||||||||||||
| public class MyFeatureTest : DriverTestFixture | ||||||||||||||||
| { | ||||||||||||||||
| [Test] | ||||||||||||||||
| public void ShouldFindElement() | ||||||||||||||||
| { | ||||||||||||||||
| driver.Url = simpleTestPage; | ||||||||||||||||
| IWebElement element = driver.FindElement(By.Id("foo")); | ||||||||||||||||
| Assert.That(element.Text, Is.EqualTo("expected")); | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| [Test] | ||||||||||||||||
| [IgnoreBrowser(Browser.Safari, "Safari doesn't support this")] | ||||||||||||||||
| public void ShouldDoSomething() | ||||||||||||||||
| { | ||||||||||||||||
| // Skipped on Safari | ||||||||||||||||
| } | ||||||||||||||||
| } | ||||||||||||||||
| ``` | ||||||||||||||||
|
|
||||||||||||||||
| ## Running Tests | ||||||||||||||||
|
|
||||||||||||||||
| Bazel creates test targets for each browser. Always use `--pin_browsers`. | ||||||||||||||||
|
RenderMichael marked this conversation as resolved.
|
||||||||||||||||
|
|
||||||||||||||||
| ```shell | ||||||||||||||||
| bazel test //dotnet/test/common:AllTests --pin_browsers=true # Default browser (Firefox) | ||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Recently we removed |
||||||||||||||||
| bazel test //dotnet/test/common:AllTests-chrome --pin_browsers=true | ||||||||||||||||
| bazel test //dotnet/test/common:AllTests-firefox --pin_browsers=true | ||||||||||||||||
| bazel test //dotnet/test/common:AllTests-edge --pin_browsers=true | ||||||||||||||||
|
|
||||||||||||||||
| # Additional Arguments | ||||||||||||||||
| bazel test //dotnet/... --flaky_test_attempts=3 --pin_browsers=true | ||||||||||||||||
| bazel test //dotnet/... --test_output=all --pin_browsers=true | ||||||||||||||||
| ``` | ||||||||||||||||
|
|
||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. A few lines about running tests in the IDE.
Suggested change
|
||||||||||||||||
| ## Attributes | ||||||||||||||||
|
|
||||||||||||||||
| ### Skipping Tests | ||||||||||||||||
|
|
||||||||||||||||
| | Attribute | When to Use | | ||||||||||||||||
| |-----------|-------------| | ||||||||||||||||
| | `[IgnoreBrowser(Browser.X, "reason")]` | Skip test for specific browser | | ||||||||||||||||
| | `[IgnorePlatform("windows", "reason")]` | Skip test on specific OS | | ||||||||||||||||
| | `[IgnoreTarget("net8", "reason")]` | Skip test on specific .NET version | | ||||||||||||||||
| | `[Ignore("reason")]` | Skip test entirely (NUnit built-in) | | ||||||||||||||||
|
|
||||||||||||||||
| ```csharp | ||||||||||||||||
| [Test] | ||||||||||||||||
| [IgnoreBrowser(Browser.Safari, "Safari doesn't support multiple instances")] | ||||||||||||||||
| [IgnoreBrowser(Browser.IE, "IE is flaky")] | ||||||||||||||||
| public void TestWithMultipleDrivers() | ||||||||||||||||
| { | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| [Test] | ||||||||||||||||
| [IgnorePlatform("windows", "Thread time not supported")] | ||||||||||||||||
| public void TestLinuxOnly() | ||||||||||||||||
| { | ||||||||||||||||
| } | ||||||||||||||||
| ``` | ||||||||||||||||
|
|
||||||||||||||||
| Browser values: `Browser.Chrome`, `Browser.Firefox`, `Browser.Edge`, `Browser.Safari`, `Browser.IE`, `Browser.Remote`, `Browser.All` | ||||||||||||||||
|
|
||||||||||||||||
| ### Driver Lifecycle | ||||||||||||||||
|
|
||||||||||||||||
| | Attribute | When to Use | | ||||||||||||||||
| |-----------|-------------| | ||||||||||||||||
| | `[NeedsFreshDriver(IsCreatedBeforeTest = true)]` | Fresh driver before test | | ||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I am going to improve it in #15536 |
||||||||||||||||
| | `[NeedsFreshDriver(IsCreatedAfterTest = true)]` | Fresh driver after test | | ||||||||||||||||
|
|
||||||||||||||||
| ```csharp | ||||||||||||||||
| [Test] | ||||||||||||||||
| [NeedsFreshDriver(IsCreatedBeforeTest = true, IsCreatedAfterTest = true)] | ||||||||||||||||
| [IgnoreBrowser(Browser.Safari, "Safari doesn't support multiple instances")] | ||||||||||||||||
| public void TestRequiringFreshDriver() | ||||||||||||||||
| { | ||||||||||||||||
| IWebDriver driver2 = EnvironmentManager.Instance.CreateDriverInstance(); | ||||||||||||||||
| try | ||||||||||||||||
| { | ||||||||||||||||
| // Test with multiple drivers | ||||||||||||||||
| } | ||||||||||||||||
| finally | ||||||||||||||||
| { | ||||||||||||||||
| driver2.Quit(); | ||||||||||||||||
| } | ||||||||||||||||
| } | ||||||||||||||||
| ``` | ||||||||||||||||
|
|
||||||||||||||||
| ## Helpers | ||||||||||||||||
|
|
||||||||||||||||
| From `DriverTestFixture`: | ||||||||||||||||
|
|
||||||||||||||||
| | Member | Description | | ||||||||||||||||
| |--------|-------------| | ||||||||||||||||
| | `driver` | Current WebDriver instance | | ||||||||||||||||
| | `simpleTestPage`, `javascriptPage`, etc. | Test page URLs | | ||||||||||||||||
| | `WaitFor<T>(condition, timeout)` | Wait for condition (default 5s) | | ||||||||||||||||
| | `CreateFreshDriver()` | Create new driver instance | | ||||||||||||||||
|
|
||||||||||||||||
| From `EnvironmentManager.Instance`: | ||||||||||||||||
|
|
||||||||||||||||
| | Member | Description | | ||||||||||||||||
| |--------|-------------| | ||||||||||||||||
| | `CreateDriverInstance()` | Create additional driver | | ||||||||||||||||
| | `CreateDriverInstance(options)` | Create driver with custom options | | ||||||||||||||||
| | `Browser` | Current browser enum value | | ||||||||||||||||
|
|
||||||||||||||||
| ## Test Organization | ||||||||||||||||
|
|
||||||||||||||||
| ``` | ||||||||||||||||
| dotnet/test/ | ||||||||||||||||
| ├── common/ # Cross-browser tests | ||||||||||||||||
| │ ├── DriverTestFixture.cs # Base class | ||||||||||||||||
| │ ├── CustomTestAttributes/ # Custom NUnit attributes | ||||||||||||||||
| │ └── *Test.cs # Test files | ||||||||||||||||
| └── support/ # Support library tests | ||||||||||||||||
| ``` | ||||||||||||||||
|
|
||||||||||||||||
| ## Build Files | ||||||||||||||||
|
|
||||||||||||||||
| * Adding tests shouldn't require Bazel changes—tests are picked up via glob. | ||||||||||||||||
| * Make sure `*Test.cs` files are in a directory with `dotnet_nunit_test_suite` in BUILD.bazel. | ||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hyperlinking to each binding readme would also be beneficial.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this is a good idea. I was going to do that, but our READMEs aren't consistent right now with what they do and don't contain, and I was thinking it might make sense to update them to mostly point to our website docs, and right now the coding agents we're targeting can't access the internet.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added an issue to track this - #16740