Skip to content

Enhance Ruby Testing Guide with setup and execution details#16805

Merged
diemol merged 5 commits into
testing_guidesfrom
aguspe-patch-1
May 10, 2026
Merged

Enhance Ruby Testing Guide with setup and execution details#16805
diemol merged 5 commits into
testing_guidesfrom
aguspe-patch-1

Conversation

@aguspe
Copy link
Copy Markdown
Contributor

@aguspe aguspe commented Dec 29, 2025

User description

Expanded the Ruby Testing Guide to include local development setup, test running methods, and environment variables.

🔗 Related Issues

💥 What does this PR do?

🔧 Implementation Notes

💡 Additional Considerations

🔄 Types of changes

  • Cleanup (formatting, renaming)
  • Bug fix (backwards compatible)
  • New feature (non-breaking change which adds functionality and tests!)
  • Breaking change (fix or feature that would cause existing functionality to change)

PR Type

Documentation


Description

  • Expanded Ruby Testing Guide with local development setup instructions

  • Added comprehensive RSpec and Rake test execution methods

  • Introduced Code Style & Linting and Documentation generation sections

  • Enhanced Guards section with conditions table and debugging tips

  • Reorganized content for better clarity and workflow guidance


Diagram Walkthrough

flowchart LR
  A["Ruby Testing Guide"] --> B["Local Development Setup"]
  A --> C["Test Execution Methods"]
  C --> C1["Bazel CI Workflow"]
  C --> C2["RSpec Local Workflow"]
  C --> C3["Rake Shortcuts"]
  A --> D["Code Style & Linting"]
  A --> E["Documentation Generation"]
  A --> F["Enhanced Guards & Debugging"]
  A --> G["Environment Variables"]
Loading

File Walkthrough

Relevant files
Documentation
TESTING.md
Comprehensive expansion of Ruby testing documentation       

rb/TESTING.md

  • Added "Local Development Setup" section with bundle install
    instructions
  • Expanded "Running Tests" section with three distinct approaches:
    Bazel, RSpec, and Rake
  • Added new "Code Style & Linting" section documenting Rubocop usage
  • Added new "Documentation" section for YARD documentation generation
  • Enhanced "Guards" section with detailed conditions table and guard
    examples
  • Added "Debugging" subsection with breakpoint instructions
  • Reorganized "Helpers" section with improved table formatting
  • Restructured "Environment Variables" section with clearer examples
  • Improved overall formatting with better markdown tables and code
    blocks
+133/-80

Expanded the Ruby Testing Guide to include local development setup, test running methods, and environment variables.
@selenium-ci selenium-ci added the C-rb Ruby Bindings label Dec 29, 2025
@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented Dec 29, 2025

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
🟢
No security concerns identified No security vulnerabilities detected by AI analysis. Human verification advised for critical code.
Ticket Compliance
🟡
🎫 #1234
🔴 Ensure that click() in Selenium 2.48 triggers JavaScript in a link's href as it did in
2.47.1.
Reproduce and resolve the regression observed on Firefox 42.0 where the test alerts in
2.47.1 but not in 2.48.x.
🟡
🎫 #5678
🔴 Investigate and fix repeated ChromeDriver instantiation failures producing "Error:
ConnectFailure (Connection refused)" on Ubuntu 16.04 with Chrome/ChromeDriver.
Ensure subsequent ChromeDriver instances do not emit connection refused errors after the
first instantiation.
Codebase Duplication Compliance
Codebase context is not defined

Follow the guide to enable codebase context checks.

Custom Compliance
🟢
Generic: Comprehensive Audit Trails

Objective: To create a detailed and reliable record of critical system actions for security analysis
and compliance.

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Meaningful Naming and Self-Documenting Code

Objective: Ensure all identifiers clearly express their purpose and intent, making code
self-documenting

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Robust Error Handling and Edge Case Management

Objective: Ensure comprehensive error handling that provides meaningful context and graceful
degradation

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Secure Error Handling

Objective: To prevent the leakage of sensitive system information through error messages while
providing sufficient detail for internal debugging.

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Secure Logging Practices

Objective: To ensure logs are useful for debugging and auditing without exposing sensitive
information like PII, PHI, or cardholder data.

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Security-First Input Validation and Data Handling

Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent
vulnerabilities

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

  • Update
Compliance status legend 🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented Dec 29, 2025

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
General
Remove markdown from code example

Remove the markdown link syntax from the reason string inside the Ruby code
block example for the exclude guard.

rb/TESTING.md [127-132]

 # Multiple conditions
 it 'something', exclude: [
   {browser: :safari},
-  {browser: :firefox, reason: '[https://github.com/SeleniumHQ/selenium/issues/123](https://github.com/SeleniumHQ/selenium/issues/123)'}
+  {browser: :firefox, reason: 'Issue: https://github.com/SeleniumHQ/selenium/issues/123'}
 ] do
 end
  • Apply / Chat
Suggestion importance[1-10]: 5

__

Why: The suggestion correctly identifies that markdown syntax within a Ruby code block will not render, and removing it improves the clarity of the example code's output.

Low
  • Update

Copy link
Copy Markdown
Member

@titusfortner titusfortner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great. A few comments and requests for clarification. Thanks!

Comment thread rb/TESTING.md Outdated
Comment thread rb/TESTING.md Outdated
Comment thread rb/TESTING.md
Comment thread rb/TESTING.md
@titusfortner titusfortner self-assigned this Dec 29, 2025
aguspe added 3 commits January 5, 2026 02:40
- Added comprehensive Type Signatures with Steep section
- Includes instructions for adding RBS files for new classes
- Added guide for updating existing type signatures
- Documented Steep commands for type checking
- Added best practices and common RBS types reference table
- Added RubyMine IDE setup instructions
- Enhanced debugging section with Bazel and RSpec options
- Added interactive REPL usage with bazel run //rb:console
Copy link
Copy Markdown
Contributor

@luke-hill luke-hill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really good - lots of this is well documented and much easier to comprehend

Comment thread rb/TESTING.md
Comment thread rb/TESTING.md
- Add Bundler install command before `bundle install` (luke-hill)
- Add Debugging section with `debug` gem and `rdbg` instructions,
  moved from trunk README#ruby (titusfortner)
@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 10, 2026

Code Review by Qodo

🐞 Bugs (4) 📘 Rule violations (0) 📎 Requirement gaps (0)

Context used

Grey Divider


Action required

1. Wrong BiDi env var 🐞 Bug ≡ Correctness
Description
rb/TESTING.md instructs enabling BiDi with BIDI=true, but the Ruby spec harness actually checks
ENV['WEBDRIVER_BIDI']; following the guide will not enable BiDi guards/websocket behavior.
Code

rb/TESTING.md[R302-313]

# Run BiDi-specific tests
bazel test //rb/spec/integration/selenium/webdriver/bidi/...
+

-### Available Variables
+### Common Variables

| Variable | Purpose | Values | Example |
-|----------|---------|--------|---------|
+| --- | --- | --- | --- |
| BIDI | Enable BiDi protocol | true, false | BIDI=true |
| WD_REMOTE_BROWSER | Specify browser for remote tests | chrome, firefox, edge, safari | WD_REMOTE_BROWSER=firefox |

</details>
Evidence
The documentation uses BIDI, but the test harness wires BiDi behavior off WEBDRIVER_BIDI in both
guard conditions and driver option wiring.

rb/TESTING.md[299-315]
rb/spec/integration/selenium/webdriver/spec_helper.rb[66-75]
rb/spec/integration/selenium/webdriver/spec_support/test_environment.rb[300-327]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`rb/TESTING.md` tells contributors to enable BiDi with `BIDI=true`, but the Ruby test harness keys off `WEBDRIVER_BIDI`.

### Issue Context
BiDi guard conditions and driver option wiring both check `ENV['WEBDRIVER_BIDI']`.

### Fix Focus Areas
- rb/TESTING.md[295-316]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Guard values don't match 🐞 Bug ≡ Correctness
Description
The Guard Conditions table documents platform: :macos and ci: true/false, but the implementation
uses Platform.os values like :macosx and Platform.ci values like
:github/:jenkins/:appveyor (or nil); documented values will not match and guards will behave
incorrectly.
Code

rb/TESTING.md[R104-112]

+| Condition | Values |
+| --- | --- |
+| `browser` | `:chrome`, `:firefox`, `:edge`, `:safari`, `:safari_preview`, `:ie` |
+| `driver` | `:remote` |
+| `platform` | `:linux`, `:macos`, `:windows` |
+| `headless` | `true`, `false` |
+| `bidi` | `true`, `false` |
+| `ci` | `true`, `false` |
+
Evidence
Guards are fed directly from Platform.os and Platform.ci, whose return values differ from what the
doc table claims, so guard metadata copied from the guide won’t match the runtime conditions.

rb/TESTING.md[104-112]
rb/spec/integration/selenium/webdriver/spec_helper.rb[66-75]
rb/lib/selenium/webdriver/common/platform.rb[37-60]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The documented guard condition values for `platform` and `ci` don’t match the actual values provided to the guards.

### Issue Context
`platform` uses `Selenium::WebDriver::Platform.os` (e.g., `:macosx`), and `ci` uses `Selenium::WebDriver::Platform.ci` (e.g., `:github`).

### Fix Focus Areas
- rb/TESTING.md[100-112]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

3. Misleading boolean env values 🐞 Bug ≡ Correctness
Description
rb/TESTING.md presents HEADLESS and DEBUG as true/false, but the code treats them as enabled
whenever the environment variable key exists, so HEADLESS=false/DEBUG=false still enables
headless mode/debug logging.
Code

rb/TESTING.md[R314-315]

| `HEADLESS` | Run tests in headless mode | `true`, `false` | `HEADLESS=true` |
| `DEBUG` | Enable debug logging | `true`, `false` | `DEBUG=true` |
Evidence
The harness sets the :headless guard based on presence and adds --headless if the env var is
set, and the logger’s default level becomes debug if ENV.key?('DEBUG') is true—none of these paths
interpret string values like "false" as disabling.

rb/TESTING.md[310-315]
rb/spec/integration/selenium/webdriver/spec_helper.rb[70-74]
rb/spec/integration/selenium/webdriver/spec_support/test_environment.rb[300-327]
rb/lib/selenium/webdriver/common/logger.rb[51-55]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The guide implies boolean env vars support `true/false`, but the implementation enables features when the env var exists (regardless of value).

### Issue Context
This affects `HEADLESS` and `DEBUG` behavior in practice (e.g., `HEADLESS=false` still enables headless mode).

### Fix Focus Areas
- rb/TESTING.md[308-316]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Advisory comments

4. Tree path has typo 🐞 Bug ⚙ Maintainability
Description
The Test Organization tree contains a stray backtick in the bidi directory name (bidi`), which
renders incorrectly and suggests a non-existent path when copied.
Code

rb/TESTING.md[R328-329]

+        ├── bidi`
+        └── spec_support/      # Test helpers
Evidence
The repository contains the rb/spec/integration/selenium/webdriver/bidi/ directory (with a BUILD
file), so the backtick is a documentation typo.

rb/TESTING.md[319-331]
rb/spec/integration/selenium/webdriver/bidi/BUILD.bazel[1-16]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
A stray backtick in the directory tree makes the `bidi` path incorrect.

### Issue Context
This is in the Test Organization section’s text block.

### Fix Focus Areas
- rb/TESTING.md[319-331]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

Comment thread rb/TESTING.md
Comment on lines 302 to 313

# Run BiDi-specific tests
bazel test //rb/spec/integration/selenium/webdriver/bidi/...

```

### Available Variables
### Common Variables

| Variable | Purpose | Values | Example |
|----------|---------|--------|---------|
| --- | --- | --- | --- |
| `BIDI` | Enable BiDi protocol | `true`, `false` | `BIDI=true` |
| `WD_REMOTE_BROWSER` | Specify browser for remote tests | `chrome`, `firefox`, `edge`, `safari` | `WD_REMOTE_BROWSER=firefox` |
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Wrong bidi env var 🐞 Bug ≡ Correctness

rb/TESTING.md instructs enabling BiDi with BIDI=true, but the Ruby spec harness actually checks
ENV['WEBDRIVER_BIDI']; following the guide will not enable BiDi guards/websocket behavior.
Agent Prompt
### Issue description
`rb/TESTING.md` tells contributors to enable BiDi with `BIDI=true`, but the Ruby test harness keys off `WEBDRIVER_BIDI`.

### Issue Context
BiDi guard conditions and driver option wiring both check `ENV['WEBDRIVER_BIDI']`.

### Fix Focus Areas
- rb/TESTING.md[295-316]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment thread rb/TESTING.md
Comment on lines +104 to +112
| Condition | Values |
| --- | --- |
| `browser` | `:chrome`, `:firefox`, `:edge`, `:safari`, `:safari_preview`, `:ie` |
| `driver` | `:remote` |
| `platform` | `:linux`, `:macos`, `:windows` |
| `headless` | `true`, `false` |
| `bidi` | `true`, `false` |
| `ci` | `true`, `false` |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

2. Guard values don't match 🐞 Bug ≡ Correctness

The Guard Conditions table documents platform: :macos and ci: true/false, but the implementation
uses Platform.os values like :macosx and Platform.ci values like
:github/:jenkins/:appveyor (or nil); documented values will not match and guards will behave
incorrectly.
Agent Prompt
### Issue description
The documented guard condition values for `platform` and `ci` don’t match the actual values provided to the guards.

### Issue Context
`platform` uses `Selenium::WebDriver::Platform.os` (e.g., `:macosx`), and `ci` uses `Selenium::WebDriver::Platform.ci` (e.g., `:github`).

### Fix Focus Areas
- rb/TESTING.md[100-112]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

@diemol diemol merged commit 1a02aa1 into testing_guides May 10, 2026
16 of 17 checks passed
@diemol diemol deleted the aguspe-patch-1 branch May 10, 2026 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants