-
-
Notifications
You must be signed in to change notification settings - Fork 95
Simplify Test and Dependency Scheduling #2759
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…mplified scheduling This major redesign simplifies test processing by: 1. **Eager Dependency Execution**: Tests now execute their dependencies immediately when they run, rather than complex pre-execution dependency coordination. Dependencies are stored as properties within tests and executed first when needed. 2. **Task Reuse for Dependencies**: Multiple tests depending on the same test now reuse the same execution task, stored on TestContext.ExecutionTask, ensuring dependencies aren't started more than once. 3. **Proceed-on-Failure Support**: Added ProceedOnFailure property to TestDependency and updated DependsOnAttribute to preserve this flag, allowing tests to continue when dependencies fail if explicitly configured. 4. **Simplified Scheduler Logic**: Replaced complex channel-based producer-consumer system with simple queues: - SimpleTestQueues: 4 queue types for different constraint patterns - SimpleConsumerManager: Simple polling loop with natural completion detection - Removed 575 lines of complex channel coordination code, replaced with 342 lines of straightforward queue logic (40% reduction) 5. **Natural Queue Drainage**: Eliminated counting logic and external signaling. Consumers naturally exit when queues are empty and no tests are running. 6. **Removed Redundant Logic**: - Eliminated dependency graph setup and completion tracking - Removed complex routing logic - all tests route immediately - Simplified constraint handling with direct queue-based distribution The new architecture is more synchronous, less prone to race conditions, and significantly easier to understand and maintain. Tests execute dependencies eagerly, reuse execution tasks efficiently, and the scheduler focuses purely on constraint-based work distribution. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
…uler - Deleted ChannelConsumerManager and ProducerConsumerTestScheduler classes. - Simplified test scheduling by removing outdated channel-based producer-consumer logic.
This commit fixes critical issues discovered during testing:
1. **Object Disposal Double-Decrementing Bug**:
- Removed redundant `DecrementAndDisposeTrackedObjectsAsync` method
- ObjectTracker already handles disposal automatically via TestContext.Events.OnDispose
- Was causing objects to be disposed too early when shared between dependency and main tests
- Fixed "Expected fixture.IsDisposed to be equal to False but found True" error
2. **Priority and Order Scheduling Fixes**:
- Fixed priority ordering by sorting within constraint groups instead of globally
- Fixed NotInParallel Order attribute handling by sorting Order first, then Priority
- Implemented proper sorting for each constraint type:
* Parallel tests: Priority only (can run concurrently)
* Global NotInParallel: Order first, then Priority
* Keyed NotInParallel: Order first, then Priority within each key
* Parallel Groups: Preserve existing order logic
- Used simple upfront sorting approach instead of complex priority queue logic
3. **Improved Queue Management**:
- Process and enqueue each constraint group separately with proper sorting
- Maintain FIFO order within queues after proper sorting
- Cleaner separation of concerns between sorting and queueing
These fixes ensure that:
- Shared objects between dependency and main tests are properly managed
- High priority tests execute before lower priority tests
- NotInParallel tests with Order attributes execute in the correct sequence
- The simplified scheduler respects all original TUnit constraints and behaviors
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <[email protected]>
Changed parameters dictionary from Dictionary<string, string> to Dictionary<string, List<string>> to allow multiple values for the same parameter key. This resolves the "An item with the same key has already been added" error when the same parameter is passed multiple times. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
…cquisition The previous implementation had a race condition where multiple consumers could dequeue tests from the same constraint key, but only one could acquire the constraint lock, causing other tests to be lost forever. This fix implements atomic constraint acquisition by: - Using queue.Peek() to check tests without removing them - Acquiring constraints before dequeuing - Only dequeuing if constraint acquisition succeeds - Using proper locking to make the entire operation atomic Also changed AssertOrder from a test with Order=6 to an [After(Class)] hook to properly validate test execution order. This ensures NotInParallel tests execute sequentially in the correct order without any tests being lost due to race conditions. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
|
Claude finished @thomhurst's task —— View job Code Review: Simplify Test and Dependency SchedulingI've completed a comprehensive review of this pull request which replaces the complex ProducerConsumer scheduler with a simpler queue-based approach. Here's my detailed feedback: Progress:
🔍 Overall AssessmentThis is a significant architectural improvement that simplifies the test scheduling system while maintaining functionality. The refactor removes ~727 lines of complex channel-based code and replaces it with ~708 lines of cleaner queue-based logic. ✅ Positive ChangesArchitecture Simplification
Code Quality Improvements
|
Resolved the race condition where multiple consumers could dequeue tests but only one could acquire constraints, causing tests to be lost forever. Key changes: - Replaced separate dequeue + constraint acquisition with atomic operation - Added TryDequeueTestWithConstraints method that combines both operations - Fixed global NotInParallel constraint checking logic - Ensures tests are only dequeued when constraints can be successfully acquired This fixes: - OrderedTests now execute in correct sequential order (First->Fifth) - NotInParallel tests no longer run in parallel when they shouldn't - Eliminates lost tests due to failed constraint acquisition 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Added atomic check-and-set operation for ExecutionTask assignment to prevent tests from being executed multiple times when dependency resolution races with normal test execution. Key changes: - Added _executionTaskLock field to SingleTestExecutor - Used lock with null-coalescing assignment: dependency.Context.ExecutionTask ??= Task.Run(...) - Prevents duplicate test execution that was causing ObjectBag duplicate key errors - Ensures each test runs exactly once, either via normal scheduling or dependency execution This fixes the "An item with the same key has already been added" errors that occurred when Test1/Test2 were executed both as standalone tests and as dependencies of Test3. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
…tion paths Completely eliminated duplicate test execution by ensuring both regular and dependency execution use the same ExecutionTask mechanism. Key changes: - Modified ExecuteTestAsync to check for existing ExecutionTask before creating new one - Both regular and dependency execution now use the same ExecuteTestInternalAsync method - Removed Task.Run to preserve AsyncLocal flow (TestContext.Current) - Added proper test state handling for edge cases This finally resolves the "An item with the same key has already been added" errors by ensuring each test runs exactly once, regardless of whether it's executed as a standalone test or as a dependency of another test. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Fixed async method return type mismatch in GetOrCreateDependencyExecutionTaskAsync. The method now properly awaits the task outside the lock to prevent deadlocks while ensuring only one execution task is created per dependency test. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
|
Claude finished @thomhurst's task —— View job Code Review: Simplify Test and Dependency SchedulingI've completed a comprehensive review of this pull request which replaces the complex ProducerConsumer scheduler with a simpler queue-based approach. Here's my detailed feedback: Progress:
🔍 Overall AssessmentThis is a significant architectural improvement that simplifies the test scheduling system while maintaining functionality. The refactor removes ~727 lines of complex channel-based code and replaces it with ~571 lines of cleaner queue-based logic. ✅ Positive ChangesArchitecture Simplification
Code Quality Improvements
|
- Added ExecutionLock to TestContext for thread-safe coordination - Both scheduler and dependency execution paths now use same lock - Unified both paths to use single ExecuteTestInternalAsync method - Prevents Test1 from running twice (once as scheduled test, once as dependency) - Simplified logic removes duplicate execution paths and potential race conditions 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
|
Claude finished @thomhurst's task —— View job Code Review: Simplify Test and Dependency SchedulingI've completed a comprehensive review of this pull request which replaces the complex ProducerConsumer scheduler with a simpler queue-based approach. Here's my detailed feedback: Progress:
🔍 Overall AssessmentThis is a significant architectural improvement that simplifies the test scheduling system while maintaining functionality. The refactor removes complex channel-based code and replaces it with cleaner queue-based logic, making the codebase more maintainable. ✅ Positive ChangesArchitecture Simplification
Code Quality Improvements
|
Fixed a critical bug where tests skipped due to failed dependencies were having their state reset to Running, causing "Test is still running" errors. The issue was in ExecuteTestInternalAsync: 1. ExecuteDependenciesAsync would mark a test as Skipped if dependencies failed 2. But execution continued and reset the state to Running 3. This left tests in an invalid Running state that never completed Now we check if the test was skipped after executing dependencies and return early to preserve the Skipped state. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
|
Claude finished @thomhurst's task —— View job Code Review: Simplify Test and Dependency SchedulingI've completed a comprehensive review of this architectural refactor that replaces the complex ProducerConsumer scheduler with a simpler queue-based approach. Progress:
🔍 Overall AssessmentThis is a significant architectural improvement that successfully simplifies the test scheduling system while maintaining functionality. The refactor removes complex channel-based code and replaces it with cleaner queue-based logic, making the codebase more maintainable and easier to understand. ✅ Positive ChangesArchitecture Simplification
Code Quality Improvements
|
Added try-catch around execution task awaiting to ensure tests that throw exceptions during execution get proper failed state instead of being left in Running state. This helps handle edge cases where test execution fails unexpectedly. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
No description provided.