Fix/memory leak crewbase #3569
Open
Fix/memory leak crewbase #3569
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 2 commits
September 21, 2025 03:36
- Add multiple import strategies for crew module loading - Improve project name normalization (handle hyphens and underscores) - Add fallback crew class detection when expected class name not found - Provide detailed debugging information in error messages - Fix module import path resolution issues - Add comprehensive error handling with better user feedback - Include tests for project name normalization and error scenarios Resolves issue crewAIInc#3528 - Persistent 'Crew class not found' error Addresses problems with: - Project names containing hyphens or underscores - Different project directory structures - Module import path conflicts - Insufficient error debugging information
Fixes issue crewAIInc#3450 - CrewBase objects cannot be garbage collected ## Root Cause Analysis: The memory leak was caused by multiple circular references: 1. Memoize decorator holding strong references to 'self' in cache 2. Callback wrapper functions creating circular references 3. MCP server method bound to instance preventing GC ## Solutions Implemented: ### 1. Weak Reference Memoization - Replaced strong reference caching with WeakKeyDictionary - Uses weak references for instance-based caching - Allows proper garbage collection while maintaining cache benefits - Added cache clearing functionality for debugging ### 2. Callback System Fix - Updated crew decorator to use weak references in callback wrappers - Prevents circular references between crew and CrewBase instances - Callbacks gracefully handle garbage collected instances ### 3. MCP Server Method Fix - Replaced bound method with functools.partial + weak reference - Eliminates circular reference from MCP server cleanup method - Maintains functionality while allowing garbage collection ### 4. Enhanced Cleanup Methods - Added __del__ method for automatic cleanup - Added explicit cleanup() method for user control - Proper resource management for MCP adapters and cached data ## Testing: - Comprehensive test suite covering all scenarios - Memory leak regression tests for issue crewAIInc#3450 - Tests for multiple instances and explicit cleanup - Verified garbage collection works correctly ## Performance Impact: - Maintains memoization performance benefits - No breaking changes to existing API - Memory usage now properly bounded - Eliminates memory leaks in long-running applications Resolves crewAIInc#3450
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes issue #3450 - CrewBase objects cannot be garbage collected
Root Cause Analysis:
The memory leak was caused by multiple circular references:
Solutions Implemented:
1. Weak Reference Memoization
2. Callback System Fix
3. MCP Server Method Fix
4. Enhanced Cleanup Methods
Testing:
Performance Impact: