Add a quality setting for avoiding using bbox in point group matching #8634
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the WalkthroughA new quality setting has been introduced that allows for comparing point groups in image space without using bounding boxes. This change involves updates across multiple files, including the addition of properties and methods in the Changes
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Outside diff range and nitpick comments (8)
changelog.d/20241101_140759_mzhiltso_compare_point_groups_in_image_space.md (1)
1-4: Enhance the changelog entry with more context.While the entry follows the correct format, it could be more informative for users by briefly explaining the purpose and impact of this feature.
Consider expanding the description like this:
### Added - A quality setting to compare point groups without using bbox + - Added a configurable quality setting to compare point groups in image space without using bounding boxes (bbox). + This improves accuracy when point groups or their bounding boxes don't accurately represent single objects. (<https://github.com/cvat-ai/cvat/pull/8634>)cvat/apps/quality_control/serializers.py (2)
102-104: LGTM: Default value configuration is appropriateThe default value of
Falsemaintains backward compatibility, and the implementation follows the established pattern.Consider condensing the nested setdefault into a single line for consistency with other similar configurations:
- extra_kwargs.setdefault("use_image_space_for_point_group_comparisons", {}).setdefault( - "default", False - ) + extra_kwargs.setdefault("use_image_space_for_point_group_comparisons", {"default": False})
126-130: Enhance help text with performance implicationsThe help text clearly explains the feature's purpose and use cases. Consider adding information about any performance implications of using image space comparisons versus bounding box comparisons.
Suggested addition to the help text:
"use_image_space_for_point_group_comparisons": """ Compare point groups in the image space, instead of using the point group bbox. Useful if point groups may not represent a single object or grouped boxes do not represent object boundaries. + Note: Image space comparisons may have different performance characteristics + compared to bbox-based comparisons. """,cvat/apps/quality_control/models.py (1)
208-209: LGTM! Consider adding field documentation.The new field is well-integrated into the model and follows the established patterns. However, it would be helpful to add a docstring to document its purpose and behavior.
Add a docstring to explain the field's purpose:
+ # Controls whether to use image space coordinates instead of bounding boxes + # when comparing point groups. When True, point coordinates are compared directly + # in the image space. When False, their bounding boxes are used for comparison. use_image_space_for_point_group_comparisons = models.BooleanField(default=False)cvat-core/src/quality-settings.ts (1)
84-90: Consider adding input validation to the setter.While the getter/setter implementation is functionally correct, consider adding validation in the setter to ensure the input value meets any required constraints.
set useImageSpaceForPointGroupComparisons(newVal: boolean) { + if (typeof newVal !== 'boolean') { + throw new Error('useImageSpaceForPointGroupComparisons must be a boolean value'); + } this.#useImageSpaceForPointGroupComparisons = newVal; }cvat-core/src/server-response-types.ts (1)
250-250: LGTM! Consider adding JSDoc comments.The new property
use_image_space_for_point_group_comparisonsis well-integrated into the interface and follows the established naming conventions. To improve maintainability, consider adding JSDoc comments to document the purpose and impact of this setting.Example documentation:
/** * When true, point group comparisons are performed in image space without using bounding boxes. * This is useful when point groups may not accurately represent a single object or when * the associated bounding box does not correctly depict the object. */ use_image_space_for_point_group_comparisons?: boolean;cvat-ui/src/components/quality-control/task-quality/quality-settings-form.tsx (1)
257-280: Consider removing unnecessary required rule from checkbox.The point comparison section is well-implemented and follows the component's established patterns. However, the
rules={[{ required: true }]}on line 272 is unnecessary for a checkbox as it always has a boolean value (true/false).<Form.Item name='useImageSpaceForPointGroupComparisons' valuePropName='checked' - rules={[{ required: true }]} >Otherwise, the implementation looks good:
- Consistent section structure with other form sections
- Proper use of Ant Design's checkbox component
- Logical placement in the form's layout
cvat/apps/quality_control/quality_reports.py (1)
1311-1316: Consider improving code organization and comments.
- The early exit comment could be more precise: "Early exit for non-overlapping bounding boxes, except for points forming an axis-aligned line"
- Remove the blank line at 1314 to maintain logical grouping of the bbox-related code.
if dm.ops.bbox_iou(a_bbox, b_bbox) <= 0: - # this early exit may not work for points forming an axis-aligned line + # Early exit for non-overlapping bounding boxes, + # except for points forming an axis-aligned line return 0 - bbox = dm.ops.mean_bbox([a_bbox, b_bbox]) scale = bbox[2] * bbox[3]
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
📒 Files selected for processing (9)
changelog.d/20241101_140759_mzhiltso_compare_point_groups_in_image_space.md(1 hunks)cvat-core/src/quality-settings.ts(4 hunks)cvat-core/src/server-response-types.ts(1 hunks)cvat-ui/src/components/quality-control/quality-control-page.tsx(1 hunks)cvat-ui/src/components/quality-control/task-quality/quality-settings-form.tsx(3 hunks)cvat/apps/quality_control/migrations/0004_qualitysettings_use_image_space_for_point_group_comparisons.py(1 hunks)cvat/apps/quality_control/models.py(1 hunks)cvat/apps/quality_control/quality_reports.py(5 hunks)cvat/apps/quality_control/serializers.py(3 hunks)
🔇 Additional comments (14)
cvat/apps/quality_control/migrations/0004_qualitysettings_use_image_space_for_point_group_comparisons.py (2)
1-18: LGTM! Migration looks well-structured and backward compatible.
The migration properly adds the new boolean field with a sensible default value that preserves existing behavior.
8-10: Verify migration dependency chain.
Let's ensure this migration is properly ordered in the dependency chain.
✅ Verification successful
Let me check the content of these migrations to verify the dependency chain properly.
Migration dependency chain is correctly ordered
The migration dependencies form a proper linear chain:
- 0001_initial depends on engine migration
- 0002_qualityreport_assignee depends on 0001_initial
- 0003_qualityreport_assignee_last_updated_and_more depends on 0002
- 0004_qualitysettings_use_image_space_for_point_group_comparisons depends on 0003
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check for any parallel migrations that might conflict
# Expected: Should only see migrations up to 0003 and this new 0004
# List all migration files in chronological order
fd --type f "^[0-9]{4}.+\.py$" "cvat/apps/quality_control/migrations" --exec ls -l {} \;
Length of output: 601
Script:
#!/bin/bash
# Check the dependencies declared in each migration file
for file in $(fd --type f "^[0-9]{4}.+\.py$" "cvat/apps/quality_control/migrations"); do
echo "=== $file ==="
rg "dependencies = " -A 2 "$file"
done
Length of output: 1988
cvat/apps/quality_control/serializers.py (2)
84-84: LGTM: Field addition is well-structured
The new field use_image_space_for_point_group_comparisons is appropriately added to the fields tuple and follows the existing naming conventions.
84-84: Verify integration with related components
The serializer changes look good, but let's verify the integration with related components.
Also applies to: 102-104
✅ Verification successful
Let me verify the implementation details of this field to ensure complete integration.
Let me check one more thing to verify the UI implementation since the previous path wasn't found.
Integration verification complete - all components properly aligned
The field is correctly integrated across all required components:
- Model: Defined as a BooleanField with default=False
- Migration: Proper migration file with correct field definition
- UI: Implemented in both quality-settings-form and quality-control-page with form validation and state management
- Serializer: Correctly included in the fields list
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check for consistent implementation across related files
# Test 1: Verify model field definition
echo "Checking model definition..."
rg -l "use_image_space_for_point_group_comparisons" "cvat/apps/quality_control/models.py"
# Test 2: Check for corresponding UI component
echo "Checking UI implementation..."
rg -l "useImageSpaceForPointGroupComparisons" "cvat-ui/src/components/quality-control/"
# Test 3: Verify migration file exists
echo "Checking migration file..."
fd -g "*use_image_space_for_point_group_comparisons*.py" "cvat/apps/quality_control/migrations/"
Length of output: 752
Script:
#!/bin/bash
# Check model field definition
echo "Model field definition:"
rg -A 2 "use_image_space_for_point_group_comparisons" "cvat/apps/quality_control/models.py"
# Check UI implementation details
echo -e "\nUI implementation:"
rg -A 2 "useImageSpaceForPointGroupComparisons" "cvat-ui/src/components/quality-control/quality-settings-form.tsx"
# Check migration content
echo -e "\nMigration details:"
cat "cvat/apps/quality_control/migrations/0004_qualitysettings_use_image_space_for_point_group_comparisons.py"
Length of output: 1188
Script:
#!/bin/bash
# Find the correct UI file location
echo "Finding UI files:"
fd "quality-settings-form" .
# Check UI implementation in found files
echo -e "\nUI implementation details:"
rg -A 2 "useImageSpaceForPointGroupComparisons" $(fd "quality-settings-form" .)
# Check quality control page implementation
echo -e "\nQuality control page implementation:"
rg -A 2 "useImageSpaceForPointGroupComparisons" "cvat-ui/src/components/quality-control/quality-control-page.tsx"
Length of output: 1204
cvat-core/src/quality-settings.ts (4)
25-25: LGTM: Property declaration follows best practices.
The new private property is well-named, properly typed, and logically placed within the class structure.
46-46: LGTM: Constructor initialization is consistent.
The property is correctly initialized from the server data, maintaining the established pattern of snake_case to camelCase conversion.
210-210: LGTM: JSON serialization maintains consistency.
The property is correctly serialized using snake_case, maintaining consistency with the server-side naming convention.
25-25: Verify integration with quality control components.
The new quality setting appears well-implemented, but let's verify its integration:
Also applies to: 46-46, 84-90, 210-210
✅ Verification successful
Let me check the actual implementation details to ensure proper integration.
Integration verified successfully - all components properly implement the new setting
The new quality setting useImageSpaceForPointGroupComparisons is properly integrated across all necessary components:
- UI components handle the setting in forms and state management
- Database migration is in place with proper default value
- Quality report implementation uses the setting for point group comparisons logic
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify the usage of the new quality setting across the codebase
# Test 1: Check for UI components using this setting
echo "Checking UI components..."
rg -l "useImageSpaceForPointGroupComparisons" "cvat-ui/src/components/quality-control"
# Test 2: Check for proper migration handling
echo "Checking migrations..."
rg -l "use_image_space_for_point_group_comparisons" "cvat/apps/quality_control/migrations"
# Test 3: Check quality report implementation
echo "Checking quality report implementation..."
rg -l "use_image_space_for_point_group_comparisons" "cvat/apps/quality_control/quality_reports.py"
Length of output: 771
Script:
#!/bin/bash
# Check the implementation details in UI components
echo "Checking UI component implementations..."
rg -A 5 "useImageSpaceForPointGroupComparisons" "cvat-ui/src/components/quality-control"
# Check migration content
echo "Checking migration implementation..."
cat "cvat/apps/quality_control/migrations/0004_qualitysettings_use_image_space_for_point_group_comparisons.py"
# Check quality report usage
echo "Checking quality report usage..."
rg -A 5 "use_image_space_for_point_group_comparisons" "cvat/apps/quality_control/quality_reports.py"
Length of output: 5442
cvat-ui/src/components/quality-control/quality-control-page.tsx (1)
228-228: LGTM! Verify form implementation.
The addition of useImageSpaceForPointGroupComparisons follows the established pattern for boolean settings. The implementation is correct and consistent with other similar settings in the quality control page.
Let's verify the form implementation to ensure the setting is properly exposed to users:
✅ Verification successful
The search results show that the form implementation is complete and consistent. The new setting is properly:
- Initialized from settings
- Has a tooltip with description
- Implemented as a checkbox form field with proper name and validation
Let me verify the type definitions to ensure full type safety:
Implementation verified and complete
The new setting useImageSpaceForPointGroupComparisons is properly implemented with:
- Type-safe boolean property in QualitySettings class
- Complete form integration with checkbox, tooltip, and validation
- Consistent getter/setter implementation
- Proper serialization for API communication
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check if the form component properly implements the new setting
# Look for the form field implementation
rg -A 5 "useImageSpaceForPointGroupComparisons" "cvat-ui/src/components/quality-control/task-quality/"
Length of output: 2349
Script:
#!/bin/bash
# Check type definitions for the new setting
rg -A 2 "useImageSpaceForPointGroupComparisons" "cvat-core/src/"
Length of output: 1477
cvat-ui/src/components/quality-control/task-quality/quality-settings-form.tsx (2)
38-38: LGTM: Initial value setup is correct.
The new setting is properly initialized from the settings prop, following the established patterns in the codebase.
93-96: LGTM: Tooltip implementation follows established patterns.
The tooltip is correctly implemented using the existing helper functions and properly sources its description from the settings object.
cvat/apps/quality_control/quality_reports.py (3)
190-191: LGTM: Well-documented parameter addition.
The new parameter is clearly named, has an appropriate default value for backward compatibility, and includes a descriptive docstring.
961-961: LGTM: Clean parameter integration in _DistanceComparator.
The new parameter is properly integrated into the constructor and initialized with a matching docstring.
Also applies to: 975-979
1306-1310: LGTM: Clear implementation of the point comparison space selection.
The logic for choosing between image space and bbox space is well-implemented and maintains backward compatibility.
…-space' into zm/compare-point-groups-in-image-space
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #8634 +/- ##
===========================================
- Coverage 74.39% 74.33% -0.07%
===========================================
Files 401 401
Lines 43421 43444 +23
Branches 3951 3951
===========================================
- Hits 32303 32292 -11
- Misses 11118 11152 +34
|
zhiltsov-max
force-pushed
the
zm/compare-point-groups-in-image-space
branch
from
4050947
to
aba7f26
Compare
zhiltsov-max
force-pushed
the
zm/compare-point-groups-in-image-space
branch
from
aba7f26
to
f943343
Compare
|
<!-- Raise an issue to propose your change (https://github.com/cvat-ai/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://docs.cvat.ai/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> Depends on #8634 Added a quality check option to consider frames matching, if both GT and job annotations have no annotations on a frame. This affects quality metrics and total counts in reports, but confusion matrices stay unchanged. This allows to use both positive and negative validation frames. ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> Unit tests ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [ ] I submit my changes into the `develop` branch - [ ] I have created a changelog fragment <!-- see top comment in CHANGELOG.md --> - [ ] I have updated the documentation accordingly - [ ] I have added tests to cover my changes - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - [ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/cvat-ai/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/cvat-ai/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/cvat-ai/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/cvat-ai/cvat/tree/develop/cvat-ui#versioning)) ### License - [ ] I submit _my code changes_ under the same [MIT License]( https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes - **New Features** - Introduced a quality setting for comparing point groups without bounding boxes. - Added an option to consider empty frames as matching in quality checks. - Enhanced quality settings form with new options for `matchEmptyFrames` and `useBboxSizeForPoints`. - **Bug Fixes** - Corrected property name from `peoject_id` to `project_id` in the API quality reports filter. - **Documentation** - Updated API schemas to include new quality settings properties. These changes improve the flexibility and accuracy of quality assessments within the application. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Motivation and context
In some cases point groups do not represent a single object or a bounding box attached to the group does not represent the object. A new quality check option is added for such cases. The new option allows to control whether to use the bbox for point group comparisons or not.
How has this been tested?
Checklist
developbranch(cvat-canvas,
cvat-core,
cvat-data and
cvat-ui)
License
Feel free to contact the maintainers if that's a concern.
Summary by CodeRabbit
Release Notes
New Features
Bug Fixes
Documentation