[GSoC2024] Fix missing related images 3d dataset exports #7699

bargav25 · 2024-03-28T20:18:14Z

Issue Addressed

I resolved issue #7375, which pertained to the omission of related images from archives when exporting tasks in the Datumaro 3D format. This problem was pinpointed to the Datumaro framework's specific handling of the media parameter during the export process. When this parameter is specified, related_images were not being considered, leading to their absence in the export results and, consequently, to incomplete dataset exports.

To overcome this limitation and guarantee the inclusion of related images in the export outcomes, I have integrated the related images into the export process via the PointCloud object.

Solution Implemented

Further analysis revealed that the PointCloud constructor is specifically designed to accept an extra_images argument, aimed at encapsulating related images. However, simply passing the related images as string paths directly was inadequate because the extra_images argument requires a list of objects that possess a .path attribute—namely,Image objects. This mismatch between the expected input format and the provided string paths called for a crucial modification in the handling of related images.

To address this, I converted the related images, represented by string paths (dm_image[1]), into Image objects prior to passing them to the PointCloud constructor. This change ensures that the extra_images parameter is supplied with data in the correct format, thus enabling the successful inclusion of related images in the dataset export process.

Testing Methodology

To verify the effectiveness of this solution, I conducted a test using the test_canvas3d.zip archive found in the cvat/apps/engine/tests/assets directory of the CVAT repository. After creating a task with this archive and exporting it in the Datumaro 3D format, I confirmed that the related images, previously missing, were present in the resulting archive. This outcome affirms the resolution of the issue.

Checklist

I submit my changes into the develop branch
I have created a changelog fragment
I have updated the documentation accordingly
I have added tests to cover my changes
I have linked related issues (see GitHub docs)
I have increased versions of npm packages if it is necessary
(cvat-canvas,
cvat-core,
cvat-data and
cvat-ui)

License

I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.

zhiltsov-max · 2024-03-29T09:25:23Z

Hi, thank you for the PR! Please add tests and make sure the paths are relative to the dataset root (not absolute). You can borrow some bits from this PR.

### Motivation and context  Closes #7703 Related: #7258 Related: #7125 Related: #7699 Changes: cvat-ai/datumaro@8a14a99...82982b1 - Fixed WiderFace dataset example - Fixed export without images in Datumaro format - no empty `media` and `point_cloud` fields should be in the results ### How has this been tested?  ### Checklist  - [ ] I submit my changes into the `develop` branch - [ ] I have created a changelog fragment  - [ ] I have updated the documentation accordingly - [ ] I have added tests to cover my changes - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - [ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning)) ### License - [ ] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern.

zhiltsov-max · 2024-04-10T16:52:25Z

Hi, any updates on this PR?

bargav25 · 2024-04-10T17:08:50Z

will add tests by EOD

…#7709)

### Motivation and context  ### How has this been tested?  ### Checklist  - [ ] I submit my changes into the `develop` branch - [ ] I have created a changelog fragment  - [ ] I have updated the documentation accordingly - [ ] I have added tests to cover my changes - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - [ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning)) ### License - [ ] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. --------- Co-authored-by: Maxim Zhiltsov <[email protected]>

cvat-ai#7697)   ### Motivation and context  Allows to download annotated video frames with custom extensions. Previously, it was only available in .png format. Resolves cvat-ai#3867 ### Checklist  - [x] I submit my changes into the `develop` branch - [ ] I have created a changelog fragment  - [ ] I have updated the documentation accordingly - [ ] I have added tests to cover my changes - [x] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - [x] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning)) ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. --------- Co-authored-by: Maxim Zhiltsov <[email protected]>

bargav25 · 2024-04-10T18:39:58Z

Added in cvat/apps/engine/tests/test_rest_api_3D.py . Pls take a look

cvat/apps/dataset_manager/bindings.py

cvat/apps/engine/tests/test_rest_api_3D.py

cvat/apps/dataset_manager/bindings.py

zhiltsov-max · 2024-04-11T08:54:59Z

Please fix the failing tests (probably, some checked paths need to be updated - the same tests had been failing before the suggestions were merged)
Please add a changelog entry following the guide.

bargav25 · 2024-04-11T19:08:47Z

I can fix the Datumaro issue. However, the Sly format is not exporting related_images, which is why the error occurs, since we set related_files = True this time.

zhiltsov-max · 2024-04-12T10:04:08Z

I can fix the Datumaro issue. However, the Sly format is not exporting related_images, which is why the error occurs, since we set related_files = True this time.

What do you mean by this? I can clearly see related images in the archive, but they have slightly different paths.

changelog.d/20240412_013221_jbargav025_related_imgs_in_export.md

Co-authored-by: Maxim Zhiltsov <[email protected]>

bargav25 · 2024-04-12T15:36:35Z

Got it. Will change asap

bargav25 · 2024-04-12T18:06:13Z

Done. Pls take a look

zhiltsov-max · 2024-04-15T09:49:23Z

cvat/apps/engine/tests/test_rest_api_3D.py

@@ -761,5 +791,5 @@ def test_api_v2_export_dataset(self):
                            with open(file_name, "wb") as f:
                                f.write(content.getvalue())
                        self.assertEqual(osp.exists(file_name), edata['file_exists'])
-                        self._check_dump_content(content, task_ann_prev.data, format_name,related_files=False)
+                        self._check_dump_content(content, task_ann_prev.data, format_name, related_files=True)


Suggested change

self._check_dump_content(content, task_ann_prev.data, format_name, related_files=True)

if edata['file_exists']:

self._check_dump_content(content, task_ann_prev.data, format_name, related_files=True)

It seems there is an error in the test.

zhiltsov-max · 2024-04-15T09:53:40Z

cvat/apps/engine/tests/test_rest_api_3D.py

+                ]
+                point_cloud_files_present = any(osp.exists(f) for f in point_cloud_files)
+                if related_files and point_cloud_files_present:
+                    checking_files.extend([


This code doesn't seem to be executed

bsekachev · 2024-04-23T06:31:36Z

Hi @bargav-bunny

Do you have any plans regarding the pull request?

bsekachev · 2024-04-25T06:59:24Z

Please, let us know if you are ready to finish the pull request. I will reopen it.

corrected bindings file

6ad671f

bargav25 requested a review from zhiltsov-max as a code owner March 28, 2024 20:18

zhiltsov-max mentioned this pull request Apr 2, 2024

Update datumaro dependency #7716

Merged

7 tasks

zhiltsov-max changed the title ~~[GSoC2024] Fix Proposal for Issue #7375 : (Related images are not included when exporting 3D datasets)~~ [GSoC2024] Fix missing related images 3d dataset exports Apr 3, 2024

Merge branch 'develop' into related_imgs_in_export

3c6ba79

SpecLad and others added 6 commits April 10, 2024 14:33

Fix spurious failures in case_17_lock_hide_features.js (cvat-ai#7742)

c5525ba

[GSoC2024] feat: add shortcut support for pinning properties (cvat-ai…

c400b36

…#7709)

Fix incorrect changelog fragment (cvat-ai#7745)

b4c125d

test to check the export of Datumaro - (save_images = True)

b0ba786

bargav25 requested review from azhavoro, bsekachev, mdacoca, SpecLad, Marishka17 and nmanovic as code owners April 10, 2024 18:33

zhiltsov-max removed request for SpecLad, mdacoca, bsekachev, azhavoro and Marishka17 April 11, 2024 08:31

Merge branch 'develop' into related_imgs_in_export

ca0c932

zhiltsov-max reviewed Apr 11, 2024

View reviewed changes

cvat/apps/dataset_manager/bindings.py Outdated Show resolved Hide resolved

cvat/apps/dataset_manager/bindings.py Outdated Show resolved Hide resolved

cvat/apps/engine/tests/test_rest_api_3D.py Outdated Show resolved Hide resolved

Apply suggestions from code review

338f7e2

zhiltsov-max reviewed Apr 11, 2024

View reviewed changes

cvat/apps/dataset_manager/bindings.py Outdated Show resolved Hide resolved

zhiltsov-max reviewed Apr 11, 2024

View reviewed changes

cvat/apps/dataset_manager/bindings.py Outdated Show resolved Hide resolved

Apply suggestions from code review

b74bbc4

bargav25 added 2 commits April 12, 2024 01:26

corrected dumping function - Datumaro

4477d66

added in logs

431f443

zhiltsov-max reviewed Apr 12, 2024

View reviewed changes

changelog.d/20240412_013221_jbargav025_related_imgs_in_export.md Outdated Show resolved Hide resolved

Update changelog.d/20240412_013221_jbargav025_related_imgs_in_export.md

c2fac5a

Co-authored-by: Maxim Zhiltsov <[email protected]>

added sly dump check

6d1f762

zhiltsov-max reviewed Apr 15, 2024

View reviewed changes

bsekachev closed this Apr 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[GSoC2024] Fix missing related images 3d dataset exports #7699

[GSoC2024] Fix missing related images 3d dataset exports #7699

bargav25 commented Mar 28, 2024 •

edited

Loading

zhiltsov-max commented Mar 29, 2024

zhiltsov-max commented Apr 10, 2024

bargav25 commented Apr 10, 2024

bargav25 commented Apr 10, 2024

zhiltsov-max commented Apr 11, 2024 •

edited

Loading

bargav25 commented Apr 11, 2024

zhiltsov-max commented Apr 12, 2024 •

edited

Loading

bargav25 commented Apr 12, 2024

bargav25 commented Apr 12, 2024

zhiltsov-max Apr 15, 2024

zhiltsov-max Apr 15, 2024

bsekachev commented Apr 23, 2024

bsekachev commented Apr 25, 2024

	self._check_dump_content(content, task_ann_prev.data, format_name, related_files=True)
	if edata['file_exists']:
	self._check_dump_content(content, task_ann_prev.data, format_name, related_files=True)

[GSoC2024] Fix missing related images 3d dataset exports #7699

[GSoC2024] Fix missing related images 3d dataset exports #7699

Conversation

bargav25 commented Mar 28, 2024 • edited Loading

Issue Addressed

Solution Implemented

Testing Methodology

Checklist

License

zhiltsov-max commented Mar 29, 2024

zhiltsov-max commented Apr 10, 2024

bargav25 commented Apr 10, 2024

bargav25 commented Apr 10, 2024

zhiltsov-max commented Apr 11, 2024 • edited Loading

bargav25 commented Apr 11, 2024

zhiltsov-max commented Apr 12, 2024 • edited Loading

bargav25 commented Apr 12, 2024

bargav25 commented Apr 12, 2024

zhiltsov-max Apr 15, 2024

Choose a reason for hiding this comment

zhiltsov-max Apr 15, 2024

Choose a reason for hiding this comment

bsekachev commented Apr 23, 2024

bsekachev commented Apr 25, 2024

bargav25 commented Mar 28, 2024 •

edited

Loading

zhiltsov-max commented Apr 11, 2024 •

edited

Loading

zhiltsov-max commented Apr 12, 2024 •

edited

Loading