Keyframe Selection: Add support for SfMData files as inputs and outputs #1406
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit only modifies the syntax of the dataio module and does not change its content in any way. The hpp and cpp files have been clang- formatted to respect the 4-spaces indentation that can be found in other modules (in the "keyframe" module, primarily), and some functions' prototypes have been re-aligned accordingly.
The ImageFeed accepted as an input: - a text file containing a list of images' paths - input images - a directory - a JSON file (SfMData) The support for the JSON file is removed from the ImageFeed. It will later be added to a specific feeder.
The SfMDataFeed supports SfMData files, which can be .sfm, .json or .abc. The provided input file is parsed, and the views are ordered according to their frame ID.
The keyframe selection now supports an SfMData file as the input, and outputs two SfMData files: - one that contains all the selected keyframes - one that contains all the frames that were not selected as keyframes
The "selectedFrames" vector is filled with 0s and 1s depending on whether the frame with the ID corresponding to the position within the vector has been selected as a keyframe or not. It was filled with the smart method, but not with the regular one. As it will be needed later on and is not costly, it should also be filled for the regular selection method.
…file This commit adds a new method that writes two output SfMData files (one containing the keyframes, the other containing all the frames that were not selected) when the input file is an SfMData file. Once all the selected keyframes have been written on disk, the input SfMData file is loaded (outside of the feed) and its views and intrinsics are extracted to be written in their corresponding output SfMData file. No information written is these files is built from scratch. Rigs are currently not supported and therefore ignored when writing the output files. If the input is a video or a sequence of images, no SfMData file is written for now.
Prior to this commit, views were ordered based solely on their frame ID. If the inpput SfMData only contains one sequence, this works perfectly fine. However, in case of a rig or several sequences dropped one after the other, there might be some identical frame IDs which would affect the entire sorting order: the sequences would be mixed together instead of being sorted back-to-back. For example, if there are two sequences with identical frame IDs, the sorting order will be "seq1#id0, seq2#id0, seq1#id1, seq2#id1, seq1#id2, seq2#id2" instead of "seq1#id0, seq1#id1, seq1#id2, seq2#id0, seq2#id1, seq2#id2". This commit sorts views with different intrinsics' serial numbers separately based on their frame IDs, and then concatenates all the sorted views together.
…xists The database, if available, will later be used to try to determine the intrinsics of input videos or image sequences.
…umber The ordering of the views when writing the output SfMData files should be identical to the one used in the SfMDataFeed. Otherwise, the views written in the output files will not reflect the keyframes that were saved on disk.
SfMData files are written as outputs for all types of inputs, with the following specifications: - for an input SfMData file, both the keyframes and frames SfMData files will be written; rigs are not supported - for an input sequence (or several) of images, both the keyframes and frames SfMData files will be written; rigs are supported and will be written in the output SfMData files - for an input video, only the keyframes SfMData file will be written, no frames SfMData file will be output; rigs are supported and will be written in the keyframes SfMData files
Following the addition of the SfMData files as outputs in the command line, the version needs to be updated.
2 tasks
mugulmd
approved these changes
Apr 18, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR builds on top of #1343 and:
SfMDatafiles as an input;SfMDatafiles: one that contains the selected keyframes, and one that contains all the frames that failed to be selected as keyframes.SfMDatafiles can now be provided as inputs for both the regular and smart keyframe selection, and the selection methods remain oblivious to the type of input they are given. Rigs are not supported withSfMDatafiles as inputs.Once the keyframes have been written, two
SfMDatafiles are written, respectively with the keyframes' information and the rejected frames' information. Depending on the type of input, the outputSfMDatafiles are written with the following specifications:SfMDatafiles will be written, with a full rig support;SfMDatafile will be written, with a full rig support. Since the frames'SfMDatafile requires the paths of all the rejected frames to be filled and the only available path (beside the keyframes') is the input video's, it is simply ignored;SfMDatafile, both the keyframes' and frames'SfMDatafiles will be written, with no rig support.While rigs are not supported with
SfMDatafiles as inputs, it is possible to provide anSfMDatafile that contains several different image sequences. If that occurs, the views will be ordered according to their frame IDs and their intrinsics' serial number, put back-to-back (e.g. for two sequences with 3 frames each in the sameSfMDatafile, the views will be sorted as "seq1#id0, seq1#id1, seq1#id2, seq2#id0, seq2#id1, seq2#id2"), and then processed by the selection methods as a single sequence.Additionally, the dataio module has been reformatted to use 4-space indentations everywhere.
The corresponding Meshroom pull request can be found here: alicevision/Meshroom#1967
Features list
ImageFeedin the dataio module;SfMDatafiles in the dataio module;SfMDatafiles as inputs for the keyframe selection;SfMDatafiles (depending on the input's type) in addition to the selected keyframes.Implementation remarks
SfMDataFeed, dedicated toSfMDatafiles has been added to dataio'sFeedProviderto read and provide images from anySfMDatafiles. It replaces the support for .json files that was provided by theImageFeedfeed.SfMDatafiles (with a .json extension) that was available in theImageFeedfeed has been completely removed.ImageFeedstill supports all types of images as well as .txt files containing a list of paths to images.SfMDatafiles is not covered by this pull request, and should be added later in a dedicated one.