Add SuperGlue model#29886

Merged

qubvel merged 169 commits intohuggingface:mainfrom

sbucaille:add_superglue

Jan 20, 2025

Commits on Nov 17, 2024

Initial commit with template code generated by transformers-cli
sbucaille
committed
Multiple additions to SuperGlue implementation :
sbucaille
committed
Few changes for SuperGlue

steven
authored and
sbucaille
committed
Multiple changes :

steven
authored and
sbucaille
committed
Reverted unintentional change

steven
authored and
sbucaille
committed
Multiple changes :

steven
authored and
sbucaille
committed
Moved things in init files

steven
authored and
sbucaille
committed
Added docs (to be finished depending on the final implementation)

steven
authored and
sbucaille
committed
Added necessary imports and some doc

steven
authored and
sbucaille
committed
Removed unnecessary import

steven
authored and
sbucaille
committed
Fixed make fix-copies bug and ran it

steven
authored and
sbucaille
committed
Deleted SuperGlueModel

steven
authored and
sbucaille
committed
Added SuperGlueImageProcessor

steven
authored and
sbucaille
committed
Changed SuperGlue to support batching pairs of images and modified ImageMatchingOutput in consequences

steven
authored and
sbucaille
committed
Changed convert_superglue_to_hf.py script to experiment different ways of reading an image and seeing its impact on performances

steven
authored and
sbucaille
committed
Added initial tests for SuperGlueImageProcessor

steven
authored and
sbucaille
committed
Added AutoModelForImageMatching in missing places and tests

steven
authored and
sbucaille
committed
Fixed keypoint_detector_output instructions

steven
authored and
sbucaille
committed
Fix style

steven
authored and
sbucaille
committed
Adapted to latest main changes

steven
authored and
sbucaille
committed
Added integration test

steven
authored and
sbucaille
committed
Fixed bugs to pass tests

steven
authored and
sbucaille
committed
Added keypoints returned by keypoint detector in the output of SuperGlue

steven
authored and
sbucaille
committed
Added doc to SuperGlue

steven
authored and
sbucaille
committed
SuperGlue returning all attention and hidden states for a fixed number of keypoints

steven
authored and
sbucaille
committed
Make style

steven
authored and
sbucaille
committed
Changed SuperGlueImageProcessor tests

steven
authored and
sbucaille
committed
Revert "SuperGlue returning all attention and hidden states for a fixed number of keypoints"

steven
authored and
sbucaille
committed
Added back hidden_states and attentions masked outputs with tests

steven
authored and
sbucaille
committed
Renamed ImageMatching occurences into KeypointMatching

steven
authored and
sbucaille
committed
Changed SuperGlueImageProcessor to raise error when batch_size is not even

steven
authored and
sbucaille
committed
Added docs and clarity to hidden state and attention grouping function

steven
authored and
sbucaille
committed
Fixed some code and done refactoring

steven
authored and
sbucaille
committed
Fixed typo in SuperPoint output doc

steven
authored and
sbucaille
committed
Fixed some of the formatting and variable naming problems

steven
authored and
sbucaille
committed
Removed useless function call

steven
authored and
sbucaille
committed
Removed AutoModelForKeypointMatching

steven
authored and
sbucaille
committed
Fixed SuperGlueImageProcessor to only accept paris of images

steven
authored and
sbucaille
committed
Added more fixes to SuperGlueImageProcessor

steven
authored and
sbucaille
committed
Simplified the batching of attention and hidden states

steven
authored and
sbucaille
committed
Simplified stack functions

steven
authored and
sbucaille
committed
Moved attention instructions into class

steven
authored and
sbucaille
committed
Removed unused do_batch_norm argument

steven
authored and
sbucaille
committed
Moved weight initialization to the proper place

steven
authored and
sbucaille
committed
Replaced deepcopy for instantiation

steven
authored and
sbucaille
committed
Fixed small bug

steven
authored and
sbucaille
committed
Changed from stevenbucaille to magic-leap repo

steven
authored and
sbucaille
committed
Renamed London Bridge images to Tower Bridge

steven
authored and
sbucaille
committed
Fixed formatting

steven
authored and
sbucaille
committed
Renamed remaining "london" to "tower"

steven
authored and
sbucaille
committed
Apply suggestions from code review

sbucaille
and
amyeroberts
committed
Added AutoModelForKeypointMatching

steven
authored and
sbucaille
committed
Changed images used in example

steven
authored and
sbucaille
committed
Several changes to image_processing_superglue and style

steven
authored and
sbucaille
committed
Fixed resample type hint

steven
authored and
sbucaille
committed
Changed SuperGlueImageProcessor and added test case for list of 2 images

steven
authored and
sbucaille
committed
Changed list_of_tuples implementation

steven
authored and
sbucaille
committed
Fix in dummy objects

steven
authored and
sbucaille
committed
Added normalize_keypoint, log_sinkhorn_iterations and log_optimal_transport docstring

steven
authored and
sbucaille
committed
Added missing docstring

steven
authored and
sbucaille
committed
Apply suggestions from code review

sbucaille
and
amyeroberts
committed
Apply suggestions from code review

sbucaille
and
amyeroberts
committed
Moved forward block at bottom

steven
authored and
sbucaille
committed
Added docstring to forward method

steven
authored and
sbucaille
committed
Added docstring to match_image_pair method

steven
authored and
sbucaille
committed
Changed test_model_common_attributes to test_model_get_set_embeddings test method signature

steven
authored and
sbucaille
committed
Removed AutoModelForKeypointMatching

steven
authored and
sbucaille
committed
Removed image fixtures and added load_dataset

steven
authored and
sbucaille
committed
Added padding of images in SuperGlueImageProcessor

steven
authored and
sbucaille
committed
Cleaned up convert_superglue_to_hf script

steven
authored and
sbucaille
committed
Added missing docs and fixed unused argument

steven
authored and
sbucaille
committed
Fixed SuperGlueImageProcessor tests

steven
authored and
sbucaille
committed
Transposed all hidden states from SuperGlue to reflect the standard (..., seq_len, feature_dim) shape
sbucaille
committed
Added SuperGlueForKeypointMatching back to modeling_auto
sbucaille
committed
Fixed image processor padding test
sbucaille
committed
Changed SuperGlue docs
sbucaille
committed
changes:
sbucaille
committed
Changes to SuperGlueImageProcessor:
sbucaille
committed
Formatting changes
sbucaille
committed
Reverted conv1d to linear conversion because of numerical differences
sbucaille
committed
fix: changed some code to be more straightforward (e.g. filtering keypoints) and converted plot from opencv to matplotlib
sbucaille
committed
fix: removed unnecessary test
sbucaille
committed
chore: removed commented code and added back hidden states transpositions
sbucaille
committed
chore: changed from "inconsistent" to "ragged" function names as suggested

sbucaille
and
amyeroberts
committed
docs: applied suggestions

sbucaille
and
amyeroberts
committed
docs: updated to display matched output

Steven Bucaille
authored and
sbucaille
committed
chore: applied suggestion for check_image_pairs_input function

sbucaille
and
amyeroberts
committed
chore: changed check_image_pairs_input function name to validate_and_format_image_pairs and used validate_preprocess_arguments function

Steven Bucaille
authored and
sbucaille
committed
tests: simplified tests for image input format and shapes

Steven Bucaille
authored and
sbucaille
committed
feat: converted SuperGlue's use of Conv1d with kernel_size of 1 with Linear layers. Changed tests and conversion script accordingly
sbucaille
committed
feat: several changes to address comments
sbucaille
committed
refactor: changed validate_and_format_image_pairs function with clarity
sbucaille
committed
refactor: changed from one SuperGlueMLP class to a list of SuperGlueMLP class
sbucaille
committed
fix: fixed forgotten init weight change from last commit
sbucaille
committed
fix: fixed rebase mistake
sbucaille
committed
fix: removed leftover commented code
sbucaille
committed
fix: added typehint and changed some of arguments default values
sbucaille
committed
fix: fixed attribute default values for SuperGlueConfig
sbucaille
committed
feat: added SuperGlueImageProcessor post process keypoint matching method with tests
sbucaille
committed
fix: fixed SuperGlue attention and hidden state tuples aggregation
sbucaille
committed
chore: fixed mask optionality and reordered tensor reshapes to be cleaner
sbucaille
committed
chore: fixed docs and error message returned in validate_and_format_image_pairs function
sbucaille
committed
fix: fixed returned keypoints to be the ones that SuperPoint returns
sbucaille
committed
fix: fixed check on number of image sizes for post process compared to the pairs in outputs of SuperGlue
sbucaille
committed
fix: fixed check on number of image sizes for post process compared to the pairs in outputs of SuperGlue (bis)
sbucaille
committed
fix: Changed SuperGlueMultiLayerPerceptron instantiation to avoid if statement
sbucaille
committed
fix: Changed convert_superglue_to_hf script to reflect latest SuperGlue changes and got rid of nn.Modules
sbucaille
committed
WIP: implement Attention from an existing class (like BERT)
sbucaille
committed
docs: Changed docs to include more appealing matching plot
sbucaille
committed
WIP: Implement Attention
sbucaille
committed
chore: minor typehint change
sbucaille
committed
chore: changed convert superglue script by removing all classes and apply conv to linear conversion in state dict + rearrange keys to comply with changes in model's layers organisation
sbucaille
committed
Revert "Fixed typo in SuperPoint output doc"
sbucaille
committed
chore: added comments in SuperGlueImageProcessor
sbucaille
committed
chore: changed SuperGlue organization HF repo to magic-leap-community
sbucaille
committed
[run-slow] refactor: small change in layer instantiation
sbucaille
committed
[run-slow] chore: replaced remaining stevenbucaille org to magic-leap-community
sbucaille
committed
[run-slow] chore: make style
sbucaille
committed
chore: update image matching fixture dataset HF repository
sbucaille
committed
[run-slow] superglue
sbucaille
committed
tests: overwriting test_batching_equivalence
sbucaille
committed
[run-slow] superglue
sbucaille
committed

Commits on Nov 18, 2024

tests: changed test to cope with value changing depending on cuda version
Steven Bucaille
committed
[run-slow] superglue
Steven Bucaille
committed
tests: changed matching_threshold value
Steven Bucaille
committed
[run-slow] superglue
Steven Bucaille
committed

Commits on Nov 19, 2024

[run-slow] superglue
Steven Bucaille
committed

Commits on Nov 20, 2024

tests: changed tests for integration
Steven Bucaille
committed
[run-slow] superglue
Steven Bucaille
committed

Commits on Nov 25, 2024

fix: Changed tensor view and permutations to match original implementation results
Steven Bucaille
committed
fix: updated convert script and integration test to include last change in model
Steven Bucaille
committed
fix: increase tolerance for CUDA variances
Steven Bucaille
committed
Apply suggestions from code review

sbucaille
and
qubvel
authored
[run-slow] superglue
Steven Bucaille
committed
chore: removed blank whitespaces
Steven Bucaille
committed
[run-slow] superglue
Steven Bucaille
committed
Revert SuperPoint image processor accident changes
Steven Bucaille
committed
[run-slow] superglue
Steven Bucaille
committed

Commits on Nov 26, 2024

refactor: reverted copy from BERT class
Steven Bucaille
committed
tests: lower the tolerance in integration tests for SuperGlue
Steven Bucaille
committed
[run-slow] superglue
Steven Bucaille
committed
chore: set do_grayscale to False in SuperPoint and SuperGlue image processors
Steven Bucaille
committed
[run-slow] superglue
Steven Bucaille
committed

Commits on Nov 27, 2024

fix: fixed imports in SuperGlue files
Steven Bucaille
committed
chore: changed do_grayscale SuperGlueImageProcessing default value to True
Steven Bucaille
committed
docs: added typehint to post_process_keypoint_matching method in SuperGlueImageProcessor
Steven Bucaille
committed
fix: set matching_threshold default value to 0.0 instead of 0.2
Steven Bucaille
committed
feat: added matching_threshold to post_process_keypoint_matching method
Steven Bucaille
committed
docs: update superglue.md to include matching_threshold parameter
Steven Bucaille
committed
docs: updated SuperGlueConfig docstring for matching_threshold default value
Steven Bucaille
committed
refactor: removed unnecessary parameters in SuperGlueConfig
Steven Bucaille
committed
fix: changed from matching_threshold to threshold
Steven Bucaille
committed

Commits on Nov 28, 2024

fix: re-revert changes to make SuperGlue attention classes copies of BERT
Steven Bucaille
committed
[run-slow] superglue
Steven Bucaille
committed

Commits on Dec 2, 2024

Commits on Dec 3, 2024

Commits on Dec 5, 2024

tests: added checks on do_grayscale test
sbucaille
committed

Commits on Dec 6, 2024

chore: reordered and added Optional typehint to KeypointMatchingOutput
sbucaille
committed

Commits on Dec 20, 2024

Commits on Jan 7, 2025

refactor: processing images done in 1 for loop instead of 4
Steven Bucaille
committed
fix: use @ instead of torch.einsum for scores computation
Steven Bucaille
committed
style: added #fmt skip to long tensor values
Steven Bucaille
committed
refactor: rollbacked validate_and_format_image_pairs valid and invalid case to more simple ones
Steven Bucaille
committed

Commits on Jan 8, 2025

refactor: prepare_imgs
Steven Bucaille
committed
refactor: simplified `validate_and_format_image_pairs`
Steven Bucaille
committed
docs: fixed doc
Steven Bucaille
committed