Skip to content

[bart] minor test fixes #35965

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gante merged 1 commit into from
Jan 30, 2025
Merged

[bart] minor test fixes #35965

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion tests/models/bart/test_modeling_bart.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -885,7 +885,7 @@ def test_inference_no_head(self):
expected_shape = torch.Size((1, 11, 1024))
self.assertEqual(output.shape, expected_shape)
expected_slice = torch.tensor(
[[0.7144, 0.8143, -1.2813], [0.7144, 0.8143, -1.2813], [-0.0467, 2.5911, -2.1845]], device=torch_device
[[[0.7144, 0.8143, -1.2813], [0.7144, 0.8143, -1.2813], [-0.0467, 2.5911, -2.1845]]], device=torch_device
)
torch.testing.assert_close(output[:, :3, :3], expected_slice, rtol=1e-3, atol=1e-3)

Expand Down
2 changes: 2 additions & 0 deletions tests/test_modeling_common.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4532,6 +4532,8 @@ def test_flash_attention_2_padding_matches_padding_free_with_position_ids(self):
config.max_position_embeddings = max_new_tokens + dummy_input.shape[1] + 1

model = model_class(config)
if "position_ids" not in inspect.signature(model.forward).parameters:
self.skipTest("Model does not support position_ids")
Copy link
Collaborator

@ydshieh ydshieh Jan 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the record: self.skipTest will skip here and break the for loop for model_class in self.all_generative_model_classes.

Here is somehow fine as it's rare to have a model architecture with some model classes having position_ids but not other model class.

Copy link
Contributor Author

@gante gante Jan 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is indeed undesirable :(

Do you have a better suggestion? continue to skip the iteration on the loop also doesn't seem adequate.

Copy link
Collaborator

@ydshieh ydshieh Jan 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can have pytest-subtests installed in CI env. and using TestCase.subTest and skip inside it where necessary.

But I have installed it yet because it's will change the test summary outputs (for which some adjustments have to be done IMO).

(I also had to fix something in pytest-subtests repository)

Copy link
Collaborator

@ydshieh ydshieh Jan 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now, let's just do whatever you prefer (skip or continue), it's fine, we are aware of the situation and might take an action.

Copy link
Contributor Author

@gante gante Jan 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay :)

I'm going to merge as is then -- as you wrote, it's uncommon for a model to have subclasses with different inputs regarding the input modality


with tempfile.TemporaryDirectory() as tmpdirname:
model.save_pretrained(tmpdirname)
Expand Down