Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run before(:all) hooks for rails magic example groups #829

Closed
wants to merge 7 commits into from
Closed

Run before(:all) hooks for rails magic example groups #829

wants to merge 7 commits into from

Conversation

JonRowe
Copy link
Member

@JonRowe JonRowe commented Sep 29, 2013

We automagically configure ExampleGroup types for the various sets of Rails
helpers, however in 2.14 and master ATM these incorrectly pickup configured
helpers. E.g. a before(:all, :type => :model) will not run for a spec in
spec/models unless the type is set manually.

This is because rspec-core configures hooks based on their current metadata
and not on any metadata set later on. The alternative approach would be to
reload the global hooks from config when the metadata is changed in rspec-core
but this fixes #825 for now.

Note that manually specifying type is not affected, nor are any child groups
or examples. It is only the top level group that is affected.

Thoughts? /cc @alindeman @myronmarston

@myronmarston
Copy link
Member

Thanks for tracking this down, @JonRowe. A few thoughts:

  • I consider register_globals to be a private API to me. I would never recommend that extension gems use it. We should treat rspec-rails the same...so I'm not sure that this is the right way to fix it.
  • You mention that 2.14 has this behavior. Do you know if it's a regression in 2.14 (or some other recent version) or has this always been a problem? If it's a regression it would be good to know what PR or commit caused it (git bisect will help track that down).
  • It seems like there's a bug (or at least an inconsistency) in rspec-core: either rspec-core should support included modules mutating the metadata, and fully support that properly, or it shouldn't support that and should prevent or print warnings when included modules do that. Currently rspec-rails allows it (and I've always thought it was a bit odd...) but I don't know if anyone else does. How hard would it be to fix rspec-core so it properly supports it? One idea is to move the hooks.register_globals call out of set_it_up and into run, so that the hooks are registered at the last possible moment. If we could fix it that way, I think the changes in rspec-rails here would become unnecessary (although it's probably worth keeping the specs to prevent future regressions).

myronmarston
myronmarston reviewed Sep 30, 2013
View reviewed changes
lib/rspec/rails/example/rails_example_group.rb
def set_metadata_type(type)
metadata[:type] = type
hooks.register_globals(self, RSpec.configuration.hooks)
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this isn't used anywhere? Looks like a useful abstraction, though; if we stick with this solution for some reason it would be good to leverage this. (Although I suspect the final fix will be in rspec-core).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops yeah, I tried to DRY it up (couldn't get it to work cause of the included scoping)

@JonRowe
Copy link
Member Author

JonRowe commented Sep 30, 2013

  • I consider register_globals to be a private API to me. I would never recommend that extension gems use it. We should treat rspec-rails the same...so I'm not sure that this is the right way to fix it.

So do I, but this seemed the best way to start a discussion about it. The problem is that rspec-rails does something I wouldn't recommend already (magically mixing in metadata like this).

  • You mention that 2.14 has this behavior. Do you know if it's a regression in 2.14 (or some other recent version) or has this always been a problem? If it's a regression it would be good to know what PR or commit caused it (git bisect will help track that down).

I haven't tried, I don't remember us changing anything that would make it a regression, I will try at some point with 2.13.

  • It seems like there's a bug (or at least an inconsistency) in rspec-core: either rspec-core should support included modules mutating the metadata, and fully support that properly, or it shouldn't support that and should prevent or print warnings when included modules do that. Currently rspec-rails allows it (and I've always thought it was a bit odd...) but I don't know if anyone else does. How hard would it be to fix rspec-core so it properly supports it? One idea is to move the hooks.register_globals call out of set_it_up and into run, so that the hooks are registered at the last possible moment. If we could fix it that way, I think the changes in rspec-rails here would become unnecessary (although it's probably worth keeping the specs to prevent future regressions).

I think this is a "sort of" bug in core, this is why I initially started poking at that to try to fix this. One way to solve this would be to make the metadata to reload global hooks when it's mutated. Or move the hook setup to be lazy and only configure from the global configuration when run (which is what your suggestion is I guess)

@JonRowe JonRowe mentioned this pull request Sep 30, 2013
Closed
@JonRowe
Copy link
Member Author

JonRowe commented Oct 1, 2013

Had to update this so it fails correctly, it's awaiting rspec/rspec-core#1089 before it'll work.

@alindeman
Copy link
Contributor

Hi all--

What's the status on this PR and rspec/rspec-core#1089? Do we need it before 3.0? Can I help?

@JonRowe
Copy link
Member Author

JonRowe commented Nov 14, 2013

So I'm still in favour of rspec/rspec-core#1089 but the alternative is we work something out that doesn't involve mutating metadata for rspec-rails integration

@myronmarston myronmarston mentioned this pull request Apr 21, 2014
Merged
myronmarston added a commit that referenced this pull request Apr 21, 2014
@myronmarston
Fix spec type inferrence.
e135076 
- In #970 we added `infer_spec_type_from_file_location!` but forgot
  to remove the code in `lib/rspec/rails/configuration.rb` that
  made it always infer - so the opt-in API was there but it
  was always enabled. Here I've made it truly opt-in.
- The logic of `infer_spec_type_from_file_location!` also setup
  the helper module inclusion via explicit `:type` metadata but
  was only intended to setup the implicit mapping of spec file
  location to type.  Here I've made the module inclusion via
  `:type` always work w/o an explicit config option needed to
  enable it.
- We used to mutate the `:type` metadata on inclusion of the helper
  module, but this caused bugs such as #825. The problem is that
  rspec-core uses a raw hash for the metadata and doesn't re-apply
  filtering/inclusion logic when the metadata hash is mutated.
  Instead, we use a new explicit `define_derived_metadata` API.

Fixes #825 and closes #829.
@myronmarston myronmarston mentioned this pull request Apr 21, 2014
Merged
@myronmarston
Copy link
Member

Closing in favor of #1002.

myronmarston added a commit that referenced this pull request Apr 22, 2014
@myronmarston
Fix spec type inference.
0387ff2 
- In #970 we added `infer_spec_type_from_file_location!` but forgot
  to remove the code in `lib/rspec/rails/configuration.rb` that
  made it always infer - so the opt-in API was there but it
  was always enabled. Here I've made it truly opt-in.
- The logic of `infer_spec_type_from_file_location!` also setup
  the helper module inclusion via explicit `:type` metadata but
  was only intended to setup the implicit mapping of spec file
  location to type.  Here I've made the module inclusion via
  `:type` always work w/o an explicit config option needed to
  enable it.
- We used to mutate the `:type` metadata on inclusion of the helper
  module, but this caused bugs such as #825. The problem is that
  rspec-core uses a raw hash for the metadata and doesn't re-apply
  filtering/inclusion logic when the metadata hash is mutated.
  Instead, we use a new explicit `define_derived_metadata` API.

Fixes #825 and closes #829.
myronmarston added a commit to rspec/rspec-core that referenced this pull request Dec 17, 2014
@myronmarston
Remove spec for behavior we will no longer be supporting.
620269e 
Fully supporting metadata mutation is fraught with difficulties
see (rspec/rspec-rails#829 and the attempted fix in #1089).
For RSpec 3, we decided to expose `define_derived_metadata`
as an officially supported way to accomplish the sorts of
things metadata mutation was used for in RSpec 2. This spec
got left behind (since it was passing at the time, it wasn't
noticed) but isn’t something we want to support going forward
since module inclusion shouldn't mutate metadata. It’s also
getting in the way of a perf optimization I'm implementing.
myronmarston added a commit to rspec/rspec-core that referenced this pull request Dec 18, 2014
@myronmarston
Remove spec for behavior we will no longer be supporting.
9062f3a 
Fully supporting metadata mutation is fraught with difficulties
see (rspec/rspec-rails#829 and the attempted fix in #1089).
For RSpec 3, we decided to expose `define_derived_metadata`
as an officially supported way to accomplish the sorts of
things metadata mutation was used for in RSpec 2. This spec
got left behind (since it was passing at the time, it wasn't
noticed) but isn’t something we want to support going forward
since module inclusion shouldn't mutate metadata. It’s also
getting in the way of a perf optimization I'm implementing.
myronmarston added a commit to rspec/rspec-core that referenced this pull request Dec 19, 2014
@myronmarston
Remove spec for behavior we will no longer be supporting.
fa61873 
Fully supporting metadata mutation is fraught with difficulties
see (rspec/rspec-rails#829 and the attempted fix in #1089).
For RSpec 3, we decided to expose `define_derived_metadata`
as an officially supported way to accomplish the sorts of
things metadata mutation was used for in RSpec 2. This spec
got left behind (since it was passing at the time, it wasn't
noticed) but isn’t something we want to support going forward
since module inclusion shouldn't mutate metadata. It’s also
getting in the way of a perf optimization I'm implementing.
@cupakromer cupakromer deleted the run_request_before_all_hooks branch December 28, 2014 15:58
MatheusRich pushed a commit to MatheusRich/rspec-core that referenced this pull request Oct 30, 2020
@myronmarston
Remove spec for behavior we will no longer be supporting.
3b328c7 
Fully supporting metadata mutation is fraught with difficulties
see (rspec/rspec-rails#829 and the attempted fix in rspec#1089).
For RSpec 3, we decided to expose `define_derived_metadata`
as an officially supported way to accomplish the sorts of
things metadata mutation was used for in RSpec 2. This spec
got left behind (since it was passing at the time, it wasn't
noticed) but isn’t something we want to support going forward
since module inclusion shouldn't mutate metadata. It’s also
getting in the way of a perf optimization I'm implementing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Global before :all hooks with specified :type are not run with 2.14.0
3 participants
@JonRowe @myronmarston @alindeman