Skip to content

Make specs pending instead of failing in no multicast environments #9566

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
waj merged 2 commits into from
Jul 6, 2020
Merged

Make specs pending instead of failing in no multicast environments #9566

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5c31c9d
Spec: Add support to mark an example as pending in-flight
jhass Jun 6, 2020
be485fa
Mark UDP multicast specs as pending when multicast is unavailable
jhass Jul 2, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 20 additions & 2 deletions spec/std/socket/udp_socket_spec.cr
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,14 +78,32 @@ describe UDPSocket do
expect_raises(Socket::Error, "Unsupported IP address family: INET. For use with IPv6 only") do
udp.multicast_interface 0
end
udp.multicast_interface Socket::IPAddress.new(unspecified_address, 0)

begin
udp.multicast_interface Socket::IPAddress.new(unspecified_address, 0)
rescue e : Socket::Error
if e.os_error == Errno::ENOPROTOOPT
pending!("Multicast device selection not available on this host")
else
raise e
end
end

Socket::IPAddress.new("224.0.0.254", port)
when Socket::Family::INET6
expect_raises(Socket::Error, "Unsupported IP address family: INET6. For use with IPv4 only") do
udp.multicast_interface(Socket::IPAddress.new(unspecified_address, 0))
end
udp.multicast_interface(0)

begin
udp.multicast_interface(0)
rescue e : Socket::Error
if e.os_error == Errno::ENOPROTOOPT
pending!("Multicast device selection not available on this host")
else
raise e
end
end

Socket::IPAddress.new("ff02::102", port)
else
Expand Down
4 changes: 4 additions & 0 deletions src/spec/dsl.cr
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,10 @@ module Spec
class AssertionFailed < SpecError
end

# :nodoc:
class ExamplePending < SpecError
end

# :nodoc:
class NestingSpecError < SpecError
end
Expand Down
2 changes: 2 additions & 0 deletions src/spec/example.cr
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ module Spec
rescue ex : Spec::AssertionFailed
@parent.report(:fail, description, file, line, Time.monotonic - start, ex)
Spec.abort! if Spec.fail_fast?
rescue ex : Spec::ExamplePending
@parent.report(:pending, description, file, line, Time.monotonic - start)
Copy link
Copy Markdown
Contributor

@Sija Sija Jul 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ex.message is not being used anywhere, is that intentional?

Copy link
Copy Markdown
Member Author

@jhass jhass Jul 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now yes.

While I felt it important to be able to specify a reason in the pending! call, making it clearer why that call is happening all of the sudden, we currently have no holistic approach to pending reasons. That is the pending method does not accept a reason in any form and there's no existing reporting format/infrastructure for this. I think pending should gain a way to collect reasons and then we should update the formatter to present them, but I felt introducing that holistic approach here to be a bit too much for what it wants to do, also because there's probably a lot more opinion on how that should be done.

rescue ex
@parent.report(:error, description, file, line, Time.monotonic - start, ex)
Spec.abort! if Spec.fail_fast?
Expand Down
60 changes: 42 additions & 18 deletions src/spec/methods.cr
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ module Spec::Methods
#
# Example:
# ```
# require "spec"
#
# describe "Int32" do
# describe "+" do
# it "adds" { (1 + 1).should eq 2 }
Expand Down Expand Up @@ -35,6 +37,8 @@ module Spec::Methods
#
# Example:
# ```
# require "spec"
#
# it "adds" { (1 + 1).should eq 2 }
# ```
#
Expand All @@ -52,6 +56,8 @@ module Spec::Methods
#
# Example:
# ```
# require "spec"
#
# pending "check cat" { cat.alive? }
# ```
#
Expand All @@ -76,6 +82,24 @@ module Spec::Methods
raise Spec::AssertionFailed.new(msg, file, line)
end

# Marks the current example pending
#
# In case an example needs to be pending on some condition that requires executing it,
# this allows to mark it as such rather than letting it fail or never run.
#
# ```
# require "spec"
#
# it "test git" do
# cmd = Process.find_executable("git")
# pending!("git is not available") unless cmd
# cmd.ends_with?("git").should be_true
# end
# ```
def pending!(msg = "Cannot run example", file = __FILE__, line = __LINE__)
raise Spec::ExamplePending.new(msg, file, line)
end

# Executes the given block before each spec in the current context runs.
#
# A context is defined by `describe` or `context` blocks, or outside of them
Expand All @@ -87,16 +111,16 @@ module Spec::Methods
# order of definition.
#
# ```
# require "spec
# require "spec"
#
# it "sample_a" {}
# it "sample_a" { }
#
# describe "nested_context" do
# before_each do
# puts "runs before sample_b"
# end
#
# it "sample_b" {}
# it "sample_b" { }
# end
# ```
def before_each(&block)
Expand All @@ -117,16 +141,16 @@ module Spec::Methods
# order of definition.
#
# ```
# require "spec
# require "spec"
#
# it "sample_a" {}
# it "sample_a" { }
#
# describe "nested_context" do
# after_each do
# puts "runs after sample_b"
# end
#
# it "sample_b" {}
# it "sample_b" { }
# end
# ```
def after_each(&block)
Expand All @@ -147,16 +171,16 @@ module Spec::Methods
# order of definition.
#
# ```
# require "spec
# require "spec"
#
# it "sample_a" {}
# it "sample_a" { }
#
# describe "nested_context" do
# before_all do
# puts "runs at start of nested_context"
# end
#
# it "sample_b" {}
# it "sample_b" { }
# end
# ```
def before_all(&block)
Expand All @@ -177,16 +201,16 @@ module Spec::Methods
# order of definition.
#
# ```
# require "spec
# require "spec"
#
# it "sample_a" {}
# it "sample_a" { }
#
# describe "nested_context" do
# after_all do
# puts "runs at end of nested_context"
# end
#
# it "sample_b" {}
# it "sample_b" { }
# end
# ```
def after_all(&block)
Expand All @@ -213,9 +237,9 @@ module Spec::Methods
# order of definition.
#
# ```
# require "spec
# require "spec"
#
# it "sample_a" {}
# it "sample_a" { }
#
# describe "nested_context" do
# around_each do |example|
Expand All @@ -224,7 +248,7 @@ module Spec::Methods
# puts "runs after sample_b"
# end
#
# it "sample_b" {}
# it "sample_b" { }
# end
# ```
def around_each(&block : Example::Procsy ->)
Expand All @@ -250,7 +274,7 @@ module Spec::Methods
# order of definition.
#
# ```
# require "spec
# require "spec"
#
# describe "main_context" do
# around_each do |example|
Expand All @@ -259,7 +283,7 @@ module Spec::Methods
# puts "runs at end of main_context"
# end
#
# it "sample_a" {}
# it "sample_a" { }
#
# describe "nested_context" do
# around_each do |example|
Expand All @@ -268,7 +292,7 @@ module Spec::Methods
# puts "runs at end of nested_context"
# end
#
# it "sample_b" {}
# it "sample_b" { }
# end
# end
# ```
Expand Down