Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI: Some tests about signal handlers sometimes fail #4063

Closed
daipom opened this issue Feb 18, 2023 · 7 comments · Fixed by #4671
Closed

CI: Some tests about signal handlers sometimes fail #4063

daipom opened this issue Feb 18, 2023 · 7 comments · Fixed by #4671
Assignees
@Watson1978 @daipom
Labels
bug Something isn't working

Comments

@daipom
Copy link
Contributor

daipom commented Feb 18, 2023

Describe the bug

test_cont_in_main_process_signal_handlers and test_cont_in_supervisor_signal_handler of SupervisorTest sometimes fail.

===============================================================================
Failure: test_cont_in_main_process_signal_handlers(SupervisorTest):
  class()
  Called 0 times.
  Expected 1 times.
/opt/hostedtoolcache/Ruby/3.0.5/x64/lib/ruby/gems/3.0.0/gems/sigdump-0.2.4/lib/sigdump.rb:74:in `block in dump_object_count'
/opt/hostedtoolcache/Ruby/3.0.5/x64/lib/ruby/gems/3.0.0/gems/sigdump-0.2.4/lib/sigdump.rb:73:in `each_object'
/opt/hostedtoolcache/Ruby/3.0.5/x64/lib/ruby/gems/3.0.0/gems/sigdump-0.2.4/lib/sigdump.rb:73:in `dump_object_count'
/opt/hostedtoolcache/Ruby/3.0.5/x64/lib/ruby/gems/3.0.0/gems/sigdump-0.2.4/lib/sigdump.rb:18:in `block in dump'
/opt/hostedtoolcache/Ruby/3.0.5/x64/lib/ruby/gems/3.0.0/gems/sigdump-0.2.4/lib/sigdump.rb:136:in `open'
/opt/hostedtoolcache/Ruby/3.0.5/x64/lib/ruby/gems/3.0.0/gems/sigdump-0.2.4/lib/sigdump.rb:136:in `_open_dump_path'
/opt/hostedtoolcache/Ruby/3.0.5/x64/lib/ruby/gems/3.0.0/gems/sigdump-0.2.4/lib/sigdump.rb:14:in `dump'
/home/runner/work/fluentd/fluentd/lib/fluent/supervisor.rb:1027:in `dump_non_windows'
/home/runner/work/fluentd/fluentd/lib/fluent/supervisor.rb:943:in `block in install_main_process_signal_handlers'
/home/runner/work/fluentd/fluentd/test/test_supervisor.rb:221:in `kill'
/home/runner/work/fluentd/fluentd/test/test_supervisor.rb:221:in `test_cont_in_main_process_signal_handlers'
     218:     sv = Fluent::Supervisor.new(opts)
     219:     sv.send(:install_main_process_signal_handlers)
     220: 
  => 221:     Process.kill :CONT, Process.pid
     222: 
     223:     sleep 1
     224: 
===============================================================================
F
===============================================================================
Failure: test_cont_in_supervisor_signal_handler(SupervisorTest):
  class()
  Called 0 times.
  Expected 1 times.
/opt/hostedtoolcache/Ruby/3.0.5/x64/lib/ruby/gems/3.0.0/gems/sigdump-0.2.4/lib/sigdump.rb:74:in `block in dump_object_count'
/opt/hostedtoolcache/Ruby/3.0.5/x64/lib/ruby/gems/3.0.0/gems/sigdump-0.2.4/lib/sigdump.rb:73:in `each_object'
/opt/hostedtoolcache/Ruby/3.0.5/x64/lib/ruby/gems/3.0.0/gems/sigdump-0.2.4/lib/sigdump.rb:73:in `dump_object_count'
/opt/hostedtoolcache/Ruby/3.0.5/x64/lib/ruby/gems/3.0.0/gems/sigdump-0.2.4/lib/sigdump.rb:18:in `block in dump'
/opt/hostedtoolcache/Ruby/3.0.5/x64/lib/ruby/gems/3.0.0/gems/sigdump-0.2.4/lib/sigdump.rb:136:in `open'
/opt/hostedtoolcache/Ruby/3.0.5/x64/lib/ruby/gems/3.0.0/gems/sigdump-0.2.4/lib/sigdump.rb:136:in `_open_dump_path'
/opt/hostedtoolcache/Ruby/3.0.5/x64/lib/ruby/gems/3.0.0/gems/sigdump-0.2.4/lib/sigdump.rb:14:in `dump'
/home/runner/work/fluentd/fluentd/lib/fluent/supervisor.rb:1027:in `dump_non_windows'
/home/runner/work/fluentd/fluentd/lib/fluent/supervisor.rb:943:in `block in install_main_process_signal_handlers'
/home/runner/work/fluentd/fluentd/test/test_supervisor.rb:305:in `kill'
/home/runner/work/fluentd/fluentd/test/test_supervisor.rb:305:in `test_cont_in_supervisor_signal_handler'
     302:     server = DummyServer.new
     303:     server.install_supervisor_signal_handlers
     304: 
  => 305:     Process.kill :CONT, Process.pid
     306: 
     307:     sleep 1
     308: 
===============================================================================

This is similar to 0d126de

To Reproduce

I can't reproduce this on my local yet.

In CI, sometimes these tests fail on Ubuntu.

Expected behavior

The tests always succeed.

Your Environment

None

Your Configuration

None

Your Error Log

None

Additional context

No response
@daipom daipom self-assigned this Mar 16, 2023
@daipom daipom moved this to To-Do in Fluentd Kanban Mar 16, 2023
@ashie ashie added the bug Something isn't working label Mar 31, 2023
@daipom daipom changed the title Some tests about signal handlers sometimes fail CI: Some tests about signal handlers sometimes fail Apr 10, 2023
@daipom daipom removed the status in Fluentd Kanban Jun 23, 2023
@Watson1978 Watson1978 self-assigned this Oct 8, 2024
@daipom
Copy link
Contributor Author

daipom commented Oct 8, 2024

I'm not sure, but I remember this issue started after the following fix.

@Watson1978
Copy link
Contributor

I'm guessing that the issue is related to dont_allow in rr gem.
If it use the object applied dont_allow in the other test through ObjectSpace.each_object, it will cause the error.

require 'bundler/inline'
gemfile do
  source 'https://rubygems.org'
  gem 'rr'
  gem 'test-unit'
  gem 'test-unit-rr'
end

class SampleTest < ::Test::Unit::TestCase
  def test_a
    dont_allow(mock!).foo
  end

  def test_b
    ObjectSpace.each_object { |obj|
      obj.class
    }
  end
end
$ ruby rr_1.rb
Loaded suite rr_1
Started
F
=================================================================================================================================================================
Failure: test_b(SampleTest):
  Called 1 time.
  Expected 0 times.
rr_1.rb:16:in `block in test_b'
     13: 
     14:   def test_b
     15:     ObjectSpace.each_object { |obj|
  => 16:       obj.class
     17:     }
     18:   end
     19: end
rr_1.rb:15:in `each_object'
rr_1.rb:15:in `test_b'
=================================================================================================================================================================
Finished in 0.001690435 seconds.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
2 tests, 0 assertions, 1 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
50% passed
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
1183.13 tests/s, 0.00 assertions/s

However, if it applied never instead of dont_allow, it works well.

require 'bundler/inline'
gemfile do
  source 'https://rubygems.org'
  gem 'rr'
  gem 'test-unit'
  gem 'test-unit-rr'
end

class SampleTest < ::Test::Unit::TestCase
  def test_a
    mock!.foo.never
  end

  def test_b
    ObjectSpace.each_object {|obj|
      obj.class
    }
  end
end
$ ruby rr_2.rb
Loaded suite rr_2
Started
Finished in 0.002811038 seconds.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
2 tests, 0 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
711.48 tests/s, 0.00 assertions/s

@Watson1978
Copy link
Contributor

The reason this occurs only occasionally is probably because the object applied dont_allow was collected by GC.

If the object is alived, looks for me it appears to be causing problems.

@Watson1978 Watson1978 mentioned this issue Oct 16, 2024
Merged
@daipom
Copy link
Contributor Author

daipom commented Oct 17, 2024

@Watson1978 Thanks!!
Looks like dont_allow of RR has some issues.

require 'bundler/inline'
gemfile do
  source 'https://rubygems.org'
  gem 'rr'
  gem 'test-unit'
  gem 'test-unit-rr'
end

class SampleTest < ::Test::Unit::TestCase
  def test_a
    dont_allow!.foo
  end

  def test_b
    ObjectSpace.each_object do |obj|
      obj.class
    end
  end
end
$ ruby test.rb
Loaded suite test
Started
F
====================================================================================================================================================================
Failure: test_b(SampleTest):
  Called 1 time.
  Expected 0 times.
test.rb:16:in `block in test_b'
     13: 
     14:   def test_b
     15:     ObjectSpace.each_object do |obj|
  => 16:       obj.class
     17:     end
     18:   end
     19: end
test.rb:15:in `each_object'
test.rb:15:in `test_b'
====================================================================================================================================================================
Finished in 0.003105795 seconds.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
2 tests, 0 assertions, 1 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
50% passed
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
643.96 tests/s, 0.00 assertions/s

@daipom
Copy link
Contributor Author

daipom commented Oct 17, 2024

It appears that #dont_allow / #dont_allow! API of RR has some issues.

@Watson1978 Could you please send an issue to https://github.com/rr/rr?

@Watson1978
Copy link
Contributor

Watson1978 commented Oct 17, 2024 •
edited
Loading

Maybe, I think dont_allow has right behavior.
Fluentd's tests have super rare case because its using ObjectSpace.each_object.
Usually, it does not refer to a temporary object generated by another test....

@daipom daipom mentioned this issue Oct 18, 2024
Closed
@daipom
Copy link
Contributor Author

daipom commented Oct 18, 2024

Hmm, I'm concerned about this dont_allow behavior, so I have sent an issue.
Thanks for your investigation!

@github-project-automation github-project-automation bot moved this from Work-In-Progress to Done in Fluentd Kanban Oct 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Watson1978 Watson1978

@daipom daipom

Labels
bug Something isn't working
Projects
Fluentd Kanban
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

tests: use never instead of dont_allow Watson1978/fluentd
3 participants
@ashie @Watson1978 @daipom