backtrace_formatter.rb - invalid byte sequence in US-ASCII (ArgumentError)

[MSP-Greg]

Subject of the issue

While running bundler tests in ruby/ruby (currently done with Azure Pipelines running an Ubuntu job and a MacOS job), the following error appears:

All in rspec-core-3.8.2/lib/rspec/core/

backtrace_formatter.rb:61:in `=~': invalid byte sequence in US-ASCII (ArgumentError)
backtrace_formatter.rb:61:in `block in matches?'
core/backtrace_formatter.rb:61:in `any?'
backtrace_formatter.rb:61:in `matches?'
backtrace_formatter.rb:55:in `exclude?'
backtrace_formatter.rb:50:in `backtrace_line'
backtrace_formatter.rb:37:in `block in format_backtrace'
backtrace_formatter.rb:37:in `map'
backtrace_formatter.rb:37:in `format_backtrace'
formatters/exception_presenter.rb:58:in `formatted_cause'

I work with trunk/master/2.7.0 and I've never come across this error before. Anyone have any ideas?

Your environment

• Ruby version: master/trunk
• rspec-core version: 3.8.2

Steps to reproduce

Run Ruby Azure Pipelines tests, the 'bundler' jobs'...

[JonRowe]

👋 Will need a bit more information to help here, this is likely an encoding error with the regular expression, so your config and command line usage would help

[MSP-Greg]

Jon,

I wanted to create an issue in case something else causes the same error. Sometime I'll investigate more, but I've got a few other 'fires' that need attending...

[MSP-Greg]

I just ran the same tests on Appveyor, and I got the same error. I briefly looked at the code:

rspec-core/lib/rspec/core/backtrace_formatter.rb

Lines 8 to 20 in 77dbc84

	def initialize
	@full_backtrace = false
	patterns = %w[ /lib\d*/ruby/ bin/ exe/rspec /lib/bundler/ /exe/bundle: ]
	patterns << "org/jruby/" if RUBY_PLATFORM == 'java'
	patterns.map! { |s| Regexp.new(s.gsub("/", File::SEPARATOR)) }
	@exclusion_patterns = [Regexp.union(RSpec::CallerFilter::IGNORE_REGEX, *patterns)]
	@inclusion_patterns = []
	return unless matches?(@exclusion_patterns, File.join(Dir.getwd, "lib", "foo.rb:13"))
	inclusion_patterns << Regexp.new(Dir.getwd)
	end

I may try and adjust the code and see if anything changes, but the only way I know how to repo it is running the bundler test suite with ruby-head/trunk.

There are a few parenthesis in the regex, and ?: isn't used. The Ruby docs for Rexgexp.union state that:

"The behavior is unspecified if any given pattern contains capture."

Not quite sure exactly what's meant by that...

[JonRowe]

"The behavior is unspecified if any given pattern contains capture."

I've always taken that to read as, "don't expect your captures to line up correctly"

[MSP-Greg]

I haven't thought about possible solutions, but this may be due to how Azure Pipelines deals with ANSI escape sequences, which don't affect the displayed logs...

Ok to close.

[JonRowe]

Mm.. I'd like a fix here, even if its just us issuing a warning about invalid escape chars...

[JonRowe]

(We know our Windows support isn't great, anything we can do to improve it is a good thing..)

[MSP-Greg]

You're testing on Appveyor (maybe add 2.6 or trunk?), so you're doing fine. I wonder if String#scrub would help here? Never used it.

Maybe something involving a str[/\e/] check?

[benoittgt]

You're testing on Appveyor (maybe add 2.6 or trunk?), so you're doing fine.

At the moment we don't :/

https://github.com/rspec/rspec-core/blob/master/appveyor.yml#L34-L41

Added 2.6 : rspec/rspec-dev#220

[JonRowe]

I'd like to see if 2.6 has this issue for us, or otherwise replicate it on appveyor, @benoittgt do you want to take care of that?

[MSP-Greg]

Bundler has some color 'add-ins' that are used in their testing, and the issue is only ocurring when bundler is tested as it exists in the ruby/ruby repo.

I'm assuming the code in question is only called when a backtrace needs to be formatted.

It may be a few days, but what I'll do is add a patch for backtrace_formatter.rb to the Azure Pipelines testing to filter the ANSI color code strings, and see if the error goes away. As mentioned, this probably also involves how Azure Pipelines handles ANSI color code strings...

[pirj]

Since 2.7 is out, I've opened #2688 and it failed hard https://ci.appveyor.com/project/rspec/rspec-core/builds/30139185
Any volunteers to move this further?