Fail JavaScript feature specs if any console messages are logged

[aduth]

Extracted from: #5481 (631d27efd42b0e314cbc9de87844cf2d01e6b3ec)

Why: Browser logging is unexpected, and should be treated as a failure, similar to what we already do with Mocha-based JavaScript tests via useConsoleLogSpy. Often, logging will indicate some failing of the JavaScript execution, which may or may not fail subsequent spec assertions. Raising the logging will often improve clarity of debugging the root cause.

[aduth]

The test failure is happening at the document capture step, where we use React Suspense + error boundary to handle the polling (source). So, in a sense, the logging is expected, though I've wondered if there's a way we can handle it more gracefully. Otherwise, we'd need to exempt this screen from the checking.

Related: facebook/react#15069

[zachmargolis]

LGTM

[zachmargolis]

maybe we do some sort like next if spec.metadata[:allow_js_error] and then add allow_js_error: true to the polling spec mentioned in #5492 (comment)

[aduth]

maybe we do some sort like next if spec.metadata[:allow_js_error] and then add allow_js_error: true to the polling spec mentioned in #5492 (comment)

Yeah, I like that idea. It's a bit trickier though, since any spec which would include document capture as part of its prerequisite setup would fail (e.g. complete_doc_auth_steps_before_verify_step), so it's not very self-contained. That being said, I only see 3 failures currently in Circle, so maybe it's fine enough.

Alternatively, we could target the offending script here as being allowed to log errors.

next if javascript_errors.reject { |e| e.include? 'js/document-capture-' }.blank?

Feels pretty gross, but also I'm really thinking maybe to refactor the document capture polling to not rely on these errors, so could be a temporary band-aid fix.

[zachmargolis]

I had an idea that inside thecomplete_doc_auth_steps_before_verify_step we could modify spec.metadata[:allow_js_error] = true ... however, ran into this error: :[

`metadata` is not available from within an example (e.g. an `it` block) or from
constructs that run in the scope of an example (e.g. `before`, `let`, etc). It is only
available on an example group (e.g. a `describe` or `context` block).

[aduth]

Many of the remaining failures smell of actual issues to me:

  1) doc auth link sent step behaves like with doc capture polling enabled automatically advances when the mobile flow is complete
     Failure/Error: raise BrowserConsoleLogError.new(javascript_errors) if javascript_errors.present?
     
     BrowserConsoleLogError:
       Unexpected browser console logging:
     
       http://127.0.0.1:41039/verify/doc_auth/link_sent - Blocked attempt to show a 'beforeunload' confirmation panel for a frame that never had a user gesture since its load. https://www.chromestatus.com/feature/5082396709879808
     Shared Example Group: "with doc capture polling enabled" called from ./spec/features/idv/doc_auth/link_sent_step_spec.rb:134
     # ./spec/rails_helper.rb:116:in `block (2 levels) in <top (required)>'
     # ./spec/rails_helper.rb:105:in `block (2 levels) in <top (required)>'

The onbeforeunload should be unbound before the redirect happens (source).

  2) Accessibility on static pages 500 page
     Failure/Error: raise BrowserConsoleLogError.new(javascript_errors) if javascript_errors.present?
     
     BrowserConsoleLogError:
       Unexpected browser console logging:
     
       http://127.0.0.1:41039/non_existent_page - Failed to load resource: the server responded with a status of 404 (Not Found)
     # ./spec/rails_helper.rb:116:in `block (2 levels) in <top (required)>'
     # ./spec/rails_helper.rb:105:in `block (2 levels) in <top (required)>'

Why is the 500 page test visiting this 404 page?

  3) Accessibility on pages that require authentication edit email page
     Failure/Error: raise BrowserConsoleLogError.new(javascript_errors) if javascript_errors.present?
     
     BrowserConsoleLogError:
       Unexpected browser console logging:
     
       http://127.0.0.1:41039/manage/email/712 - Failed to load resource: the server responded with a status of 404 (Not Found)
     # ./spec/rails_helper.rb:116:in `block (2 levels) in <top (required)>'
     # ./spec/rails_helper.rb:105:in `block (2 levels) in <top (required)>'

It's 404ing when trying to delete the sampled email address?

  4) doc auth test credentials allows proofing with test credentials
     Failure/Error: raise BrowserConsoleLogError.new(javascript_errors) if javascript_errors.present?
     
     BrowserConsoleLogError:
       Unexpected browser console logging:
     
       http://127.0.0.1:41039/verify/doc_auth/verify_document_status - Failed to load resource: the server responded with a status of 400 (Bad Request)
     # ./spec/rails_helper.rb:116:in `block (2 levels) in <top (required)>'
     # ./spec/features/idv/doc_auth/test_credentials_spec.rb:15:in `block (3 levels) in <top (required)>'
     # ./spec/features/idv/doc_auth/test_credentials_spec.rb:14:in `block (2 levels) in <top (required)>'
     # ./spec/rails_helper.rb:105:in `block (2 levels) in <top (required)>'

Shouldn't be hitting a 400 error on this test submission?