Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[close #50] Annotate rspec errors #136

Merged
merged 1 commit into from
Sep 22, 2020
Merged

[close #50] Annotate rspec errors #136

merged 1 commit into from
Sep 22, 2020

Conversation

schneems
Copy link
Contributor

For intermittent failures it's important to get as much debugging information as possible about the test in progress. This PR annotates any rspec expectation failures that happen in a deploy {} block with the application name. Here's an example of what it will look like:

     Failure/Error: expect(true).to eq(false)

       App: hatchet-t-873bcea87c (default_ruby)

       expected: false
            got: true

       (compared using ==)

       Diff:
       @@ -1 +1 @@
       -false
       +true

In this example if the failure was due to an app.run then we could manually app.run against the same app (assuming it had not been reaped yet) in order to better debug.

For intermittent failures it's important to get as much debugging information as possible about the test in progress. This PR annotates any rspec expectation failures that happen in a `deploy {}` block with the application name. Here's an example of what it will look like:

```
     Failure/Error: expect(true).to eq(false)

       App: hatchet-t-873bcea87c (default_ruby)

       expected: false
            got: true

       (compared using ==)

       Diff:
       @@ -1 +1 @@
       -false
       +true
``` 


In this example if the failure was due to an `app.run` then we could manually `app.run` against the same app (assuming it had not been reaped yet) in order to better debug.
@Malax
Copy link
Member

Malax commented Sep 22, 2020

Would it be possible to also get the example description (the sentence created with describe, context, and it)? Otherwise, this looks good and will greatly improve the debugging experience! 👍

@schneems
Copy link
Contributor Author

Would it be possible to also get the example description (the sentence created with describe, context, and it)? Otherwise, this looks good and will greatly improve the debugging experience! 👍

Since this is me adding additional info to rspec, yes that info will be there, but only for failing tests. For tests that are retried, the output will still be the same as today. I could maybe change that, but I would have to work upstream in the rspec-retry repo which I've not done before.

@schneems schneems merged commit 91347bb into main Sep 22, 2020
@schneems schneems deleted the schneems/test-metadata branch September 22, 2020 14:18
@schneems schneems mentioned this pull request Sep 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants