Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggestion: use TAP output format #601

Closed
jasonkarns opened this issue Jun 4, 2022 · 3 comments · Fixed by #1749
Closed

Suggestion: use TAP output format #601

jasonkarns opened this issue Jun 4, 2022 · 3 comments · Fixed by #1749
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@jasonkarns
Copy link

TAP output is a cross platform format for emitting “test” results that is simple to emit and simple to parse. It has wide support across many language platforms. It also has the benefit of being a “baseline” output format because there are many tools that can convert TAP o other formats. As such, the ability to emit TAP then makes it trivial to pipe hurl to other tools to covert the out into the format of their choice (without hurl needing to support dozens of different formatters)

http://testanything.org/

@fabricereix
Copy link
Collaborator

Hello @jasonkarns,
It could be indeed a good output format for Hurl tests. It seems suited for command-line apps.
Do you have any samples input Hurl file/TAP expected output?
Thanks

@jcamiel jcamiel added the enhancement New feature or request label Jun 9, 2022
@fourjay
Copy link

fourjay commented Aug 22, 2022

FWIW, here's a web page devoted to TAP
https://testanything.org/
There's a "typical" output example on the root page.
There's also a full spec
https://testanything.org/tap-version-14-specification.html

quick description from memory:
first line is number of tests (in FN..LN format)
Then each test reports one line per test (if successful)
OK or not OK
test number
dash
description.

It'd probably be reasonable to have a description field as part of the hurl DSL.
That seems a reasonable step anyways.
I do not think the description is required,
but it could make the reporting a lot more useful, and not just here
It also creates a hook for "self documenting" tests

@jcamiel jcamiel added this to the 4.0.0 milestone Jun 9, 2023
@fabricereix fabricereix modified the milestones: 4.0.0, 4.1.0 Jun 28, 2023
@fabricereix fabricereix self-assigned this Jun 29, 2023
@fabricereix fabricereix linked a pull request Jul 11, 2023 that will close this issue
@jasonkarns
Copy link
Author

slick! 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants