fix #16256: nimout: <empty> should give error (vacuously true); improve a few tests #18089
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Merged
timotheecour
added
the
Ready For Review
timotheecour
changed the title
Araq
reviewed
May 26, 2021
testament/specs.nim
Outdated
| @@ -401,6 +403,9 @@ proc parseSpec*(filename: string): TSpec = | |||
| if skips.anyIt(it in result.file): | |||
| result.err = reDisabled | |||
|
|
|||
| if nimoutFound and result.nimout.len == 0 and not result.nimoutFull: | |||
| doAssert false, "empty `nimout` is vacuously true, use `nimoutFull:true` if intentional" | |||
There was a problem hiding this comment.
Report spec errors as errors, doAssert is not a mechanism for reporting errors. Errors are not bugs.
…); improve some tests
timotheecour
force-pushed
the
pr_fix_16256_testament_nimout_empty
branch
from
be3162a
to
1994128
Compare
|
I can't comment on the testament changes, but the test changed look good (it's a big improvement!). |
2 tasks
PMunch
pushed a commit
to PMunch/Nim
that referenced
this pull request
Mar 28, 2022
…); improve a few tests (nim-lang#18089) * fix nim-lang#16256: nimout: <empty> should give error (vacuously true); improve some tests * renamed: tests/stdlib/t9710.nim -> tests/misc/t9710.nim * improve tests * fix non-DRY tests * improve $nim_prs_D/tests/stdlib/t9091.nim * renamed: tests/stdlib/t9091.nim -> tests/misc/t9091.nim * fixup * address comment: doAssert => result.parseErrors
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
nimout: <empty>should give error (vacuously true) #16256eg:
targets: "c"is bad (prevents honoring $target which can be cpp in CI via NIM_COMPILE_TO_CPP for eg)action: "compile"whereas the test should actually runexitcode: 0oraction: "run"which are the defaulttimeout: 60.0which are there for no valid reasonnimout: ""which is vacuously true (ie testament:nimout: <empty>should give error (vacuously true) #16256, which i'm fixing in this PR)--styleCheck:hint --panics:onwhich have no effect as far as the test is concernedoutput:withaction: "compile") or don't check the resultstos.nimoutput: "test AObj"which should be replaced by doAssert / check(also fixes #13952 (comment), #13952 (comment))
(not that there are exceptions to above rules but they're rare and don't apply in this PR)
future work
testament should give doAssert/spec error when it detects bad combinations instead of silently accepting and succeeding with some arbitrary resolution, eg:
action: "compile"+outputshould errortargets: "c cpp"+cmd: "nim c ..."should errorideally, it would also give error under some
--specChecktestament flag for vacuously true spec flags or suspicious combinations of flags to help users improve their tests