builtins.genericClosure: improve errors#14499
Merged
Ericson2314 merged 3 commits intoNixOS:masterfrom Nov 7, 2025
Merged
Conversation
Covers error conditions for: - Invalid argument types (not an attrset) - Missing required attributes (startSet, operator) - Type mismatches (startSet/operator not correct type) - Element validation (elements not attrsets, missing key attribute) - Key comparison errors (incompatible types, uncomparable types) - Operator return value validation (not a list)
github-actions
bot
added
the
with-tests
Show which element(s) are involved at each error point: - When an element is missing the "key" attribute, show the element - When an element is not an attribute set, show the element - When comparing keys fails, show both elements being compared - When calling operator fails, show which element was being processed This provides concrete context using ValuePrinter with errorPrintOptions. Note: errorPrintOptions uses maxDepth=10 by default, which may print quite deeply nested structures in error messages. This could potentially be overwhelming, but follows the existing default for error contexts.
We now have functional tests for these. The unit tests added negligible
value while imposing a much higher maintenance cost.
The maintenance cost is high:
- No automatic accept option
- They broke 5+ times during this session due to implementation changes (trace count, ordering)
- They require understanding ANSI escape codes, Uncolored() wrappers, trace reversal
- They test empty traces HintFmt("") from withTrace(pos, "") - pure implementation detail
- They're fragile: adding any trace anywhere breaks the exact count assertions
The additional value over functional tests is minimal:
- Functional tests already verify the error message
- Functional tests already show trace order and content (as users see it, helps review)
- Unit tests verify "exactly 3 traces, not 2 or 4" - but users don't count traces
- Unit tests verify empty traces exist - but users never see them
The white-box testing catches the wrong things:
- It catches "you added helpful context" as a failure
- It doesn't catch "the context is confusing" (which functional tests would show)
- It enforces implementation details that should be allowed to evolve
roberth
force-pushed
the
genericClosure-errors
branch
from
4aec461 to
3ee8e45
Compare
2 tasks
roberth
added
error-messages
xokdvium
reviewed
Nov 7, 2025
| ASSERT_TRACE2( | ||
| "genericClosure { startSet = [ true ]; operator = item: [{ key = ''a''; }]; }", | ||
| TypeError, | ||
| HintFmt("expected a set but found %s: %s", "a Boolean", Uncolored(ANSI_CYAN "true" ANSI_NORMAL)), |
Contributor
There was a problem hiding this comment.
FWIW I did add HasSubstrIgnoreANSIMatcher for similar cases so that one doesn't have to worry about exact colors in unit tests. That could alleviate some of the issues.
(Posting this primarily for visibility. I've been using that utility in unit tests where I needed to validate error messages in a somewhat concise manner)
But yeah, it's not worth keeping these tests considering how low-value and fragile they are.
Ericson2314
approved these changes
Nov 7, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The messages weren't too helpful.
Review commit by commit:
Context
Add 👍 to pull requests you find important.
The Nix maintainer team uses a GitHub project board to schedule and track reviews.