Track scanner (partial) refactor changes #1398
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
To handle escape sequences better, such as ignoring them in comments, the branch "revise-escape-sequence-scanning" was started as more major refactor.
However, that has become too large and too hard to get right. This PR splits the non-escape sequence portion of the changes. After this, the remaining escape sequence changes will be added back.
.github/workflows/*.yml: Until mini-tweaks branch on mathics-scanner is merged, we need to test this from the mini-tweaks branch
Use TranslateErrorNew, which has more information about the scanner error raised inside the exception
mathics/builtin/messages.py: correct Syntax URL
mathics/core/convert/sympy.py mpypy doesn't like using BasicsSympy as a class type (it is a variable of a class type?)
incomplete() -> get_more_input(). See PR in mathics_scanner for more info
tokeniser.sntx_message now returns the message parameters it used. These are passed along inside the exception handling
add parse_incrementally_by_line()
mathics/docpipeline.py tests now record what section and test number they came from. This does make error testing matching harder, but these are expected to get reduced and converted to pytests. Or use "..." in the doctests. We err on the side of information over vagueness and catering to a misguided testing system.
pyproject.toml: Bump minimum Mathics-Scanner needed. We are making incompatible changes in the Mathics-Scanner project again.
test/builtin/atomic/test_strings2.py: we were (I think) inadvertently unescaping some strings that were meant to be escape sequences
test/core/parser/test_parser.py: test result matches WMA closer