Skip to content

Conversation

@rocky
Copy link
Member

@rocky rocky commented May 21, 2025

To handle escape sequences better, such as ignoring them in comments, the branch "revise-escape-sequence-scanning" was started as more major refactor.

However, that has become too large and too hard to get right. This PR splits the non-escape sequence portion of the changes. After this, the remaining escape sequence changes will be added back.

  • .github/workflows/*.yml: Until mini-tweaks branch on mathics-scanner is merged, we need to test this from the mini-tweaks branch

  • Use TranslateErrorNew, which has more information about the scanner error raised inside the exception

  • mathics/builtin/messages.py: correct Syntax URL

  • mathics/core/convert/sympy.py mpypy doesn't like using BasicsSympy as a class type (it is a variable of a class type?)

  • incomplete() -> get_more_input(). See PR in mathics_scanner for more info

  • tokeniser.sntx_message now returns the message parameters it used. These are passed along inside the exception handling

  • add parse_incrementally_by_line()

  • mathics/docpipeline.py tests now record what section and test number they came from. This does make error testing matching harder, but these are expected to get reduced and converted to pytests. Or use "..." in the doctests. We err on the side of information over vagueness and catering to a misguided testing system.

  • pyproject.toml: Bump minimum Mathics-Scanner needed. We are making incompatible changes in the Mathics-Scanner project again.

  • test/builtin/atomic/test_strings2.py: we were (I think) inadvertently unescaping some strings that were meant to be escape sequences

  • test/core/parser/test_parser.py: test result matches WMA closer

@rocky rocky force-pushed the mini-tweaks branch 12 times, most recently from 922646e to f603b75 Compare May 21, 2025 22:51
@rocky rocky requested a review from mmatera May 22, 2025 16:37
@rocky rocky changed the title WIP Towards scanner refactor Track scanner (partial) refactor changes May 22, 2025
@rocky rocky force-pushed the mini-tweaks branch 2 times, most recently from 7b1d6f6 to 8b2fd9a Compare May 23, 2025 19:12
Copy link
Contributor

@mmatera mmatera left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apart from the typos, LGTM


success = test_case(
doctest,
f"<test-{section.title}-{index}>",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea!

return the result, the source code for this and evaluated
messages created in evaluation.
If there was was a TranslateError, the source code returned is "" and the result is None.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was was -> was

@rocky rocky merged commit 9a2c3e2 into master May 25, 2025
12 of 14 checks passed
@rocky rocky deleted the mini-tweaks branch May 25, 2025 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants