Tests for MakeBoxes #1579

mmatera · 2025-12-30T02:11:20Z

This PR activates some pytests I wrote a while ago, -and adds more- to start quantifying the progress in the MakeBoxes refactor. Ideally, test/builtin/test_makeboxes.py should start to pass more tests as we make progress in driving the behavior to a one compatible with WMA.

rocky · 2025-12-30T10:19:04Z

test/builtin/test_makeboxes.py

+        (
+            r"MakeBoxes[3.142`3]",
+            r"3.142`3",
+            "StandadForm with Prec real shows all the stored digits, and prec",


Suggested change

"StandadForm with Prec real shows all the stored digits, and prec",

"StandardForm with real precision shows all the stored digits, and precision",

rocky · 2025-12-30T11:40:44Z

test/builtin/test_makeboxes.py

    [
-        (r"MakeBoxes[1.4]", r"1.4`", None),
-        (r"MakeBoxes[1.4`]", r"1.4`", None),
+        (r"MakeBoxes[1.4`]", r"1.4`", "StandardForm always shows a prec mark."),


Suggested change

(r"MakeBoxes[1.4`]", r"1.4`", "StandardForm always shows a prec mark."),

(r"MakeBoxes[1.4`]", r"1.4`", "StandardForm always shows a precision mark."),

rocky · 2025-12-30T11:41:45Z

test/builtin/test_makeboxes.py

+        (
+            r"MakeBoxes[OutputForm[3.142`3]]",
+            r'InterpretationBox[PaneBox["3.14"], 3.14, Editable -> False]',
+            "OutputForm trims digits up to prec.",


Suggested change

"OutputForm trims digits up to prec.",

"OutputForm trims digits up to precision.",

The abbreviation "prec" can confuse me to mean precedence or preceding. While I see this as a parameter name in some WMA documentation, outside of that I am not seeing this in common use as a shorthand for precision.

rocky · 2025-12-31T15:08:42Z

test/builtin/test_makeboxes.py

+        ("MakeBoxes[x]", '"x"', "StandardForm"),
+        (
+            "MakeBoxes[InputForm[x]]",
+            'InterpretationBox[StyleBox["x", ShowStringCharacters -> True, NumberMarks -> True], InputForm[x], Editable -> True, AutoDelete -> True]',


WMA has x, instead of InputForm[x] and there are no quotes around "x" in StyleBox["x"]. What gives?

rocky · 2025-12-31T15:10:54Z

test/builtin/test_makeboxes.py

+        ),
+        (
+            "MakeBoxes[OutputForm[x]]",
+            'InterpretationBox[PaneBox[""x""], OutputForm[x], Editable -> False]',


Also I see a duplicate set of quotes. I am wondering about the merits of having exhaustive tests that are wrong, versus having limited tests that are right with tests that are skipped that need to be worked on.

I am trying to get useful references from WMA. This is exactly what I get when I evaluate MakeBoxes[OutputForm[x]]//InputForm in the WMA CLI.

What I see is:

InterpretationBox[PaneBox["\"x\"", BaselinePosition -> Baseline], x, Editable -> False]

Note the extra backslashes. This is slightly different.

rocky · 2025-12-31T15:13:52Z

test/builtin/test_makeboxes.py

            r"MakeBoxes[3.142`3]",
            r"3.142`3",
-            "StandardForm with PrecisionReal shows all the stored digits, and precision",
+            "StandadForm with PrecisionReal shows all the stored digits, and precision.",


Seems like a regression here.

mmatera · 2026-01-02T21:34:40Z

With this change, we can do something like this:

./convert_yaml2json.py makeboxes_tests.yaml &  wolframscript -f makeboxes_tests.m

and compare with

./convert_yaml2json.py makeboxes_tests.yaml &  mathics -f makeboxes_tests.m

This is the output of the first command (using wolframscript):

Read json
Found 4 expressions to test.

Basic Forms
===========

  Arithmetic
  ==========
    FullForm   [OK]
    InputForm   [OK]
    OutputForm   [OK]
    StandardForm   [OK]
    TeXForm   [OK]

  Expression
  ==========
    Format   [OK]
    TraditionalForm   [OK]
    FullForm   [OK]
    InputForm   [OK]
    OutputForm   [OK]
    StandardForm   [OK]
    TeXForm   [OK]

  Integer_negative
  ================
    FullForm   [OK]
    InputForm   [OK]
    OutputForm   [OK]
    StandardForm   [OK]
    TeXForm   [OK]

  Integer_positive
  ================
    FullForm   [OK]
    InputForm   [OK]
    OutputForm   [OK]
    StandardForm   [OK]
    TeXForm   [OK]

  PrecisionReal
  =============
    FullForm   [OK]
    InputForm   [OK]
    OutputForm   [OK]
    StandardForm   [OK]
    TeXForm   [OK]

  Symbol
  ======
    FullForm   [OK]
    InputForm   [OK]
    OutputForm   [OK]
    StandardForm   [OK]
    TeXForm   [OK]

Numbers
=======

  MachineReal, positive
  =====================
    StandardForm   [OK]
    OutputForm   [OK]

  MachineReal, negative
  =====================
    StandardForm   [OK]
    OutputForm   [OK]

  MachineReal, Large
  ==================
    StandardForm   [OK]

  MachineReal, Very Large
  =======================
    StandardForm   [OK]

  PrecisionReal, Few Digits
  =========================
    StandardForm   [OK]
    OutputForm   [OK]

  PrecisionReal, Many Digits
  ==========================
    StandardForm   [OK]

  PrecisionReal, Many Accuracy
  ============================
    StandardForm   [OK]

  PrecisionReal, Zero_LowPrec
  ===========================
    StandardForm   [OK]

  PrecisionReal, Zero_LargePrec
  =============================
    StandardForm   [OK]

  PrecisionReal, Zero_Accuracy
  ============================
    StandardForm   [OK]

Graphics
========

  Disk
  ====
    StandardForm   [OK]
    OutputForm   [OK]

  Sphere
  ======
    StandardForm   [OK]
    OutputForm   [OK]

Parsing string form
===================

  RowBox
  ======
    StandardForm   [OK]

  FractionBox
  ===========
    StandardForm   [OK]

  FractionBox bracket
  ===================
    StandardForm   [OK]

  SqrtBox
  =======
    StandardForm   [OK]

  FormBox
  =======
    StandardForm   [OK]

test/format/makeboxes_tests.yaml

rocky · 2026-01-02T21:58:36Z

There are a lot of changes here. I will try to pull this branch locally and go over it in detail tomorrow.

…SuperscriptBox with a nested expression.>>

test/format/makeboxes_tests.m

test/format/makeboxes_tests.yaml

rocky · 2026-01-03T16:34:57Z

LGTM - I tried this out locally, but there are just way too many tests, and there are way too many changes to be able to verify this. I guess we'll make corrections down the line.

Co-authored-by: R. Bernstein <[email protected]>

rocky · 2026-01-03T16:47:29Z

test/format/makeboxes_tests.yaml

+    StandardForm:
+      expr: MakeBoxes[0.``30]
+      expect: "\"0``30.\""
+      msg: "In Mathics, precision is always an integer number."


Suggested change

msg: "In Mathics, precision is always an integer number."

msg: "In Mathics3, precision is always an integer number."

rocky · 2026-01-03T16:47:52Z

test/format/makeboxes_tests.yaml

+    StandardForm:
+      expr: MakeBoxes[1.4`20]
+      expect: "\"1.4`20.\""
+      msg: "In Mathics, precision is always an integer number."


Please let's use Mathics3 everywhere.

rocky · 2026-01-03T16:49:25Z

test/format/makeboxes_tests.yaml

+      expect: "GraphicsBox[{DiskBox[{0, 0}, 1]}]"
+    OutputForm:
+      expr: "MakeBoxes[Graphics[{Disk[{0,0}, 1]}]//OutputForm]"
+      expect: InterpretationBox[PaneBox["\"-Graphics-\""], OutputForm[Graphics[{Disk[{0, 0}, 1]}]], Editable -> False]


In wolframscript we don't see InterpretationBox, but instead GraphicsBox. Is this a notebook thing, and if so, the idea is that we follow notebook when there are discrepencies?

And when I try this in the current mathics3 I get:

In[1]= MakeBoxes[Graphics[{Disk[{0,0}, 1]}]//OutputForm]" " Out[1]= GraphicsBox[{DiskBox[{0, 0}, 1]}, AlignmentPoint → Center, AspectRatio → Automatic, Axes → False, AxesLabel → None, AxesOrigin → Automatic, AxesStyle → {}, Background → Automatic, BaseStyle → {}, BaselinePosition → Automatic, ContentSelectable → Automatic, CoordinatesToolOptions → Automatic, Epilog → {}, FormatType → TraditionalForm, Frame → False, FrameLabel → None, FrameStyle → {}, FrameTicks → Automatic, FrameTicksStyle → {}, GridLines → None, GridLinesStyle → {}, ImageMargins → 0., ImagePadding → All, ImageSize → Automatic, LabelStyle → {}, LogPlot → False, Method → Automatic, PlotLabel → None, PlotRange → Automatic, PlotRangeClipping → False, PlotRangePadding → Automatic, PlotRegion → Automatic, PreserveImageOptions → Automatic, Prolog → {}, RotateLabel → True, Ticks → Automatic, TicksStyle → {}]

Yep, this is the kind of things I want to fix, after having a clear reference to do that.

If we are testing things that will have be changed later, it would be better to remove these or mark them skipped.

By the end of this month, I expect to have fixed this too, according to WMA standard.

Co-authored-by: R. Bernstein <[email protected]>

mmatera · 2026-01-03T17:09:07Z

The goal of all these tests (which mostly are tests that have been around for a while) is to detect differences between the output in Mathics and in WMA. Changes are most related to make the tests available also from both CLI, and adding a few structures that appear in the evaluation of these tests.

tests for MakeBoxes

4810a6c

mmatera mentioned this pull request Dec 30, 2025

OutputForm #1570

Draft

fix parametrize

a4963b6

rocky reviewed Dec 30, 2025

View reviewed changes

merge

b3984f3

mmatera force-pushed the test_makeboxes branch from 838a77e to b3984f3 Compare December 31, 2025 10:53

Merge branch 'master' into test_makeboxes

8cbe6dd

rocky reviewed Dec 31, 2025

View reviewed changes

more typos

f4a8d60

rocky reviewed Dec 31, 2025

View reviewed changes

mmatera added 3 commits January 2, 2026 18:06

move more tests to the yaml file

b56e59a

skiponfail

4e972c1

adding a message

4da8180

mmatera added 2 commits January 2, 2026 18:39

skip empty tests

7894482

set all the tests according to WMA standards

6499f5d

rocky reviewed Jan 2, 2026

View reviewed changes

test/format/makeboxes_tests.yaml Show resolved Hide resolved

rocky reviewed Jan 2, 2026

View reviewed changes

test/format/makeboxes_tests.yaml Outdated Show resolved Hide resolved

rocky reviewed Jan 2, 2026

View reviewed changes

test/format/makeboxes_tests.yaml Outdated Show resolved Hide resolved

mmatera added 2 commits January 3, 2026 10:56

store a version with the right format in WMA. Improve the case for <<…

82a07b3

…SuperscriptBox with a nested expression.>>

Merge remote-tracking branch 'origin/master' into test_makeboxes

308839c

mmatera force-pushed the test_makeboxes branch from c348028 to 308839c Compare January 3, 2026 13:56

mmatera added 4 commits January 3, 2026 10:57

typos

7eb0096

adding comments. decide which tests to do according the interpreter.

ce95171

use simple quotes in string to number test

bd11c5d

simplify the rule for MakeBoxes[InputForm]. remove unused import

aae6a35

rocky reviewed Jan 3, 2026

View reviewed changes

test/format/makeboxes_tests.m Outdated Show resolved Hide resolved

rocky reviewed Jan 3, 2026

View reviewed changes

test/format/makeboxes_tests.yaml Outdated Show resolved Hide resolved

Update test/format/makeboxes_tests.yaml

2b4bd2e

Co-authored-by: R. Bernstein <[email protected]>

rocky reviewed Jan 3, 2026

View reviewed changes

mmatera and others added 2 commits January 3, 2026 13:58

Update test/format/makeboxes_tests.m

d90abaa

Co-authored-by: R. Bernstein <[email protected]>

Mathics -> Mathics3

be675d8

mmatera merged commit 1aa6cc0 into master Jan 3, 2026
21 checks passed

mmatera deleted the test_makeboxes branch January 3, 2026 17:23

	"StandadForm with Prec real shows all the stored digits, and prec",
	"StandardForm with real precision shows all the stored digits, and precision",

	(r"MakeBoxes[1.4`]", r"1.4`", "StandardForm always shows a prec mark."),
	(r"MakeBoxes[1.4`]", r"1.4`", "StandardForm always shows a precision mark."),

	"OutputForm trims digits up to prec.",
	"OutputForm trims digits up to precision.",

	msg: "In Mathics, precision is always an integer number."
	msg: "In Mathics3, precision is always an integer number."

Uh oh!

Tests for MakeBoxes #1579

Tests for MakeBoxes #1579

Uh oh!

Conversation

mmatera commented Dec 30, 2025

Uh oh!

rocky Dec 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rocky Dec 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rocky Dec 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rocky Dec 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mmatera commented Jan 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rocky commented Jan 2, 2026

Uh oh!

Uh oh!

Uh oh!

rocky commented Jan 3, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rocky Jan 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rocky Jan 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mmatera commented Jan 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rocky Dec 30, 2025 •

edited

Loading

rocky Dec 30, 2025 •

edited

Loading

rocky Dec 31, 2025 •

edited

Loading

rocky Dec 31, 2025 •

edited

Loading

mmatera commented Jan 2, 2026 •

edited

Loading

rocky Jan 3, 2026 •

edited

Loading

rocky Jan 3, 2026 •

edited

Loading