Non-canonical warnings ignored in test cases

[tueda]

Because the test suite expects warnings to take the form Line 1 --> Warning: ..., warnings caused by setup parameter changes (#529) are ignored.

Output log:

$ ./check/check.rb ./build/sources/vorm Issue468 Issue563 -v
warning: failed to get the wordsize of './build/sources/vorm'
warning: assuming wordsize = 4
Check /home/tueda/work/form/build/sources/vorm
FORM 5.0.0-beta.1 (Nov 14 2024, v5.0.0-beta.1-81-g0289435)  Run: Mon Nov 18 20:19:14 2024
Loaded suite ./check/check
StartedCommand: /home/tueda/work/form/build/sources/vorm 1.frm


===============================================================================
Issue468 (fixes.frm:2721) SUCCEEDED
===============================================================================
FORM 5.0.0-beta.1 (Nov 14 2024, v5.0.0-beta.1-81-g0289435)  Run: Mon Nov 18 20:19:14 2024
    #-
Warning: LargeSize adjusted: 26880512 -> 134400480
Warning: SmallSize adjusted: 100000 -> 12800016
Warning: SmallEsize adjusted: 200000 -> 19200032
Warning: TermsInSmall adjusted: 5000 -> 5008

Time =       0.02 sec    Generated terms =          2
            test         Terms in output =          2
                         Bytes used      =    1298208
~~~Enable prf

Time =       0.08 sec    Generated terms =          2
            test         Terms in output =          1
                         Bytes used      =     324576

Time =       0.09 sec    Generated terms =          1
            test         Terms in output =          1
                         Bytes used      =         48

   test =
      prf(2,1);

  0.09 sec out of 0.10 sec
===============================================================================

.Command: /home/tueda/work/form/build/sources/vorm 1.frm

===============================================================================
Issue563 (fixes.frm:2861) SUCCEEDED
===============================================================================
FORM 5.0.0-beta.1 (Nov 14 2024, v5.0.0-beta.1-81-g0289435)  Run: Mon Nov 18 20:19:14 2024
    #: SubTermsInSmall 50
    
    CFunction f,g;
    Symbol a;
    
    * Generate a function arg with more than SubTermsInSmall terms:
    * Make sure one of the factors has more than SubTermsInSmall itself.
    * Use functions to trigger LocalConvertToPoly and the EndSort there.
    Local F = f((g(1)+g(2)) * (<g(1)>+...+<g(65)>)) - g(1+2, 1+...+65);
    
    FactArg f;
    Argument f;
    	Identify g(a?) = a;
Warning: TermsInSmall adjusted: 50 -> 64
    EndArgument;
    Identify g(?a) = f(?a);
    
    Print F;
    .end

Time =       0.06 sec    Generated terms =          2
               F         Terms in output =          0
                         Bytes used      =          4

   F = 0;

  0.06 sec out of 0.06 sec
===============================================================================

.
Finished in 0.185177066 seconds.
-------------------------------------------------------------------------------
2 tests, 4 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed
-------------------------------------------------------------------------------
10.80 tests/s, 21.60 assertions/s

Maybe the check:

https://github.com/vermaseren/form/blob/0289435fc48b3ae379c4c1fc8058de7f844dd08a/check/check.rb#L632-L639

could be somewhat relaxed, or FORM warning messages could be formatted more consistently (see also #514).

[tueda]

How can I fix these warnings by adjusting the setup parameters??

#:MaxTermSize 200K
#:SortIOSize 200K
#:SubSortIOSize 200K
#:SubSmallSize 100K
#:SubSmallExtension 200K
#:SubTermsInSmall 5K

#:LargeSize 134400480
#:SmallSize 12800016
#:SmallExtension 19200032
#:TermsInSmall 5008

S x;
CF f;
L F = f(1+x,f(1+x,f(1+x)));
.end

FORM 5.0.0-beta.1 (Nov 21 2024, v5.0.0-beta.1-86-g328e810)  Run: Fri Nov 22 14:13:58 2024
    #:MaxTermSize 200K
    #:SortIOSize 200K
    #:SubSortIOSize 200K
    #:SubSmallSize 100K
    #:SubSmallExtension 200K
    #:SubTermsInSmall 5K

    #:LargeSize 134400480
    #:SmallSize 12800016
    #:SmallExtension 19200032
    #:TermsInSmall 5008

    S x;
    CF f;
    L F = f(1+x,f(1+x,f(1+x)));
    .end
Warning: LargeSize adjusted: 26880512 -> 134400480
Warning: SmallSize adjusted: 100000 -> 12800016
Warning: SmallEsize adjusted: 200000 -> 19200032
Warning: TermsInSmall adjusted: 5000 -> 5008
Warning: LargeSize adjusted: 26880512 -> 134400480
Warning: SmallSize adjusted: 100000 -> 12800016
Warning: SmallEsize adjusted: 200000 -> 19200032
Warning: TermsInSmall adjusted: 5000 -> 5008
Warning: LargeSize adjusted: 26880512 -> 134400480
Warning: SmallSize adjusted: 100000 -> 12800016
Warning: SmallEsize adjusted: 200000 -> 19200032
Warning: TermsInSmall adjusted: 5000 -> 5008

Time =       0.00 sec    Generated terms =          1
               F         Terms in output =          1
                         Bytes used      =        272
  0.00 sec out of 0.00 sec

(Test commit to detect all warnings: tueda@33a76e1)

[jodavies]

These are from the sub buffers. As far as I remember I could not see a good way to detect in AllocSort if it is a sub allocation or not, though looking again at the function, near the end it tests AM.S0 != 0 to see if it should allocate a file or not. I will look again whether this suffices, but I seem to remember trying it...

[jodavies]

Regarding the formatting, one can of course put an & at the start of the messages. I think I did not do this because the line information is not very useful in this particular case -- the buffer sizes have nothing to do with the line where the first module terminates.

[tueda]

For now, I'm trying to detect all warnings by simply searching Warning:, so putting & is not needed.

Maybe we should introduce a way to ignore known warnings, like

assert succeeded?(exclude_warnings=["LargeSize adjusted", "SmallSize adjusted", "SmallEsize adjusted", "TermsInSmall adjusted"])

which checks

assert !warning?(exclude=["LargeSize adjusted", "SmallSize adjusted", "SmallEsize adjusted", "TermsInSmall adjusted"])

[jodavies]

Checking AM.S0 does not work in the case of tform workers allocating their primary buffers. And I think using GETIDENTITY and then AT.S0 doesn't work (yet) when the master thread does its first allocations.

One option would be to add an argument to AllocSort and then set in the caller whether it is a primary or sub-sorting buffer, to get better warning messages.

[jodavies]

How does jodavies@19dc25f seem? It is a trivial change, there are very few places where AllocSort is called.

[tueda]

Looks fine. Now I understand how they come "from the sub buffers". Thanks!