[Bug report] cocopp_commands.tex does not contain pptablesheader

[MLopez-Ibanez]

I'm trying to get my head around COCO and I'm having some issues generating the info for the tables. I run the following command:

python3 -m cocopp -o ppdata/mutations exdata/EA_AOS_on_bbob_budget100xD/DE-best-1 exdata/EA_AOS_on_bbob_budget100xD/DE-best-2 exdata/EA_AOS_on_bbob_budget100xD/DE-current_to_best-1 exdata/EA_AOS_on_bbob_budget100xD/DE-current_to_pbest exdata/EA_AOS_on_bbob_budget100xD/DE-current_to_pbest_archived exdata/EA_AOS_on_bbob_budget100xD/DE-current-to-rand-1 exdata/EA_AOS_on_bbob_budget100xD/DE-rand-1 exdata/EA_AOS_on_bbob_budget100xD/DE-rand-2 exdata/EA_AOS_on_bbob_budget100xD/DE-Random exdata/EA_AOS_on_bbob_budget100xD/DE-rand-to-best-2

But the cocopp_commands.tex file generated seems to be missing information required by the templates, such as pptablesheader (or pptableheader).

• Operating system: Ubuntu 18.04
• COCO version: latest version from master

[brockho]

Many thanks for your bug report. It indeed pointed to a mistake on our side: we had reverted the average runtime (aRT) to the previous expected runtime (ERT), but only in the postprocessing module and not (yet) in the LaTeX templates. That meant that you could not compile the LaTeX templates without errors. This should be fixed now in the development branch. A release should follow soon.

The problem with the missing pptableheader seems to be caused by using the wrong LaTeX template. What you will need is the file templateBBOBmany.tex (for historical reasons, the "many" in the name stands for 3 or more algorithms, the "cmp" for 2 algorithms and the "article" for 1 algorithm, you want to display).

I think, we should consider also moving for the bbob test suite to a single LaTeX template as for the newer suites. As soon as I have time, I will take care of this.

[brockho]

Add-on, just to understand what is happening: the cocopp_commands.tex file is generated with each call of the postprocessing, but depending on the number of algorithms compared and thus what the corresponding LaTeX template needs. The pptableheader is only created when postprocessing a single algorithm (and hence needed only in the templateBBOBarticle.tex).

[MLopez-Ibanez]

Add-on, just to understand what is happening: the cocopp_commands.tex file is generated with each call of the postprocessing, but depending on the number of algorithms compared and thus what the corresponding LaTeX template needs. The pptableheader is only created when postprocessing a single algorithm (and hence needed only in the templateBBOBarticle.tex).

Sorry, I meant actually pptablesheader, but the output generated does not contain either of them anyway.

[MLopez-Ibanez]

Could it be because I only run for dimension 20 and you have this code?

coco/code-postprocessing/cocopp/compall/pptables.py

Line 720 in 55cc524

if len(tables_header) > 0 and df[0] == testbedsettings.current_testbed.tabDimsOfInterest[0]:

[brockho]

You are right, this looks like the reason. When I postprocess any three algorithms (with dimension 2), I get the \pptablesheader. If you can live with it for now, you can add the following by hand into the cocopp_commands.tex file:

\providecommand{\pptablesheader}{
\begin{tabularx}{1.0\textwidth}{@{}c@{}|*{7}{@{}r@{}X@{}}|@{}r@{}@{}l@{}}
$\Delta f_\mathrm{opt}$ & \multicolumn{2}{@{\,}l@{\,}}{1e1} & \multicolumn{2}{@{\,}l@{\,}}{1e0} & \multicolumn{2}{@{\,}l@{\,}}{1e-1} & \multicolumn{2}{@{\,}l@{\,}}{1e-2} & \multicolumn{2}{@{\,}l@{\,}}{1e-3} & \multicolumn{2}{@{\,}l@{\,}}{1e-5} & \multicolumn{2}{@{\,}l@{\,}}{1e-7} & \multicolumn{2}{|@{}l@{}}{\#succ}\\\hline
}

We will discuss in the core developer team whether we should change the actual code accordingly.

Once again, many thanks for filing the issue and even more for finding the likely reason for the "bug"!

[MLopez-Ibanez]

You are right, this looks like the reason. When I postprocess any three algorithms (with dimension 2), I get the \pptablesheader. If you can live with it for now, you can add the following by hand into the cocopp_commands.tex file:

I just changed the code to avoid the check. Is that problematic?

diff --git a/code-postprocessing/cocopp/compall/pptables.py b/code-postprocessing/cocopp/compall/pptables.py
index d190605..fd2fade 100644
--- a/code-postprocessing/cocopp/compall/pptables.py
+++ b/code-postprocessing/cocopp/compall/pptables.py
@@ -717,7 +717,9 @@ def main(dict_alg, sorted_algs, output_dir='.', function_targets_line=True, late
     if len(additional_commands) > 0:
         for command in additional_commands:
             prepend_to_file(latex_commands_file, [command])
-    if len(tables_header) > 0 and df[0] == testbedsettings.current_testbed.tabDimsOfInterest[0]:
+    print(f"DEBUGGIN: {df[0]}")
+    print(f"DEBUGGIN: {testbedsettings.current_testbed.tabDimsOfInterest[0]}")
+    if len(tables_header) > 0: # and df[0] == testbedsettings.current_testbed.tabDimsOfInterest[0]:
         extraeol = [r'\hline']
         res = tableXLaTeX([tables_header], spec=spec, extra_eol=extraeol, add_end_tabular=False)
         prepend_to_file(latex_commands_file, ['\\providecommand{\\pptablesheader}{', res, '}'])                                                                                                                                                                      

[brockho]

I just changed the code to avoid the check. Is that problematic?

That should not be problematic. The \providecommand{...} will take care that only the last definition is used. The reason, why we only write it once for the first dimension, is that it does not appear multiple times in the cocopp_commands.tex file and unnecessarily blows up the file size.

[brockho]

I changed the "problematic" line now to

if len(tables_header) > 0 and df[0] == min(df):

to avoid the problem. Hence, closing this issue.