Post processing issue: cannot find workable URL

[eleanorsleightholm]

Hello,

I'm trying to run coco but I receive the following warnings during the post-processing:

UserWarning: failed to connect to https://numbbo.github.io/data-archive/data-archive warnings.warn("failed to connect to " + url)
UserWarning: failed to connect to https://numbbo.github.io/gforge/data-archive
UserWarning: failed to connect to https://coco.gforge.inria.fr/data-archive
UserWarning: failed fo find workable URL or local folder for official archives warnings.warn("failed fo find workable URL or local folder for official archives")
COCO INFO: Results will be output to folder exdata/random_search_of_cocoex.solvers_2D_on_bbob-005 *** benchmarking random_search from cocoex.solvers on suite bbob ***


When I navigate to the coco_urls, I am unable to access the first link because my browser blocks it and the final two links don't exist. Please could you help with this?

Thanks,
Ellie

[nikohansen]

It works for me. Can you access https://numbbo.github.io/data-archive/ ? Is it possible that you were disconnected from the internet? https://numbbo.github.io/data-archive/data-archive/ gives a 404 Page not found :( error for me which is hence expected.

[eleanorsleightholm]

Yes, I can access https://numbbo.github.io/data-archive/ and no, I was connected to internet at the time. Still not entirely sure what the issue is.

This is the full traceback after I run the 'example_experiment_for_beginners.py' file.

/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/cocopp/archiving.py:1399: UserWarning: failed to connect to https://numbbo.github.io/data-archive/data-archive
warnings.warn("failed to connect to " + url)
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/cocopp/archiving.py:1399: UserWarning: failed to connect to https://numbbo.github.io/gforge/data-archive
warnings.warn("failed to connect to " + url)
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/cocopp/archiving.py:1399: UserWarning: failed to connect to https://coco.gforge.inria.fr/data-archive
warnings.warn("failed to connect to " + url)
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/cocopp/archiving.py:1401: UserWarning: failed fo find workable URL or local folder for official archives
warnings.warn("failed fo find workable URL or local folder for official archives")
COCO INFO: Results will be output to folder exdata/scipy-optimize-fmin-001
2D 11h04:37s

(solving 2D, 3D, etc...)

Post-processing (1)
Traceback (most recent call last):
File "/User/.../example1.py", line 47, in
cocopp.main(observer.result_folder) # re-run folders look like "...-001" etc
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/cocopp/rungeneric.py", line 376, in main
data_archive = archiving.official_archives.all # was: archiving.COCODataArchive()
AttributeError: 'NoneType' object has no attribute 'all'


Thank you for any help you can give me on this issue.

[nikohansen]

This doesn't actually need the data archive at all and hence it shouldn't make this assignment in rungeneric. Otherwise, could it be related to a firewall on the computer?

[nikohansen]

Executing the following code, e.g. in a Jupyter notebook, is a workaround. It writes minimal archive definition files in some folders under ~/.cocopp.

import os
from cocopp import archiving as ac

def has_all(a):
    return hasattr(a, 'all') and a.all is not None
def make_skeleton(official_archives):
    for url, class_ in official_archives._list:
        path = ac._url_to_folder_name(url)
        if path.endswith('/test'):
            continue
        if path.endswith('/all'):
            path = path[:-4]
            url = url[:-4]
        ac._makedirs(path)  # should not be necessary
        df = ac._definition_file_to_read(path)
        if os.path.exists(df):
            print('file "{}" exists'.format(df))
        else:
            with open(df, 'w') as f:
                f.write("[('_url_', '{}')]\n".format(url))

if ac.official_archives is None:
    ac.official_archives = ac.OfficialArchives(ac.coco_url)
if not has_all(ac.official_archives):
    make_skeleton(ac.official_archives)
    print('made archive skeleton for {}'.format(ac.coco_url))
    ac.official_archives.set_as_attributes_in()
    try: ac.official_archives.update_all()  # try overwrite fake definition files
    except: pass

After this, cocopp.main as invoked above (or cocopp from a system shell) with only folder arguments should always work.

[nikohansen]

This bug will be fixed in the next release.

[olafmersmann]

Can we close the issue?

[eleanorsleightholm]

Yes, thank you for all your help.

[nikohansen]

Thanks for the detailed report, it always helps to improve the code basis.