Recipe Robot python script not working with AutoPkg 2.0

[n8felton]

Describe the bug
After upgrading to AutoPkg 2.0.2, the python script for recipe-robot no longer works.

To Reproduce
Steps to reproduce the behavior:

1. Install AutoPkg 2.0.2
2. git pull https://github.com/homebysix/recipe-robot.git
3. cd recipe-robot/scripts
4. ./recipe-robot
5. python stack trace

Expected behavior
Normal usage of the recipe-robot python script

Desktop (please complete the following information):

• macOS: 10.14.6 (18G3020)
• Recipe Robot Version: 1.2.1
• which python: /usr/bin/python
• python --version: Python 2.7.16

Additional context

Traceback (most recent call last):
  File "./recipe-robot", line 42, in <module>
    import recipe_robot_lib
  File "/Users/n8felton/src/recipe-robot/scripts/recipe_robot_lib/__init__.py", line 26, in <module>
    from .recipe_generator import generate_recipes
  File "/Users/n8felton/src/recipe-robot/scripts/recipe_robot_lib/recipe_generator.py", line 34, in <module>
    from . import processor
  File "/Users/n8felton/src/recipe-robot/scripts/recipe_robot_lib/processor.py", line 38, in <module>
    import autopkglib
  File "/Library/AutoPkg/autopkglib/__init__.py", line 185
    raise PreferenceError(f"Could not synchronize preference {key}")
                                                                  ^
SyntaxError: invalid syntax

[homebysix]

Thanks for the report! It's true, I've been busy updating other projects for Python 3 and neglected Recipe Robot. But I hope to bring it up to date soon.

[homebysix]

Status update: Python 3 compatibility for the script is mostly done. There are two main issues remaining to solve:

• I've transitioned to using curl for web requests, similar to AutoPkg 2. However, I'm not handling GitHub tokens correctly yet, and getting 403 errors as a result.
• The swift notification listener is not working as expected, causing the app to fail to display the output from a Recipe Robot run if the script raised a RoboError. I may need some help with this one.

Work in progress on the https://github.com/homebysix/recipe-robot/tree/2.0.0-dev branch.

[jelockwood]

@homebysix
Firstly I hope you are still well and surviving the current pandemic. However it has been a couple of months since your last update and therefore Recipe Robot still remains completely broken at least in its last official release.

Any news you are able to provide?

[homebysix]

@jelockwood - Feel free to try the branch linked above. The command line portion works well, except for the GitHub 403 errors that I'm still addressing.

I'll open another issue for the RoboError problem, to solicit some help with that. (#160)

[HewittJC]

@homebysix I think I found a fix for the 403 issue.

You can tell curl to dump headers to std out and dump the body of the response to /dev/null which effectively produces the same result.

Found HERE

So here is how I tested the workaround:

def get_headers(url, headers=None):                                                                                                                            
    """Get a URL's HTTP headers, parse them, and return them."""                                                                                               
    curl_cmd = prepare_curl_cmd()                                                                                                                              
    add_curl_headers(curl_cmd, headers)                                                                                                                        
    curl_cmd.extend(["--head", url])                                                                                                                           
    curl_cmd = ["/usr/bin/curl", "-sSL", "-D", "-", url, "-o", "/dev/null"]                                                                                    
    output = download_with_curl(curl_cmd, text=True)                                                                                                           
    parsed_headers = parse_headers(output, url=url)                                                                                                            
    return parsed_headers

I just did a hard override to the curl command there. But it got me past the 403 errors.

It seems that S3 is the issue. S3 does not like the HEAD requests. See HERE for info on that.

[homebysix]

@n8felton - I'm pleased to report that I've got a release candidate for Recipe Robot 2.0.0, finally. Could you try downloading the release candidate, and let me know if that solves your issue? Thank you.

[homebysix]

This should be solved in the 2.0.0 release