Auto update keeps running in background.

[dozer47528]

This is my project: https://github.com/dozer47528/alfred2-hasher

The auto update thread keeps running and the CPU usage is very high.

[deanishe]

What version of OSX & Alfred do you have?

This appears to be related to similar issues with other workflows, but I haven't been able to replicate the problem here.

[dozer47528]

macOS Sierra 10.12.4
Alfred 3.3.1

[idpaterson]

Same versions here but I'm not having any luck reproducing that with the hasher workflow even messing with the update-related cpickle files. Is this an occasional problem or does it get stuck every time you execute the workflow?

I don't know if my reported issue Python hits 100% CPU is related or if the user was just seeing my workflow synchronizing their data. At least in my experience the workflow's sync is not CPU-bound since the API that I have to use is so slow... I was not able to get any further info from the user and have not seen it happen myself but will be watching this issue just in case.

[StartEnd]

I have the same problem
macOS Sierra 10.12.4
Alfred 3.3.1

[deanishe]

Definitely an issue with Sierra and the background code.

I'm not sure how much I can do about it, as I can't install Sierra and see the problem for myself.

[idpaterson]

@StartEnd thanks for posting here. I was just able to get the 100% CPU issue here after a few tries with @dozer47528's workflow. @deanishe I'll take a look at this and @StartEnd if I find anything I'll give you a preview build of the Wunderlist workflow to test.

[idpaterson]

Very interesting. To help isolate the problem here after a lot of debugging I changed the background script from

cmd = ['/usr/bin/python', update_script, 'check', github_slug,
                   version]

to

cmd = ['/usr/bin/perl', '-e', 'print 0']

The completely innocuous perl process hangs in exactly the same way. Does this give anyone any ideas?

[idpaterson]

This problem is resolved if only a single fork is performed for backgrounding, but that could introduce other issues. @deanishe is that worth considering as a workaround or does it at least better point to the root issue?

[idpaterson]

Quite possibly a better workaround, the problem is also resolved if the intermediary parent process is not killed immediately. Adding a delay of 250ms (chosen arbitrarily) allows the subprocess call to run without hanging. I am not sure whether this has the possibility of introducing similar problems as forking only once.

    # Do second fork.
    try:
        pid = os.fork()
        if pid > 0:
            time.sleep(0.25)
            sys.exit(0)  # Exit second parent.

[janclarin]

I have the same problem, it's drained my battery on a few occasions now. I'm also on macOS 10.12.4 and Alfred 3.3.1. This occurred today with the alfred-wunderlist-workflow and also in my workflow.

Thanks for looking into it, I thought it was just me.

[idpaterson]

I'm going to move forward with a pull request based on the delayed parent process exit in case that ends up being the best solution and will publish a corresponding beta version of the Wunderlist workflow over in idpaterson/alfred-wunderlist-workflow/issues/147

[deanishe]

Thanks, @idpaterson. Adding a delay isn't ideal, but it shouldn't be a big issue in the second fork.

[idpaterson]

Unfortunately after more testing, the delay (even a longer delay) is not stopping the hung process which unless anyone has better ideas puts me back to forking only once which does reliably avoid the hung process.

[deanishe]

May be a bug in my backgrounding code.

Perhaps you could try replacing my daemon code with python-daemon? Flush standard streams before disconnecting from them? Pass close_fds to subprocess.call()?

[idpaterson]

Thanks! No luck so far, but I have a bit more to try still just ran out of time for today. I had not noticed before but this is repeatable just running the background.py script in a terminal which eliminates any strange behavior from Alfred.

To test, I am running the following from within the cache directory using the hasher workflow (any with alfred-workflow will work). Here is the argcache file that I am using based on the simple perl print 0 script __workflow_update_check.argcache.txt. About 70% of the time for me after running, perl is hung at 100% cpu.

cp __workflow_update_check.argcache.txt __workflow_update_check.argcache
alfred_version=3.3.1 /usr/bin/python /Volumes/Macintosh\ HD\ 2/OneDrive/Configs/Alfred/Alfred.alfredpreferences/workflows/user.workflow.61DF0517-BCF1-4BE4-A240-06B7BF49F8A9/workflow/background.py  __workflow_update_check

I am also trying to create a minimal case that will reproduce this behavior but have not had any luck. I have not been able to get it to happen directly under python-daemon but can't identify any significant difference. Slicing and dicing the background script to bare minimum it's still hanging; I'm missing something.

[deanishe]

Thanks very much for trying. I'll try and get some time with a Sierra machine this weekend and have a look myself.

[deanishe]

Should be fixed in v1.25

Essentially, I replaced sys.exit(0) with os._exit(0) in the forking code. The old code was hanging left, right and centre, but I couldn't get the new code to hang on a Sierra machine.

Fingers crossed…

[vitorgalvao]

Does alfred-workflow update itself as well, or is that something Workflow developers will have to do?

I’m guessing the latter, but wanted to confirm.

[deanishe]

Nope. It can only fetch .alfredworkflow/.alfred3workflow files from GitHub releases and ask Alfred to install them.

Annoyingly, this bug affects the update mechanism, so a lot of folks might have to update by hand 😞

[deanishe]

FWIW, Alfred-Workflow is on the Cheeseshop, so upgrading is as simple as pip install --target . --upgrade Alfred-Workflow.

[idpaterson]

Since the process does not freeze 100% of the time (at least for me) I did get an update prompt in my workflow when the new version was published and was able to install it after a few tries, but YMMV.

[deanishe]

Yeah, you should be able to get the workflow to update quickly enough if you kill the rogue process a few times, but that takes some technical chops versus downloading the new version manually.

The Packal updater doesn't work on Alfred 3, either, and the next best help, my Packal search workflow is buggy, too 😒

Which reminds me … I have to upload my updated workflows to Packal, too.

I really hope the bug is fixed. This has caused so many people so much work.

[janclarin]

Thank you for all of your hard work!

[vitorgalvao]

For anyone looking to update alfred-workflow on all their Workflows, this should suffice (don’t forget to change the first line):

cd "{{WHATEVER YOUR WORKFLOWS DIRECTORY IS}}"

IFS=$'\n'

for i in $(find . -name 'workflow.py'); do
  grep --quiet 'Dean Jackson <[email protected]>' "${i}" || continue
  pip install --target "$(dirname "$(dirname "${i}")")" --upgrade Alfred-Workflow
done

Explanation:

• Change the internal field separator to just newlines, just in case.
• Find all workflow.py files.
• Make sure the file contains Dean Jackson <[email protected]> (so we know it’s not just some other workflow.py).
  • If it’s not the right one, restart the loop with the next one.
  • If it is the right one, update.

Now we just need to get --target fixed in pip. How I loathe pip; it’s reliably the most unreliable CLI tool on my machines. Seriously one of the main reasons that always pushes me away from python. Solved it temporarily with the help of this comment.

[deanishe]

WRT pip --target, I always use the following command, as the Homebrew Python/pip won't accept --target due to the above bug:

/usr/bin/python -m pip install --target . Alfred-Workflow

That uses the system Python, which works better with pip.

I have that (minus the package name) saved as an alias. Of course, it requires pip to be installed where the system Python can see it.

[deanishe]

This workflow should be able to fix any borked workflows a user has installed.

[deanishe]

Also, I managed to track down where I originally got the forking code from. A Python Cookbook comment by (I think) @noahspurrier from almost 15 years ago.

And I've been using it for almost that long, too. Always worked flawlessly on Linux and OSX until bloody Sierra.

[vitorgalvao]

until bloody Sierra.

Until 10.12.4, more precisely. That update also broke shotcut (crashed on launch). And Ars Technica saying there wasn’t much to this update. We beg to differ!

The amount of disruption this caused is a testament to how good and popular Alfred-Workflow is, though. Thank you for working on it, @deanishe.

[deanishe]

The amount of disruption this caused is a testament to how good and popular Alfred-Workflow is

Thanks very much. Although it arguably says different things about the popularity and the goodness…

This is what you get for writing a library instead of a program, I guess. When something is broken, it doesn't only affect your own stuff.

Thanks for your fixer-upper script, btw, @vitorgalvao. I copied your method for the workflow version, just removing the dependency on pip.

[garyhesse]

Ran Fixum to update the Wunderlist workflow (0.6.3); debug now shows this error when I attempt to call the workflow:
[2017-08-17 10:43:53][ERROR: input.scriptfilter] Code 1: Traceback (most recent call last): File "alfred-wunderlist-workflow.py", line 21, in <module> sys.exit(wf.run(main, plaintext_exceptions='--commit' in wf.args)) TypeError: run() got an unexpected keyword argument 'plaintext_exceptions'

[idpaterson]

@garyhesse please report that on the Wunderlist workflow over here in case anyone else has that issue. I was using a fork of alfred-workflow that had an earlier version of a feature for showing exceptions, this should not be anything that affects other workflows via alfred-workflow.