Revise stops tracking a file/module mid-session

[mhauru]

I have a script that imports a local module and does something with it, and my usual work flow is editing the module with a text editor and including the script from the REPL after each edit. Revise usually keeps track of the changes to the module, but I've been intermittently running into cases where suddenly for no apparent reason Revise stops revising. No changes to the file I've been editing, or at least in my current case any file in the same module, are registered any more. Changes to files from other modules still are. I've found no way around this other than restarting my session.

The changes that I'm making should not be anything particularly odd, no macros, generated functions or type changes are involved.

Revise's logs show the last two entries being a DeleteMethod and an Eval for the method that the last succesfull edit was for. I'm not familiar with how the logs usually look like, but nothing seems obviously amiss to me. For what it's worth, all(e.group == "Action" for e in rlogger.logs) returns true.

Revise.revise() does nothing, Revise.revise(ModuleBeingEdited) spits out a long list of Replacing docs log messages, but doesn't help the situation. Revise.watched_files includes the correct folder and the file I'm trying to edit.

I'm running Revise 2.6.0, Julia 1.4, and Ubuntu 19.10. The problem isn't brand new, I was already having it for instance on Julia 1.3.

The above probably isn't enough to solve this, but please give me directions of what to do and check to debug this further.

[timholy]

Hmm, interesting. This may be a failure of Julia's FileWatching, not sure. Maybe it would be easiest to snoop on Revise's monitoring of the files? Presuming that you get this:

julia> Revise.watching_files[]
false

then the right function to spy on is revise_dir_queued, maybe dev Revise and then insert a @show dirname as the first line of the method body? You'll see a line of output every time you change a file; if that stops happening, then it means Revise isn't being notified of changes and the bug is upstream.

Conversely, if you do keep getting notified of changes but Revise doesn't update the package, it implies a fault in Revise. It would be worth looking at Revise.revision_queue and see if it's populated with files that need attention but that for some reason revise() isn't processing them.

[mhauru]

Revise.watching_files[] indeed is false. I already had to kill my session where this was happening yesterday, but I think I checked Revise.revision_queue and it was empty. I'll set up monitoring the calls to revise_dir_queued and let you know what I find the next time this occurs.

[timholy]

I should also mention that #449 might be interesting in your debugging efforts, you could try checking out that branch. However it may also generate some spurious warnings (e.g., if you delete a source file).

[mhauru]

Indeed, a simple print statement tells me that revise_dir_queued just doesn't get called after things fail.

I also finally managed to find a simple test case that reliably reproduces this, at least on my machine:
I've got a file main.jl that says

function main()
    println("hello")
    sleep(5)
end

I includet that, call main(), and during that sleep edit main.jl and Ctrl+C to interrupt the sleep. That final edit gets processed correctly, but any changes after this don't. You can replace sleep with readline and get the same failure, but not with for instance sort(randn(100000000)).

[timholy]

Your test case was extremely useful. This is actually a pretty "deep" bug, one that comes down to how Julia itself handles Tasks: JuliaLang/julia#25790 (comment). It would be better to implement a fix in Julia itself, but as that comment indicates, I believe I may have finally found a workaround.