PyPlot backend not working in IJulia notebook #258

swt30 · 2016-05-16T17:12:27Z

For some reason (jupyter update?) I can no longer use pyplot in the jupyter notebook:

using Plots
x = linspace(-3,3)
y = sin(x)
plot(x,y)

produces

LoadError: PyError (:PyObject_Call) <class 'OSError'>
OSError('Julia exception: MethodError(flush,(IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=51736, maxsize=Inf, ptr=51737, mark=-1),))',)
  File "/data/vault/swt30/Software/miniconda3/lib/python3.5/site-packages/matplotlib/backends/backend_qt5agg.py", line 196, in print_figure
    FigureCanvasAgg.print_figure(self, *args, **kwargs)
  File "/data/vault/swt30/Software/miniconda3/lib/python3.5/site-packages/matplotlib/backend_bases.py", line 2232, in print_figure
    **kwargs)
  File "/data/vault/swt30/Software/miniconda3/lib/python3.5/site-packages/matplotlib/backends/backend_svg.py", line 1206, in print_svg
    return self._print_svg(filename, svgwriter, fh_to_close, **kwargs)
  File "/data/vault/swt30/Software/miniconda3/lib/python3.5/site-packages/matplotlib/backends/backend_svg.py", line 1235, in _print_svg
    renderer.finalize()
  File "/data/vault/swt30/Software/miniconda3/lib/python3.5/site-packages/matplotlib/backends/backend_svg.py", line 295, in finalize
    self.writer.flush()
  File "PyCall.jl", line 1, in <lambda>

Using PyPlot directly or another backend works fine. julia 0.4.5.

The text was updated successfully, but these errors were encountered:

swt30 · 2016-05-17T11:16:17Z

I tried again, this time allowing PyPlot to install its own version of matplotlib from Conda.jl. Same problem. I'm going to try downgrading my notebook and see if that helps.

swt30 · 2016-05-17T11:51:39Z

Downgrading the notebook had no effect. It looks like it may be an issue in Plots? The first bad commit is ed1cce8, producing the following error:

MethodError: `writemime` has no method matching writemime(::Base.AbstractIOBuffer{Array{UInt8,1}}, ::MIME{symbol("text/html")}, ::Plots.Plot{Plots.PyPlotPackage})
Closest candidates are:
  writemime(::IO, !Matched::AbstractString, ::Any)
  writemime(::IO, ::MIME{symbol("text/html")}, !Matched::Method)
  writemime(::IO, ::MIME{symbol("text/html")}, !Matched::MethodTable)
  ...

 in writemime at multimedia.jl:43
 in sprint at iostream.jl:206
 in display_dict at /home/swt30/.julia/v0.4/Plots/src/Plots.jl:233

Then after fd4dd13 the error becomes the one above.

tbreloff · 2016-05-17T11:53:37Z

My best guess is that it has to do with your qt5 install, but I'm not
great at diagnosing PyCall issues. Did you try another PyPlot backend (not
Plots backend)? Try the Agg backend to write to png?

On Tuesday, May 17, 2016, Scott Thomas [email protected] wrote:

I tried again, this time allowing PyPlot to install its own version of
matplotlib from Conda.jl. Same problem. I'm going to try downgrading some
packages from Conda and see if that helps.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#258 (comment)

tbreloff · 2016-05-17T11:59:12Z

Also please check out the dev branch of Plots. You're referencing a commit
from 2 months ago. Plots is very fast moving now. (Even dev is super out of
date ;)

On Tuesday, May 17, 2016, Tom Breloff [email protected] wrote:

My best guess is that it has to do with your qt5 install, but I'm not
great at diagnosing PyCall issues. Did you try another PyPlot backend (not
Plots backend)? Try the Agg backend to write to png?

On Tuesday, May 17, 2016, Scott Thomas <[email protected]
javascript:_e(%7B%7D,'cvml','[email protected]');> wrote:

I tried again, this time allowing PyPlot to install its own version of
matplotlib from Conda.jl. Same problem. I'm going to try downgrading some
packages from Conda and see if that helps.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#258 (comment)

swt30 · 2016-05-17T12:20:55Z

Yeah, it is from a while ago, isn't it? That's why I was completely baffled when Plots just stopped working yesterday. I think I updated Plots and various python packages, but if this affected anyone else then I would have expected it to be raised already.

Anyway, same problem on the latest dev too! This works fine:

import PyCall
PyCall.pygui(:tk) # the pyplot backend is now TkAgg
import PyPlot
x = linspace(-3,3)
y = sin(x)
PyPlot.plot(x,y)

and then I can call PyPlot.savefig on it. But Plots doesn't like it: Plots.plot(x,y) gives

PyError (:PyObject_Call) <class 'OSError'>
OSError('Julia exception: MethodError(flush,(IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=19075, maxsize=Inf, ptr=19076, mark=-1),))',)
  File "/data/vault/swt30/Software/miniconda3/lib/python3.5/site-packages/matplotlib/backend_bases.py", line 2232, in print_figure
    **kwargs)
  File "/data/vault/swt30/Software/miniconda3/lib/python3.5/site-packages/matplotlib/backends/backend_svg.py", line 1206, in print_svg
    return self._print_svg(filename, svgwriter, fh_to_close, **kwargs)
  File "/data/vault/swt30/Software/miniconda3/lib/python3.5/site-packages/matplotlib/backends/backend_svg.py", line 1235, in _print_svg
    renderer.finalize()
  File "/data/vault/swt30/Software/miniconda3/lib/python3.5/site-packages/matplotlib/backends/backend_svg.py", line 295, in finalize
    self.writer.flush()
  File "PyCall.jl", line 1, in <lambda>

So it seems like this call to flush is hitting the same problem across different PyPlot backends.

tbreloff · 2016-05-17T12:39:37Z

A couple things:

I don't know how well python 3.5 is supported with PyPlot
Have you tried a local conda install? Do: ENV["PYTHON"] = ""; Pkg.build("PyPlot")
Doing the PyCall.pygui(:tk) may not work exactly as you expect... you should instead try setting the env var before the using: ENV["MPLBACKEND"] = "Agg"; import PyPlot

If after all that, PyPlot works but Plots doesn't, then maybe I can be convinced that it's a Plots issue.

@stevengj Any other ideas? Am I missing something?

cstjean · 2016-05-17T12:47:17Z

I'm getting the same issue (Julia 0.4.5, OSX, IJulia), though I can't tell if it was working before. There was a new PyCall.jl tagged yesterday, it's likely related.

Someone in julia-users claims that it's related to IJulia

FWIW, plotlyjs is also failing silently for me today (and I don't remember Pkg.updating yesterday), but that's likely something else. gadfly() works.

swt30 · 2016-05-17T12:50:42Z

Yep, I've done ENV["PYTHON"] = "" (that's what I meant by 'allowing PyPlot to install its own version of matplotlib', sorry if unclear) to no effect. This also installs python 2.7, so it's not a python version issue. And finally, same issue using the agg backend. Thanks @cstjean, good to know it's not just me.

swt30 · 2016-05-17T13:16:57Z

OK, the first failing commit for me in PyCall is JuliaPy/PyCall.jl@453100f, which touched the flush function, so that looks promising.

tbreloff · 2016-05-17T13:18:27Z

Ok yes it possible that something changed in PyPlot/PyCall recently. @stevengj this is how Plots displays to IJulia... maybe you know:

const _pyplot_mimeformats = Dict(
    "application/eps"         => "eps",
    "image/eps"               => "eps",
    "application/pdf"         => "pdf",
    "image/png"               => "png",
    "application/postscript"  => "ps",
    "image/svg+xml"           => "svg"
)


for (mime, fmt) in _pyplot_mimeformats
    @eval function Base.writemime(io::IO, ::MIME{symbol($mime)}, plt::AbstractPlot{PyPlotBackend})
        finalizePlot(plt)
        fig = getfig(plt.o)
        fig.o["canvas"][:print_figure](
            io,
            format=$fmt,
            # bbox_inches = "tight",
            # figsize = map(px2inch, plt.plotargs[:size]),
            facecolor = fig.o["get_facecolor"](),
            edgecolor = "none",
            dpi = DPI
        )
    end
end

tbreloff · 2016-05-17T13:24:10Z

I just updated and I can reproduce your error! My update command included:

NFO: Upgrading Conda: v0.1.9 => v0.2.0
INFO: Upgrading PyCall: v1.4.0 => v1.6.0

and

Package plan for installation in environment /home/tom/.julia/v0.4/Conda/deps/usr:

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    mkl-11.3.3                 |                0       122.1 MB
    numpy-1.11.0               |           py27_1         6.2 MB
    conda-4.0.6                |           py27_0         185 KB
    ------------------------------------------------------------
                                           Total:       128.5 MB

The following packages will be UPDATED:

    conda: 4.0.5-py27_0  --> 4.0.6-py27_0 
    mkl:   11.3.1-0      --> 11.3.3-0     
    numpy: 1.11.0-py27_0 --> 1.11.0-py27_1

Fetching packages ...
mkl-11.3.3-0.t 100% |###############################| Time: 0:00:26   4.87 MB/s
numpy-1.11.0-p 100% |###############################| Time: 0:00:02   2.84 MB/s
conda-4.0.6-py 100% |###############################| Time: 0:00:00 778.79 kB/s
Extracting packages ...
[      COMPLETE      ]|##################################################| 100%
Unlinking packages ...
[      COMPLETE      ]|##################################################| 100%
Linking packages ...
[      COMPLETE      ]|##################################################| 100%
INFO: PyCall is using /home/tom/.julia/v0.4/Conda/deps/usr/bin/python (Python 2.7.11) at /home/tom/.julia/v0.4/Conda/deps/usr/bin/python, libpython = /home/tom/.julia/v0.4/Conda/deps/usr/lib/libpython2.7.so

Hopefully someone with better PyCall knowledge can help!

swt30 · 2016-05-17T13:26:02Z

I'll open an issue at PyCall - it looks like the PyCall v.1.5.0 tag broke something. Thanks for your help diagnosing this!

tbreloff · 2016-05-17T13:34:11Z

Looking at a diff between PyCall 1.4 --> 1.5, I see this (might be what you were referring to):

-for f in (jl_IO_close, jl_IO_fileno, jl_IO_flush, jl_IO_isatty, jl_IO_readable, jl_IO_writable, jl_IO_readline, jl_IO_readlines, jl_IO_seek, jl_IO_seekable, jl_IO_tell, jl_IO_writelines, jl_IO_read_text, jl_IO_read, jl_IO_readall_text, jl_IO_readall, jl_IO_readinto, jl_IO_write)
-    precompile(f, (PyPtr,PyPtr))
-end
+# TODO: precompilation of the io.jl functions

Could be the culprit?

cstjean · 2016-05-17T13:35:52Z

OK, the first failing commit for me in PyCall is JuliaPy/PyCall.jl@453100f, which touched the flush function, so that looks promising.

Hey, that's my commit! Oops.

It's likely that the issue is in how the flush function was translated. Investigating.

tbreloff · 2016-05-17T13:37:41Z

Hey, that's my commit! Oops

😝

At least we have the right person involved!

tbreloff · 2016-05-17T13:39:55Z

And the short-term fix seems to be:

Pkg.pin("Conda", v"0.1.9")
Pkg.pin("PyCall", v"1.4.0")

If someone resolves this, please post here!

swt30 · 2016-05-17T13:40:10Z

looks like the flush function dropped this check:

if method_exists(flush, Tuple{typeof(io)})      
    flush(io)       
end

cstjean · 2016-05-17T13:41:45Z

Yeah, it works with it. I'm preparing a PR.

cstjean · 2016-05-17T13:48:00Z

JuliaPy/PyCall.jl#272

tbreloff · 2016-05-17T13:48:58Z

Thanks for the speedy response @cstjean !!

stevengj · 2016-05-17T14:01:34Z

It's kind of odd that flush(::IOBuffer) doesn't exist, even as a no-op. Probably this should be added to Julia Base too, if only to make it easier to write generic code.

swt30 · 2016-05-17T19:47:37Z

Fixed by JuliaPy/PyCall.jl@31a80a2 in PyCall master or v1.6.1 when that gets tagged.

change to generic dispatch

swt30 changed the title ~~MethodError(flush...) from pyplot in notebook~~ PyPlot backend not working in IJulia notebook May 17, 2016

swt30 mentioned this issue May 17, 2016

v1.5.0 tag broke Plots.jl inside IJulia JuliaPy/PyCall.jl#271

Closed

stevengj mentioned this issue May 17, 2016

make a default flush(io)=nothing method for all IO streams JuliaLang/julia#16403

Merged

swt30 closed this as completed May 17, 2016

Jonas-a-Zimmermann pushed a commit to Jonas-a-Zimmermann/Plots.jl that referenced this issue Oct 29, 2024

Merge pull request JuliaPlots#258 from JuliaPlots/as/corrplot2

f4c0c23

change to generic dispatch

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PyPlot backend not working in IJulia notebook #258

PyPlot backend not working in IJulia notebook #258

swt30 commented May 16, 2016 •

edited

Loading

swt30 commented May 17, 2016 •

edited

Loading

swt30 commented May 17, 2016

tbreloff commented May 17, 2016

tbreloff commented May 17, 2016

swt30 commented May 17, 2016 •

edited

Loading

tbreloff commented May 17, 2016

cstjean commented May 17, 2016 •

edited

Loading

swt30 commented May 17, 2016 •

edited

Loading

swt30 commented May 17, 2016 •

edited

Loading

tbreloff commented May 17, 2016

tbreloff commented May 17, 2016

swt30 commented May 17, 2016

tbreloff commented May 17, 2016

cstjean commented May 17, 2016

tbreloff commented May 17, 2016

tbreloff commented May 17, 2016

swt30 commented May 17, 2016

cstjean commented May 17, 2016

cstjean commented May 17, 2016

tbreloff commented May 17, 2016

stevengj commented May 17, 2016

swt30 commented May 17, 2016

PyPlot backend not working in IJulia notebook #258

PyPlot backend not working in IJulia notebook #258

Comments

swt30 commented May 16, 2016 • edited Loading

swt30 commented May 17, 2016 • edited Loading

swt30 commented May 17, 2016

tbreloff commented May 17, 2016

tbreloff commented May 17, 2016

swt30 commented May 17, 2016 • edited Loading

tbreloff commented May 17, 2016

cstjean commented May 17, 2016 • edited Loading

swt30 commented May 17, 2016 • edited Loading

swt30 commented May 17, 2016 • edited Loading

tbreloff commented May 17, 2016

tbreloff commented May 17, 2016

swt30 commented May 17, 2016

tbreloff commented May 17, 2016

cstjean commented May 17, 2016

tbreloff commented May 17, 2016

tbreloff commented May 17, 2016

swt30 commented May 17, 2016

cstjean commented May 17, 2016

cstjean commented May 17, 2016

tbreloff commented May 17, 2016

stevengj commented May 17, 2016

swt30 commented May 17, 2016

swt30 commented May 16, 2016 •

edited

Loading

swt30 commented May 17, 2016 •

edited

Loading

swt30 commented May 17, 2016 •

edited

Loading

cstjean commented May 17, 2016 •

edited

Loading

swt30 commented May 17, 2016 •

edited

Loading

swt30 commented May 17, 2016 •

edited

Loading