rpy2 import error due to libreadline undefined symbol

[horta]

> conda --version

conda 3.5.5

> echo "import rpy2.robjects" > foo.py
> python foo.py

Traceback (most recent call last):
  File "test.py", line 1, in <module>
    import rpy2.robjects
  File "/homes/horta/research/software/opt/anaconda/lib/python2.7/site-packages/rpy2/robjects/__init__.py", line 15, in <module>
    import rpy2.rinterface as rinterface
  File "/homes/horta/research/software/opt/anaconda/lib/python2.7/site-packages/rpy2/rinterface/__init__.py", line 100, in <module>
    from rpy2.rinterface._rinterface import *
ImportError: /nfs/research2/stegle/software/opt/anaconda/bin/../lib/libreadline.so.6: undefined symbol: PC

The same does not happen if ipython is used, or if I perform the same under an interactive python session. rpy2 was installed using pip.

By the way:

> which python

/homes/horta/research/software/opt/anaconda/bin/python

> which ipython

/homes/horta/research/software/opt/anaconda/bin/ipython

And the following linking between folders is true: /homes/horta/research -> /nfs/research/stegle/ -> /nfs/research2/stegle/

[asmeurer]

@tpn

[wgmueller1]

I have the same issue.

[mattexx]

@horta
I have seen this issue in ubuntu lucid, presumably because lucid uses libreadline 6.1.x. My workaround:

import readline
import rpy2.robjects

Caveat is that I am not using conda's rpy2 recipe, rather the climate recipe, which uses the local R installation.

[twmr]

I have the same issue. I observe it when compiling other software in a conda env

Linking CXX executable mshrable
~/miniconda/envs/py27/lib/libreadline.so.6: undefined reference to `BC'
~/miniconda/envs/py27/lib/libreadline.so.6: undefined reference to `tgetnum'
~/miniconda/envs/py27/lib/libreadline.so.6: undefined reference to `PC'
~/miniconda/envs/py27/lib/libreadline.so.6: undefined reference to `tputs'
~/miniconda/envs/py27/lib/libreadline.so.6: undefined reference to `tgetent'
~/miniconda/envs/py27/lib/libreadline.so.6: undefined reference to `tgetflag'
~/miniconda/envs/py27/lib/libreadline.so.6: undefined reference to `tgoto'
~/miniconda/envs/py27/lib/libreadline.so.6: undefined reference to `UP'
~/miniconda/envs/py27/lib/libreadline.so.6: undefined reference to `tgetstr

[asmeurer]

Is that coming from a lack of ncurses?

[twmr]

@asmeurer you are right - adding -lncurses to the linker fixes the linker error. However, without the readline conda package I do not need to add -lncurses to the linker flags.

The reason is that the system readline library links to libtinfo, whereas the conda readline does not.

(py27) » ldd /usr/lib64/libreadline.so.6.3 
    linux-vdso.so.1 =>  (0x00007fff137cd000)
    libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007f81226e3000)
    libc.so.6 => /lib64/libc.so.6 (0x00007f8122325000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f8122b8e000)
(py27) » ldd /home/thomas/miniconda/envs/py27/lib/libreadline.so.6
    linux-vdso.so.1 =>  (0x00007fff159d3000)
    libc.so.6 => /lib64/libc.so.6 (0x00007f2e4c5e9000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f2e4cc1e000)

[twmr]

This is the patch in the fedora readline repo, which links readline to libltinfo

http://pkgs.fedoraproject.org/cgit/readline.git/tree/readline-6.2-shlib.patch

[twmr]

https://github.com/thisch/conda-recipes/tree/readlineltinfo

[twmr]

@tpn what do you think about the shlib patch?

[xgdgsc]

I also get this error today when trying to run rpy2 tests. Can it be fixed?

[ccordoba12]

I think this is going to be fixed when we fix issue #42. I'll test it tomorrow.

[ccordoba12]

Or maybe not =p. The cause of this problem seems to be conda's readline is not linked against ncurses.

[asmeurer]

Try to manually install my readline (conda install -c asmeurer readline). That is linked against ncurses.

[tacaswell]

We are also having issues with this at BNL.

attn @danielballan

[asmeurer]

@tacaswell , does my readline fix it?

[tacaswell]

We independently found a version that works on some machines but not others (https://github.com/NSLS-II/conda-prescriptions/tree/master/releases/readline/6.2 , a 6.2.5 recipe is in one of my branches and built versions are on my channels).

It seems to work in an ubuntu box, but am having issues finding libtinfo during the link step on an arch linux box. I am sort of baffled as libtinfo and libreadline are in the same directory, ldd libreadline finds the correct libtinfo, but during the linking stage ld does not find find libtinfo, but does find libreadline. It is also part of a rather complicated build (epics-base) which does not use auto* tool chain and with a pre-release build of gcc so lots of confounding issues.

If I have have time tonight I will try your readline on my arch system and @danielballan is on the osx issue (which is slightly different as it does not find _BC).

Which is a very long way to say I don't know, but I will get back to you.

[danielballan]

@asmeurer Thanks for the suggestion, but unfortunately the readline on your channel breaks worse. Instead of not finding what it needs in readline, it can't even locate the readline lib.

/Users/dallan/miniconda/envs/_build/epics/bin/darwin-x86/antelope -l -d ../parse.y
dyld: Library not loaded: @rpath/./libreadline.6.2.dylib
  Referenced from: /Users/dallan/miniconda/envs/_build/epics/lib/darwin-x86/libCom.3.14.12.dylib
  Reason: image not found

[asmeurer]

Oh that would mean that you'll have to rebuild the conda package that has libCom with the newest version of conda-build (an unfortunate side-effect of the switch to using @rpath in the install names on OS X).

[tacaswell]

libCom is one of the things being built in this recipe.

[asmeurer]

Are you using the latest version of conda-build? Where is the recipe?

[tacaswell]

Either recipe in this folder: https://github.com/tacaswell/conda-prescriptions/tree/working/releases/epics-base

And I believe latest version of conda-build everywhere we have tried this.

[tacaswell]

And no luck on my mis-behaving linux system with your readline/ncurses

/usr/bin/g++ -o antelope  -L/home/tcaswell/.virtualenvs/_build/epics/lib/linux-x86_64 -Wl,-rpath,/home/tcaswell/.virtualenvs/_build/epics/lib/linux-x86_64       -m64                closure.o error.o lalr.o lr0.o main.o mkpar.o output.o reader.o skeleton.o symtab.o verbose.o warshall.o   -lCom   
/home/tcaswell/.virtualenvs/_build/lib/libreadline.so.6: undefined reference to `PC'
/home/tcaswell/.virtualenvs/_build/lib/libreadline.so.6: undefined reference to `tgetflag'
/home/tcaswell/.virtualenvs/_build/lib/libreadline.so.6: undefined reference to `tgetent'
/home/tcaswell/.virtualenvs/_build/lib/libreadline.so.6: undefined reference to `UP'
/home/tcaswell/.virtualenvs/_build/lib/libreadline.so.6: undefined reference to `tputs'
/home/tcaswell/.virtualenvs/_build/lib/libreadline.so.6: undefined reference to `tgoto'
/home/tcaswell/.virtualenvs/_build/lib/libreadline.so.6: undefined reference to `tgetnum'
/home/tcaswell/.virtualenvs/_build/lib/libreadline.so.6: undefined reference to `BC'
/home/tcaswell/.virtualenvs/_build/lib/libreadline.so.6: undefined reference to `tgetstr'
collect2: error: ld returned 1 exit status

[jeffalstott]

+1 person this happening to. I used @mattexx's workaround:

import readline
import rpy2.robjects

[cbonilla20]

I used

import readline
import rpy2.robjects

and works for me, but is a little bit dirty solution.

[jseabold]

I just ran into this or similar issue with R 3.2.2 installed via apt-get from http://cran.rstudio.com/bin/linux/ubuntu trusty/ while doing a fresh install with miniconda.

$ R
/usr/lib/R/bin/exec/R: /home/ubuntu/miniconda3/lib/liblzma.so.5: no version information available (required by /usr/lib/R/lib/libR.so)
/usr/lib/R/bin/exec/R: symbol lookup error: /home/ubuntu/miniconda3/lib/libreadline.so.6: undefined symbol: PC
$ conda --version
conda 3.16.0

Installing readline from Aaron's channel didn't help. Copying over the system readline shared objects to ~/miniconda3/lib seemed to have done the trick for better or worse.

[toinsson]

same here, temporay fix for me is to remove conda readline from the environment:

conda remove --force readline

and install the python bindings with pip install readline.

That way, I presume it is using the system readline.

[lidaof]

@toinsson's solution works for me. thanks.

[cstich]

@toinsson's solution also worked for me.

[iacercalixto]

@toinsson's solution worked here too.

Some more info: I wasn't being able to open the bc binary (in /usr/bin/bc) and was previously getting the following error:

bc: symbol lookup error: /home/icalixto/anaconda2/lib/libreadline.so.6: undefined symbol: PC

After the workaround suggested, it now works again.

[jhamrick]

I also just ran into this issue. @toinsson's workaround works for me as well.

[yourinoelnelson]

Ran into the same issue with conda version 4.2.9 while trying to install the R kernell, got the libreadline.so.6: undefined symbol: PC error while trying to install git2r package. The work around seem to solve that issue.

[stringhamc]

@toinsson workaround worked for me, I also had to do a 'pip uninstall readline' as well.

[mingwandroid]

Can you try:

conda install -c rdonnellyr r-base

[tbonza]

This was an issue for me too. Using Fedora 25, Python 3.5, IPython, R version 3.3.2. Successful workaround:

$ vi ~/.bashrc
# Rpy2
export LD_LIBRARY_PATH="/usr/lib64/R/lib:$LD_LIBRARY_PATH"
$ source ~/.bashrc
$ sudo dnf -y install readline-devel
$ ipython
[1] import readline
[2] import rpy2.robjects
[3] import pandas.rpy.common as com

The use case here is reading in an SPSS file. See "Is there a Python module to open SPSS files?" for more details on my use case

[mingwandroid]

Conda now has R 3.3.2, if you use that you should not need to use your system R and then you should not need to set LD_LIBRARY_PATH either.

[tbonza]

That's a good point. I'm actually using miniconda but forgot to mention it. LD_LIBRARY_PATH will still be necessary for miniconda, right?

…

Sent from my phone

On Dec 23, 2016, at 10:42 AM, Ray Donnelly ***@***.***> wrote: Conda now has R 3.3.2, if you use that you should not need to use your system R and then you should not need to set LD_LIBRARY_PATH either. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[mingwandroid]

I'm not sure what you mean. Provided you use the conda defaults channels to get rpy2 and R you do not need to use LD_LIBRARY_PATH, regardless of whether you installed from Anaconda or Miniconda initially.

If you try to mix conda software with non-conda software then hacks like setting LD_LIBRARY_PATH may become necessary.

[tbonza]

Oh okay, I'll try installing R from conda default and see if that resolves the issue. Happy holidays!

…

Sent from my phone

On Dec 23, 2016, at 2:30 PM, Ray Donnelly ***@***.***> wrote: I'm not sure what you mean. Provided you use the conda defaults channels to get rpy2 and R you do not need to use LD_LIBRARY_PATH, regardless of whether you installed from Anaconda or Miniconda initially. If you try to mix conda software with non-conda software then hacks like setting LD_LIBRARY_PATH may become necessary. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[mingwandroid]

@tbonza, can this be closed now?

[tbonza]

Yes, thanks!

…

Sent from my phone

On Feb 9, 2017, at 5:38 PM, Ray Donnelly ***@***.***> wrote: @tbonza, can this be closed now? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[mingwandroid]

Thank you for the confirmation.

[docsteveharris]

I have this problem on Mac OS 10.13, and @toinsson's solution didn't work

(root) ~ ❯❯❯ R                                                                            ⏎
dyld: Library not loaded: @rpath/libreadline.6.2.dylib
  Referenced from: /Users/steve/anaconda3/lib/R/lib/libR.dylib
  Reason: image not found
[1]    6747 abort      R