Can't compile code when using openssl from MacPorts

[learnopengles]

Hi team,

I was trying out dinghy and had some issues running cargo install dinghy due to some undefined git symbols. It seems that it doesn't work when openssl is installed from MacPorts, but it does work when installed from Homebrew.

Here's the linker error:

 = note: Undefined symbols for architecture x86_64:
  "_iconv", referenced from:
      _git_path_iconv in liblibgit2_sys-ce5bbe4559aebb33.rlib(path.c.o)
     (maybe you meant: _git_path_iconv_init_precompose, _git_path_iconv , _git_path_iconv_clear )
  "_iconv_close", referenced from:
      _git_path_iconv_clear in liblibgit2_sys-ce5bbe4559aebb33.rlib(path.c.o)
  "_iconv_open", referenced from:
      _git_path_iconv_init_precompose in liblibgit2_sys-ce5bbe4559aebb33.rlib(path.c.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Here is my config:

macOS 10.12.2
Xcode 8.2.1
git version 2.10.1 (Apple Git-78)

sudo port install cmake
sudo port install openssl
export OPENSSL_INCLUDE_DIR=/opt/local/include
export OPENSSL_LIB_DIR=/opt/local/lib
cargo clean
cargo install dinghy

See also: sonos/dinghy#3

---

Here's some sample code which reduces the problem:

Cargo.toml

[package]
name = "testing"
version = "0.1.6-pre"

[dependencies]
cargo = "0.15"

main.rs

extern crate cargo;

use std::{env};
use cargo::util::important_paths::find_root_manifest_for_wd;

fn main() {
    let wd_path = find_root_manifest_for_wd(None, &env::current_dir().unwrap()).unwrap();
    let cfg = cargo::util::config::Config::default().unwrap();
    let mode = cargo::ops::CompileMode::Test;    
        
    let wd = cargo::core::Workspace::new(&wd_path, &cfg).unwrap();    
    let options = cargo::ops::CompileOptions {
        config: &cfg,
        jobs: None,
        target: None,
        features: &[],
        all_features: false,
        no_default_features: false,
        spec: &[],
        filter: cargo::ops::CompileFilter::new(false, &[], &[], &[], &[]),
        mode: mode,
        release: false,        
        message_format: cargo::ops::MessageFormat::Human,
        target_rustdoc_args: None,
        target_rustc_args: None,
    };
    let _ = cargo::ops::compile(&wd, &options).unwrap();
}

[alexcrichton]

Do you have a version of iconv in macports as well? It looks like libgit2 may have been compiled against one and linked against another.

[learnopengles]

Hi Alex, yes I found a libiconv in the macports dir. I tried this from a fresh install with just openssl and cmake installed.

---

Here is the full build output: http://pastebin.com/jNPXdEEL

[learnopengles]

I can see these lines related to iconv:

1. When compiling the C code for libgit2-sys-0.6.6:

-- Found Iconv: -L/usr/lib -liconv

Then when running rustc on libgit2-sys-0.6.6/lib.rs:

-L native=/opt/local/lib

Where there is also an iconv.

So I'm not sure I'm reading this right but it seems your hypothesis was correct! How could we fix this?

[alexcrichton]

Unfortunately I'm not sure there's really a great way to fix this other than coercing libgit2 to find iconv in /opt/local/lib first (which I'm not sure how to do)

[learnopengles]

This PR seems relevant: https://github.com/libgit2/libgit2/pull/3092/files

I don't know a great way to fix it either but without a fix, the build will be broken for macports users. Is there a way to override this through a linker path or something else from our side as the consumer of libgit2? Maybe following their behavior on the Rust side?

[learnopengles]

I can get it working by adding this to libgit2-sys' build.rs:

cfg.define("CMAKE_PREFIX_PATH", "/opt/local/");

I'm able to compile and run the git2 tests without running into that linker error.

It's actually not required to modify build.rs, as this works, too:

CMAKE_PREFIX_PATH=/opt/local/ cargo install dinghy

---

This makes everything work! However, it's definitely not newbie friendly! It took me a while to figure out that I needed to do this + also override the OPENSSL env vars to get everything working.

I also don't know how great of a solution this is; the libgit2 team intentionally removed /opt/local from their search path and they seem to prefer defaulting to the system libraries, while cargo does search in /opt/local by default (and right now I'm not sure how to override that). Sys programming is not my main job :), so I'm really not sure which way is "right".

[alexcrichton]

In any case this seems like a bug for the git2 crate not Cargo itself. It could perhaps do tricks to figure out what's being printed where, but this is unfortunately just a situation where having lots of libs that shadow each other doesn't work out well when there's no real dependencies between them.

[learnopengles]

It might be cargo-related if there is an issue with the way cargo is looking up the paths, though I don't really know if this is or not. Why does cargo look in /opt without being told to? Is it using the path? Is this easily overridable using env vars?

Otherwise sure, I could close this issue and open one up on the git2 crate.

I'm just wondering how to make this all more newbie-friendly and how to teach the user about these semantics because right now, it's a bit of a frustrating experience.

[alexcrichton]

Oh sorry what I meant was that Cargo isn't involved here at all, this is all libgit2-sys's build script (changes necessary there at least)

[learnopengles]

Oh, sorry, so you mean the build.rs? OK I'll move the issue over :)

[alexcrichton]

Yeah this is all basically logic that should go into libgit2-sys's build.rs. Either in the way of error message or in terms of probing for the right libs.

[learnopengles]

OK I see now -- still learning about all of this. I opened up https://github.com/alexcrichton/git2-rs/issues/180 to track over there.

[nickolay]

I opened up https://github.com/alexcrichton/git2-rs/issues/180 to track over there.

Sorry for the spam, but the updated link is rust-lang/git2-rs#180 and to build git2-rs I ended up using:

port install openssl libgit2 pkgconfig
LIBGIT2_SYS_USE_PKG_CONFIG=1 cargo ...

update git2-rs 0.13+ no longer requires the LIBGIT2_SYS_USE_PKG_CONFIG hack, but does require libgit2 to be >= 1.0.0, which was added to macports after a multi-month delay.

[vtenfys]

This issue seems to have arisen again on macOS 11; see my comment here: rust-lang/git2-rs#180 (comment)

Unfortunately there isn't a nice workaround other than deactivating the libiconv port before building and reactivating afterwards, as described here: https://stackoverflow.com/a/34140282/4247209