Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go tool rejects #cgo LDFLAGS: -Wl,-rpath,@executable_path/../Frameworks #40559

Open
abemedia opened this issue Aug 3, 2020 · 17 comments · May be fixed by #70939
Open

cmd/go: go tool rejects #cgo LDFLAGS: -Wl,-rpath,@executable_path/../Frameworks #40559

abemedia opened this issue Aug 3, 2020 · 17 comments · May be fixed by #70939
Labels
help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Darwin
Milestone

Comments

@abemedia
Copy link

abemedia commented Aug 3, 2020

What version of Go are you using (go version)?

$ go version
go version go1.14.6 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/adam/Library/Caches/go-build"
GOENV="/Users/adam/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/adam/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.14.6/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.14.6/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/adam/Work/go-test/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/rp/mzkkm6297lv16qf_qgszt3wm0000gn/T/go-build776046366=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I tried to build a MacOS bundle using dynamically linked libraries. As per convention these are stored in MyApp.app/Contents/Frameworks which requires LC_RPATH to point there.

I added the following flag:

#cgo LDFLAGS: -Wl,-rpath,@executable_path/../Frameworks

What did you expect to see?

go build builds binary with LC_RPATH set to @executable_path/../Frameworks

What did you see instead?

I got an invalid flag error upon running go build.

I also tried replacing @executable_path for $ORIGIN as I would in Linux but then I got an image not found error when running the resulting binary (I assume this isn't supported on MacOS).

@ianlancetaylor ianlancetaylor changed the title cgo: cannot set @rpath to point to Frameworks on darwin cmd/go: go tool rejects #cgo LDFLAGS: -Wl,-rpath,@executable_path/../Frameworks Aug 4, 2020
@ianlancetaylor
Copy link
Member

What does @executable_path mean?

Any fix for this will have to be very careful. For the GNU tools, among others, starting a parameter with @ causes it to be replaced with the contents of a file with the string following the @. So if we do need to handle @executable_path, we need to be careful to only do it in a context in which we know for sure that it won't be replaced by a file. Otherwise all of this flag vetting no longer works.

@ianlancetaylor ianlancetaylor added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Darwin labels Aug 4, 2020
@ianlancetaylor ianlancetaylor added this to the Backlog milestone Aug 4, 2020
@ianlancetaylor ianlancetaylor added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Aug 4, 2020
@abemedia
Copy link
Author

abemedia commented Aug 4, 2020

@exexutable_path is like the MacOS version of $ORIGIN on Linux i.e. it is the path that the binary is in at runtime to be able to dynamically link to a library using a relative path.

@abemedia
Copy link
Author

abemedia commented Aug 4, 2020

Here's some more info: https://wincent.com/wiki/@executable_path,_@load_path_and_@rpath
While we're at it we should probably add @loader_path as well as @executable_path.

@jayconrod
Copy link
Contributor

It looks like @ is specifically forbidden as a leading character since we started validating cgo flags in 1dcb5836.

A wrinkle in all this is that GNU binutils uniformly accept
@foo on the command line to mean "if the file foo exists,
then substitute its contents for @foo in the command line".
So we must also reject @x.go, flags and flag arguments
beginning with @, and so on.

That would still be an issue here. Perhaps we could support specifically @loader_path and @executable_path (nothing else that starts with @), and only allow those when GOOS=darwin?

@abemedia
Copy link
Author

abemedia commented Aug 4, 2020

Yes, that sounds like the ideal solution!

@ianlancetaylor ianlancetaylor added help wanted and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Aug 4, 2020
@abemedia
Copy link
Author

abemedia commented Aug 5, 2020

If someone can point me to where I need to look I'm happy to have a bash at a PR.

@jayconrod
Copy link
Contributor

security.go contains regular expressions for allowed flags.

cfg.GOOS may be used to check the target operating system. I don't think we have any regular expressions that are allowed on one platform but not others, but that would be necessary here.

cgo_bad_directives.txt tests this filtering, so that would need to be expanded. See README.txt in that directory for information on the test format.

@snmed
Copy link

snmed commented Aug 26, 2020

Actually I'm evaluating go for a project on mac and i've a poc that needs to load an external *.dylib from a directory relative to the binary. So I stumbled over @executable_path and this issue with go, so is there any known workaround to get a go binary working that can load a *.dylib from a directory relative to the binary?

Any suggestions would be great, I really would like to use go for the project, but we need to load external C libraries that are not installed in any of the OS search paths.

Update:
Sorry for the post but I've found at least a workaround for that:

  1. Compile my go file with cgo flags
package native

/*
#cgo LDFLAGS: -L../path/to/libs -Wl,-rpath,./libs -lgreeter
#include "path/to/libs/include/greeter.h"
#include <stdlib.h>
*/
import "C"

// Rest of code omitted for brevity
  1. Run go build -o mybinary
  2. Run the install_name_tool
# You can check the correct name for change with otool -L mybinary
install_name_tool -change @rpath/libgreeter.dylib @executable_path/libs/x64/libgreeter.dylib mybinary

I'm not sure if this would break anything else, but so far everything is working as expected. I still hope that the go tooling will allow the @executable_path and @loader_path flags in the future.

@ianlancetaylor
Copy link
Member

@snmed I'm glad you found a way. You can also set set the CGO_LDFLAGS_ALLOW environment variable. https://golang.org/cmd/go/#hdr-Environment_variables

@snmed
Copy link

snmed commented Aug 27, 2020

@ianlancetaylor Thanks for your advice, that worked very well with CGO_LDFLAGS_ALLOW="-Wl,-rpath,@executable_path/.*" go build, now only the vscode extensions still shows the invalid flag in #cgo LDFLAGS:-Wl,-rpath,@executable_path/libs maybe there is some setting for that. A workaround for that is to set the variable for the whole development environment.

battlmonstr added a commit to erigontech/silkworm-go that referenced this issue Dec 14, 2023
on Linux search relative to $ORIGIN (see https://stackoverflow.com/a/72577483/1009546 )
on macOS it should search relative to @executable_path, but Go doesn't support it:
golang/go#40559
battlmonstr added a commit to erigontech/silkworm-go that referenced this issue Dec 14, 2023
on Linux search relative to $ORIGIN (see https://stackoverflow.com/a/72577483/1009546 )
on macOS it should search relative to @executable_path, but Go doesn't support it:
golang/go#40559
@t0rr3sp3dr0
Copy link

I've just hit this issue and the CGO_LDFLAGS_ALLOW workaround doesn't work for me. I want to be able to go install my project without having to instruct people to set that variable.

Since @ doesn't actually read files on macOS, would it be acceptable to patch the code to allow it on Darwin? If yes, I can submit a PR implementing that.

@ianlancetaylor
Copy link
Member

Some people do use the GNU tools on Darwin, and those tools do read files when they see an @ at the start of an option. So we must not in general ignore @ on Darwin.

If the specific string @executable_path is magic, then perhaps we can recognize that string on Darwin. Are there other magic strings like that one?

@t0rr3sp3dr0
Copy link

t0rr3sp3dr0 commented Dec 19, 2024

For -rpath, @executable_path and @loader_path are the only special values that you can provide. See https://forums.developer.apple.com/forums/thread/736728.

@t0rr3sp3dr0
Copy link

t0rr3sp3dr0 commented Dec 19, 2024

The only other argument that I’m aware of that can receive a value starting with @ is -install_name. It accepts @rpath to indicate the library may be installed at any of the paths defined by -rpath. I can open a new issue for that one if you prefer.

@t0rr3sp3dr0
Copy link

More info about these 3 special values at the very bottom of this man page: https://www.unix.com/man-page/osx/1/dyld/.

@t0rr3sp3dr0
Copy link

I did some extra research and it looks like the GNU ld doesn't support Darwin as a target. The only way to use it on macOS is by targeting another platform to produce an ELF file. You cannot use it to produce a Mach-O file. See Homebrew/homebrew-core#17794.

Since @executable_path, @loader_path, and @rpath are only meaningful in Mach-O, can we check the target platform and allow or disallow these flags based on that?

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/638075 mentions this issue: cmd/go/internal/work: allow @ character in some -Wl, linker flags on darwin

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Darwin
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants