Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Broken build with static libraries #2831

Closed
1 of 7 tasks
j-vizcaino opened this issue Apr 17, 2020 · 8 comments
Closed
1 of 7 tasks

Broken build with static libraries #2831

j-vizcaino opened this issue Apr 17, 2020 · 8 comments

Comments

@j-vizcaino
Copy link

Description

Starting with 1.4.0, the pkg-config file installed by the build does not point to the expected librdkafka.a but to a missing librdkafka-static.a.

I believe the regression was introduced by 86461c0

How to reproduce

./configure --prefix=/usr/local --enable-lz4-ext
make check install
  • /usr/local/lib contains librdkafka.a
  • /usr/local/lib/pkgconfig/rdkafka-static.pc points to librdkakfa-static.a but it is not installed
  • with librdkafka < 1.4.0 rdkafka-static.pc points to librdkafka.a

Those exact same steps work without any issue in librdkafka < 1.4.0 and allow me to use static build flavor with the confluent-kafka-go wrapper library.

The reason why I can't use the fully static build librdkafka-static.a is because my Go program uses another C library that depends on LZ4. If both libs are built statically, the final linker stage complains about duplicate symbols.

Checklist

IMPORTANT: We will close issues where the checklist has not been completed.

Please provide the following information:

  • librdkafka version (release number or git tag): 1.4.0
  • Apache Kafka version: <REPLACE with e.g., 0.10.2.3>
  • librdkafka client configuration: <REPLACE with e.g., message.timeout.ms=123, auto.reset.offset=earliest, ..>
  • Operating system: Ubuntu 16.04
  • Provide logs (with debug=.. as necessary) from librdkafka
  • Provide broker log excerpts
  • Critical issue
@edenhill edenhill added this to the v1.5.0 milestone Apr 28, 2020
@alx696
Copy link

alx696 commented Jun 15, 2020

before v1.3, easy to build alpine image: go build -tags static_all. But v1.4 can not.

@edenhill
Copy link
Contributor

edenhill commented Jun 15, 2020

Librdkafka is included as a static object since v1.4.0, should be sufficient to build your app with -tags musl if you are building for alpine.

@alx696
Copy link

alx696 commented Jun 19, 2020

@edenhill With -tags musl can not build:

$ go build -tags musl -o ./docker/app/main
# github.com/confluentinc/confluent-kafka-go/kafka
/usr/bin/ld: /home/m/go/pkg/mod/github.com/confluentinc/[email protected]/kafka/librdkafka/librdkafka_musl_linux.a(rdkafka_txnmgr.o): in function `rd_kafka_txn_register_partitions':
(.text+0xa10): undefined reference to `strlcpy'
/usr/bin/ld: /home/m/go/pkg/mod/github.com/confluentinc/[email protected]/kafka/librdkafka/librdkafka_musl_linux.a(rdkafka_txnmgr.o): in function `rd_kafka_txn_handle_AddOffsetsToTxn':
(.text+0x3e4f): undefined reference to `strlcpy'
/usr/bin/ld: (.text+0x42ac): undefined reference to `strlcpy'
/usr/bin/ld: (.text+0x43dc): undefined reference to `strlcpy'
/usr/bin/ld: /home/m/go/pkg/mod/github.com/confluentinc/[email protected]/kafka/librdkafka/librdkafka_musl_linux.a(rdkafka_txnmgr.o): in function `rd_kafka_txn_handle_TxnOffsetCommit':
(.text+0x4993): undefined reference to `strlcpy'
/usr/bin/ld: /home/m/go/pkg/mod/github.com/confluentinc/[email protected]/kafka/librdkafka/librdkafka_musl_linux.a(rdkafka_txnmgr.o):(.text+0x4ca2): more undefined references to `strlcpy' follow
collect2: error: ld returned 1 exit status
  • System: Ubuntu 20.04
  • Golang: go version go1.14.2 linux/amd64

With -tags static_all, build ok, but can not run in alpine. v1.4.0 good for linux(do not need to install librdkafka first), but had problem for alpine.

@edenhill
Copy link
Contributor

If you are building for Alpine (musl) you will most likely need to build in an Alpine docker container so that the Go toolchain picks up the correct C library to link.

@j-vizcaino
Copy link
Author

If you are building for Alpine (musl) you will most likely need to build in an Alpine docker container so that the Go toolchain picks up the correct C library to link.

Wait, that's not what the initial issue was about. Unless this has been addressed somehow (see issue description), I think this issue should be kept open.

@edenhill
Copy link
Contributor

Whoops, sorry!

@edenhill edenhill reopened this Jun 29, 2020
@edenhill edenhill removed this from the v1.5.0 milestone Jul 6, 2020
@abrownsword
Copy link

Has this issue been addressed?

@xcaptain
Copy link

Build in an Alpine container solved this issue, but can anyone provide some instructions on building on a Linux dev machine?

@edenhill edenhill closed this as completed Apr 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants