Broken build with static libraries

[j-vizcaino]

Description

Starting with 1.4.0, the pkg-config file installed by the build does not point to the expected librdkafka.a but to a missing librdkafka-static.a.

I believe the regression was introduced by 86461c0

How to reproduce

./configure --prefix=/usr/local --enable-lz4-ext
make check install

• /usr/local/lib contains librdkafka.a
• /usr/local/lib/pkgconfig/rdkafka-static.pc points to librdkakfa-static.a but it is not installed
• with librdkafka < 1.4.0 rdkafka-static.pc points to librdkafka.a

Those exact same steps work without any issue in librdkafka < 1.4.0 and allow me to use static build flavor with the confluent-kafka-go wrapper library.

The reason why I can't use the fully static build librdkafka-static.a is because my Go program uses another C library that depends on LZ4. If both libs are built statically, the final linker stage complains about duplicate symbols.

Checklist

IMPORTANT: We will close issues where the checklist has not been completed.

Please provide the following information:

• librdkafka version (release number or git tag): 1.4.0
• Apache Kafka version: <REPLACE with e.g., 0.10.2.3>
• librdkafka client configuration: <REPLACE with e.g., message.timeout.ms=123, auto.reset.offset=earliest, ..>
• Operating system: Ubuntu 16.04
• Provide logs (with debug=.. as necessary) from librdkafka
• Provide broker log excerpts
• Critical issue

[alx696]

before v1.3, easy to build alpine image: go build -tags static_all. But v1.4 can not.

[edenhill]

Librdkafka is included as a static object since v1.4.0, should be sufficient to build your app with -tags musl if you are building for alpine.

[alx696]

@edenhill With -tags musl can not build:

$ go build -tags musl -o ./docker/app/main
# github.com/confluentinc/confluent-kafka-go/kafka
/usr/bin/ld: /home/m/go/pkg/mod/github.com/confluentinc/[email protected]/kafka/librdkafka/librdkafka_musl_linux.a(rdkafka_txnmgr.o): in function `rd_kafka_txn_register_partitions':
(.text+0xa10): undefined reference to `strlcpy'
/usr/bin/ld: /home/m/go/pkg/mod/github.com/confluentinc/[email protected]/kafka/librdkafka/librdkafka_musl_linux.a(rdkafka_txnmgr.o): in function `rd_kafka_txn_handle_AddOffsetsToTxn':
(.text+0x3e4f): undefined reference to `strlcpy'
/usr/bin/ld: (.text+0x42ac): undefined reference to `strlcpy'
/usr/bin/ld: (.text+0x43dc): undefined reference to `strlcpy'
/usr/bin/ld: /home/m/go/pkg/mod/github.com/confluentinc/[email protected]/kafka/librdkafka/librdkafka_musl_linux.a(rdkafka_txnmgr.o): in function `rd_kafka_txn_handle_TxnOffsetCommit':
(.text+0x4993): undefined reference to `strlcpy'
/usr/bin/ld: /home/m/go/pkg/mod/github.com/confluentinc/[email protected]/kafka/librdkafka/librdkafka_musl_linux.a(rdkafka_txnmgr.o):(.text+0x4ca2): more undefined references to `strlcpy' follow
collect2: error: ld returned 1 exit status

• System: Ubuntu 20.04
• Golang: go version go1.14.2 linux/amd64

With -tags static_all, build ok, but can not run in alpine. v1.4.0 good for linux(do not need to install librdkafka first), but had problem for alpine.

[edenhill]

If you are building for Alpine (musl) you will most likely need to build in an Alpine docker container so that the Go toolchain picks up the correct C library to link.

[j-vizcaino]

If you are building for Alpine (musl) you will most likely need to build in an Alpine docker container so that the Go toolchain picks up the correct C library to link.

Wait, that's not what the initial issue was about. Unless this has been addressed somehow (see issue description), I think this issue should be kept open.

[edenhill]

Whoops, sorry!

[abrownsword]

Has this issue been addressed?

[xcaptain]

Build in an Alpine container solved this issue, but can anyone provide some instructions on building on a Linux dev machine?