llvm-ar-generated library + Xcode 9 ranlib result in malformed archive

[JohanEngelen]

Bugzilla Link	34808
Version	trunk
OS	MacOS X
Attachments	Zip archive with libbug.o object file to reproduce the error

Extended Description

Running ranlib (Xcode 9) on a library generated by llvm-ar results in a malformed archive.

Using the attached object file libbug.o, there is no error using the system tools:

• rm -f libbug.ar.a
• ar rcs libbug.ar.a libbug.o
• ranlib libbug.ar.a
• nm -g libbug.ar.a (no error)

Now with llvm-ar, we get an error:

• rm -f libbug.llvmar.a
• llvm-ar rcs libbug.llvmar.a libbug.o
• ranlib libbug.llvmar.a (system ranlib, with llvm-ranlib no error happens)
• nm -g libbug.llvmar.a (errors) (also errors with llvm-nm)

The error is:
"nm: libbug.llvmar.a truncated or malformed archive (terminator characters in archive member ".o" not the correct "`\n" values for the archive member header at offset 864)"

(bug was discovered in LDC when using the internal archiver (LLVM). See ldc-developers/ldc#2350)

[llvmbot]

Looking at the hexdump of the corrupted archive

00000350 63 2e 74 6c 73 5f 61 6e 63 68 6f 72 00 00 00 00 |c.tls_anchor....|
00000360 20 20 20 20 20 20 20 20 30 20 20 20 20 20 20 20 | 0 |
00000370 20 20 20 20 30 20 20 20 20 20 30 20 20 20 20 20 | 0 0 |
0000038 36 34 34 20 20 20 20 20 32 35 34 38 20 20 20 20 #1016 2548 |
00000390 20 20 60 0a 6c 69 62 62 75 67 2e 6f 00 00 00 00 | `.libbug.o....|

the second member was supposed to start at 0x360. Given where the header ends, it should actually start at 0x358. It looks like that area was set to nulls.

I can't find anything wrong with the archive llvm is producing.

The last revision to change the llmv-ar output in this case was r294557.

Kavin, can you check if I am missing anything wrong on the llvm-ar output? If it is OK, can you check what triggers the bug in ranlib and what can be done to avoid it?

BTW, using llvm-ranlib is a reasonable workaround for this bug, no?

[JohanEngelen]

BTW, using llvm-ranlib is a reasonable workaround for this bug, no?

Not really. We are integrating the archiver into LDC, so that we have a complete cross-compile compiler :) But now that the internal archiver is creating soon-to-be-invalidated-by-ranlib archives, we probably have to disable it on OSX. (e.g. CMake will automatically run ranlib on the libraries)

[llvmbot]

BTW, using llvm-ranlib is a reasonable workaround for this bug, no?

Not really. We are integrating the archiver into LDC, so that we have a
complete cross-compile compiler :) But now that the internal archiver is
creating soon-to-be-invalidated-by-ranlib archives, we probably have to
disable it on OSX. (e.g. CMake will automatically run ranlib on the
libraries)

That is really annoying given that llvm-ar creates archives with symbol tables in them :-(

[llvmbot]

I will try to look into this today and see what is up. Hopefully I'll find the time.

[llvmbot]

Could not reproduce this. Re-installing the official Xcode 9.0 tools and rebuilding top of tree llvm from r314844 in hopes of reproducing. More later ...

[llvmbot]

Got it reproduced. Trying to figure out what is going on now.

[llvmbot]

This is a bug in Xcode's ranlib(1) when it attempts to update table of contents in place. This optimization that was introduced to rewrite the table to contents when possible.

The bug was in this case the table of contents name is using the extended name but of different size and different name. The archive built by llvm-ar has an extended name size of 12 and the name .SYMDEF where as the new one built by ranlib would have a size of 20 and a name of “.SYMDEF SORTED”.

The fix is to have ranlib(1) not do this optimization when the extended name is not the same or the same size.

Tracked with rdar://34800959

Not sure what the best work around in the llvm world is. It does appear that if you explicitly don't build a table of contents in the archive, that is use S not s, it will work:

• rm -f libbug.llvmar.a
• llvm-ar rcS libbug.llvmar.a libbug.o (using S not s)
• ranlib libbug.llvmar.a
• nm -g libbug.llvmar.a

[JohanEngelen]

I cannot reproduce this anymore after the 19 October XCode update (9.0.1). Has ranlib been fixed?