[devel regression] {.exportc.} tagged procs no longer export to compiled .so objects

[kaushalmodi]

This is a serious show stopper for my SystemVerilog projects relying on Nim libaries.

In my Nim libraries, I export the procs using the {.exportc.} pragma, and the C layer of SystemVerilog imports those exported symbols to the SystemVerilog side.

But now the {.exportc.} procs do not get exported at all.

Example

Compile the below file on Linux using nim c --out:libdpi.so --app:lib -d:release --gc:none libdpi.nim.

# libdpi.nim
proc hello() {.exportc.} =
  let
    str = when defined(cpp):
            "C++!"
          else:
            "C!"
  echo "Hello from Nim via " & str

Now run nm ./libdpi.so | grep -P '\bT\b' (Note: This is nm binary found on Linux systems, not nim.)

Current Output

!! hello symbol is not exported !!

0000000000007ab0 T NimMain
0000000000007b54 T _fini
00000000000007f8 T _init

Expected Output

Note that the hello symbol is exported: 0000000000005f80 T hello.

0000000000006070 T NimMain
00000000000060a0 T NimMainInit
0000000000005f70 T NimMainInner
0000000000006040 T PreMain
0000000000005f60 T PreMainInner
00000000000060f4 T _fini
00000000000018b0 T _init
0000000000004bf0 T addChar
00000000000040c0 T copyString
0000000000003eb0 T cstrToNimstr
0000000000001c10 T echoBinSafe
0000000000005f80 T hello
0000000000005710 T incrSeqV3
0000000000004d10 T mnewString
0000000000005610 T mulInt
00000000000047b0 T newObj
00000000000056b0 T newSeq
0000000000004f70 T nimIntToStr
00000000000054f0 T raiseExceptionEx
00000000000057c0 T raiseIndexError2
00000000000055b0 T raiseOverflow
0000000000004b20 T rawNewString
0000000000003af0 T rawNewStringNoInit
0000000000004b50 T resizeString
0000000000004d20 T setLengthStr
0000000000003ee0 T signalHandler
0000000000003e80 T toNimStr

Possible Solution

None that I know of.

Additional Information

• It was working probably before this PR got merged: compiler/ccgtypes: hide exportc proc unless it has dynlib #13199 /cc @alaviss Can you please confirm is this outcome matches with what your PR expected to do?
• This issue is blocking my efforts to pair Nim with SystemVerilog.

$ nim -v
Nim Compiler Version 1.1.1 [Linux: amd64]
Compiled at 2020-02-13
Copyright (c) 2006-2019 by Andreas Rumpf

git hash: 1e303100f8ca32e60b154bf997a9102f36eb23d0
active boot switches: -d:release

[kaushalmodi]

I already found this difference.. when the symbol exporting was working, the hello symbol had capital T before it in the nm output.

> nm ./libdpi_64.good.so | rg hello 
0000000000005f80 T hello

In the not-working .so, the hello symbol has lower-case t before it.

> nm ./libdpi_64.bad.so | rg hello 
00000000000079c0 t hello

I do not understand the deal betweeen T and t, but apparently it definitely needs to be T for the Nim/SystemVerilog interop to work.

[kaushalmodi]

Ref:

• https://stackoverflow.com/a/37534357/1219634

• http://web.mit.edu/gnu/doc/html/binutils_3.html

  The symbol type. At least the following types are used; others are, as well, depending on the object file format. If lowercase, the symbol is local; if uppercase, the symbol is global (external).

• t means that the symbol is local

• T means that the symbol is global <-- This is what we need for {.exportc.} annotated procs.

[kaushalmodi]

This got fixed after adding dynlib pragma to exportc:

proc hello() {.exportc, dynlib.} =
  let
    str = when defined(cpp):
            "C++!"
          else:
            "C!"
  echo "Hello from Nim via " & str

Now I see:

> nm ./libdpi_64.so | rg '\bT\b'
0000000000007ad0 T NimMain
0000000000007b74 T _fini
0000000000000818 T _init
00000000000079e0 T hello

[kaushalmodi]

@alaviss Can you please add a changelog for this, and also update the manual: https://nim-lang.github.io/Nim/manual#foreign-function-interface-exportc-pragma

It needs to be highlighted that when the user expects to export a symbol to a .so, etc., it needs to have both {.exportc, dynlib.}.

This also needs to change: https://nim-lang.github.io/Nim/manual#foreign-function-interface-dynlib-pragma-for-export

This pragma only has an effect for the code generation on the Windows target, so when this pragma is forgotten and the dynamic library is only tested on Mac and/or Linux, there won't be an error.

---

I seriously wonder if there's a way for .exportc. to now implicitly do .dynlib.

From my perspective, it looks like:

• Most of .exportc.'s use requires .dynlib. too, so may be .exportc. should automatically behave as if .dynlib. was used.
• .dynlib. can then be deprecated ..
• Can .exportc. respect a special symbol in its argument? Like: .exportc: "!". ? Another example with a specified export symbol name: {.export: "!foo".}.

That would then export the symbols locally (the nm t symbol) instead of globally (the nm T symbol).

This proposed change would be much less breaking as the {.export: "!".} use, I believe, would be mainly internal to the nim repo, right?

[juancarlospaco]

Awesome research.

[timotheecour]

I seriously wonder if there's a way for .exportc. to now implicitly do .dynlib.

nah, visibility is not same as mangling. The change to dynlib was a good one.