Add support for symbol visibility export files #4747

jpakkane · 2019-01-08T22:50:31Z

Not finished yet!

Documentation missing. Possibly other stuff missing. Open questions:

should this be called symbol_export_files instead since it can have several
should we limit it to at most one file
how should this tie in with free form linker scripts
are linker scripts a thing outside of GNU/Clang

emersion · 2019-01-09T15:52:27Z

This is neat, I'm still worried about using the same name for a file that is different on each platform.

Clang supports -exported_symbols_list on all platforms it seems.

how should this tie in with free form linker scripts

version script != linker script

are linker scripts a thing outside of GNU/Clang

About version scripts, I don't think macOS/Windows supports symbol versioning.

jpakkane · 2019-01-13T18:54:50Z

This is neat, I'm still worried about using the same name for a file that is different on each platform.

This is is a bit problematic but the alternative is to have a kwarg per platform (and compiler). That causes kwarg explosion and does not really work either.

Clang supports -exported_symbols_list on all platforms it seems.

I would hazard a guess that -exported_symbols_list would only be used on Mac platforms even if Clang supported it everywhere. At least on Linux you'd want to use the classical approach so one file would work both with GCC and Clang.

version script != linker script

Yes. The point was what should we do if both of those are defined. Or possibly if we should use the same code to handle both as they are semantically similar.

emersion · 2019-01-14T10:27:26Z

test cases/common/211 version file/meson.build

+if cc.get_id() == 'msvc'
+  suffix = '.sym'
+else
+  suffix = host_machine.system() == 'darwin' ? '-clang.map' : '.map'


meson.build:14:0: ERROR: Nonexisting file bob-clang.map.

emersion · 2019-01-14T10:37:07Z

mesonbuild/compilers/c.py

@@ -1156,6 +1156,10 @@ def get_linker_always_args(self):
            return basic + ['-Wl,-headerpad_max_install_names']
        return basic

+    def get_symbol_export_file_args(self, path):
+        if self.compiler_type.is_osx_compiler:


Does GCC support -exported_symbols_list on macOS? (Be careful, GCC on Linux interprets -exported_symbols_list as the -e flag with the value xported_symbols_list)

emersion · 2019-01-14T10:38:37Z

One thing I'm still worried about is that the user needs to implement the exact same "flag matching logic" than Meson in his meson.build file. The user needs to guess which flag Meson will use to feed the proper file. For instance, if building on Linux with Clang, -exported_symbols_list should not be used even if it's supported because it'll get used as a version script.

Maybe a more fail-safe API would be something among the lines of:

shared_library(symbol_export_file: {
  'version_script': '…',
  'msvc': '…',
  'macos': '…', # This one seems to be macOS-specific because all symbols seem to need a leading underscore
})

ignatenkobrain · 2019-01-27T21:00:58Z

mesonbuild/backend/vs2010backend.py

@@ -1067,6 +1067,9 @@ def gen_vcxproj(self, target, ofname, guid):
        extra_link_args += compiler.get_option_link_args(self.environment.coredata.compiler_options)
        (additional_libpaths, additional_links, extra_link_args) = self.split_link_args(extra_link_args.to_native())

+        unquoted_args, _ =  self.get_export_symbol_file_args(compiler, target, absolute_paths=True)


[Flake8] [E222] multiple spaces after operator

_{posted by Sider}

ignatenkobrain · 2019-01-27T21:00:59Z

mesonbuild/backend/backends.py

+            args += linker.get_symbol_export_file_args(final_path)
+            dep_files.append(relpath)
+        return (args, dep_files)
+


[Flake8] [W391] blank line at end of file

_{posted by Sider}

jpakkane · 2019-01-27T22:02:07Z

Finally passes on all platforms. Now we need to make the actual decisions on whether to include this or not (and in what form).

jpakkane · 2019-01-28T22:02:00Z

Maybe a more fail-safe API would be something among the lines of:

You can replicate that already with something like this:

symbol_files = {'windows': 'foo.sym',
     ...
   }

shared_library(..., version_script: symbol_files[host_machine.system()])

emersion · 2019-01-28T22:03:51Z

This won't work, because GCC on macOS will use --version-script and Clang on macOS will use -exported-symbols-list.

jpakkane · 2019-01-28T22:11:42Z

But those two files have different syntax, right? They would need to point to different files then. So in order to make it work, we'd need to parametrize the kwarg both on the compiler and the platform (and, because everything is terrible, possibly other things as well). That's a fairly big state space explosion.

emersion · 2019-01-28T22:13:01Z

The version script should work with GCC on all platforms.

jpakkane · 2019-01-28T22:21:15Z

Well if the scripts are unique to a compiler, then the same applies as above, just use the compiler id as the key. If they are not, and you need to specify both a compiler and a platform to know which file to use then you need to do manual work anyway.

emersion · 2019-01-28T22:22:52Z

Point is, Meson does the complex matching already, why duplicate the work? Also what happens if Meson's matching changes for a specific OS/compiler combination?

dcbaker · 2019-02-13T23:29:46Z

So, odd question. Have we considered implementing one format (any of the existing formats) and having meson automatically create rules to convert said format if necessary? ie, meson always uses a gcc style visibility script, and we convert that to a mac symbol visibility or msvc symbol visibility script at configure or build time? or are there dragons there I'm not thinking of?

It would be really nice to not have to define 5 different files to do the same thing (or play the awful #define game)

jpakkane · 2019-02-13T23:38:00Z

Thing I can think of:

the contents are very different, many GNU symbol scripts do versioning, which the other formats can't do (that I know of)
different platforms may export a different set of symbols so the format would need to support that

The "correct" solution is to use visibility macros rather than linker symbol scripts for everything except GNU style sumbol versioning. That requires code changes, though, and people seem to resist doing that due to various reasons.

bonzini · 2019-02-27T14:12:25Z

The "correct" solution is to use visibility macros rather than linker symbol scripts for everything except GNU style sumbol versioning. That requires code changes, though, and people seem to resist doing that due to various reasons.

FWIW, libtool used a simple list file and automatically generated a version script when needed. See -export-symbols here.

jpakkane · 2019-02-27T15:23:15Z

Libtool does not support the Visual Studio linker and it is unknown whether it supports lld either.

bonzini · 2019-02-28T10:22:19Z

Libtool does not support the Visual Studio linker and it is unknown whether it supports lld either

I was just remarking on the interface of giving a simple list of exported symbols. Such a list can be converted to a DEF file for LINK.EXE, and can be passed to -exported_symbols_list on macOS. Ut also covers lld because it supports DEF files on Windows and -exported_symbols_list on Mac.

emersion · 2019-02-28T10:24:12Z

can be passed to -exported_symbols_list on macOS

Note: on macOS symbols need to be prefixed with an underscore. See e.g. https://github.com/emersion/mrsh/blob/master/libmrsh.darwin.sym

jpakkane · 2019-03-03T19:45:16Z

Just realized that there is already vs_module_defs. This should work together with that in some way, methinks. Currently it does not.

ePirat · 2019-06-05T03:51:07Z

The "correct" solution is to use visibility macros rather than linker symbol scripts for everything except GNU style sumbol versioning. That requires code changes, though, and people seem to resist doing that due to various reasons.

This is not possible in all cases, for example you might use nasm or some other compiler that does not support any such annotations so there are cases where you need to limit symbols visibility when linking the final library.

jpakkane mentioned this pull request Jan 8, 2019

Meson needs native understanding of symbol version scripts #3047

Open

emersion reviewed Jan 14, 2019

View reviewed changes

jpakkane force-pushed the versionfile branch from 912b7b9 to 5bdd604 Compare January 16, 2019 22:32

ignatenkobrain reviewed Jan 27, 2019

View reviewed changes

jpakkane force-pushed the versionfile branch from cf5e7cf to 2cd1079 Compare February 12, 2019 20:21

jpakkane added 3 commits February 18, 2019 23:15

Move linker test to common tests to prepare for cross-platform support.

85768ff

Add support for symbol export files on Linux.

661c668

Add symbol export file support to MSVC.

432379c

jpakkane force-pushed the versionfile branch from 2cd1079 to 2675af2 Compare February 18, 2019 21:23

jpakkane added 2 commits February 19, 2019 23:25

Disable cygwin and lld platforms.

17b087f

Renamed test directory.

3c73e30

jpakkane force-pushed the versionfile branch from 2675af2 to 3c73e30 Compare February 19, 2019 21:26

nickbroon mentioned this pull request Jul 6, 2020

Support a cross-platform symbol export list for library targets #4298

Open

jpakkane mentioned this pull request Aug 12, 2020

add linker version-script helpe #7578

Closed

sdroege mentioned this pull request May 17, 2023

rust: Need to automatically do symbol export handling and --gc-sections when linking a Rust staticlib into another target #11792

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for symbol visibility export files #4747

Add support for symbol visibility export files #4747

jpakkane commented Jan 8, 2019 •

edited

Loading

emersion commented Jan 9, 2019 •

edited

Loading

jpakkane commented Jan 13, 2019

emersion Jan 14, 2019

emersion Jan 14, 2019

emersion commented Jan 14, 2019

ignatenkobrain Jan 27, 2019

ignatenkobrain Jan 27, 2019

jpakkane commented Jan 27, 2019

jpakkane commented Jan 28, 2019

emersion commented Jan 28, 2019

jpakkane commented Jan 28, 2019

emersion commented Jan 28, 2019

jpakkane commented Jan 28, 2019

emersion commented Jan 28, 2019 •

edited

Loading

dcbaker commented Feb 13, 2019

jpakkane commented Feb 13, 2019

bonzini commented Feb 27, 2019

jpakkane commented Feb 27, 2019

bonzini commented Feb 28, 2019

emersion commented Feb 28, 2019

jpakkane commented Mar 3, 2019

ePirat commented Jun 5, 2019

Add support for symbol visibility export files #4747

Are you sure you want to change the base?

Add support for symbol visibility export files #4747

Conversation

jpakkane commented Jan 8, 2019 • edited Loading

emersion commented Jan 9, 2019 • edited Loading

jpakkane commented Jan 13, 2019

emersion Jan 14, 2019

Choose a reason for hiding this comment

emersion Jan 14, 2019

Choose a reason for hiding this comment

emersion commented Jan 14, 2019

ignatenkobrain Jan 27, 2019

Choose a reason for hiding this comment

ignatenkobrain Jan 27, 2019

Choose a reason for hiding this comment

jpakkane commented Jan 27, 2019

jpakkane commented Jan 28, 2019

emersion commented Jan 28, 2019

jpakkane commented Jan 28, 2019

emersion commented Jan 28, 2019

jpakkane commented Jan 28, 2019

emersion commented Jan 28, 2019 • edited Loading

dcbaker commented Feb 13, 2019

jpakkane commented Feb 13, 2019

bonzini commented Feb 27, 2019

jpakkane commented Feb 27, 2019

bonzini commented Feb 28, 2019

emersion commented Feb 28, 2019

jpakkane commented Mar 3, 2019

ePirat commented Jun 5, 2019

jpakkane commented Jan 8, 2019 •

edited

Loading

emersion commented Jan 9, 2019 •

edited

Loading

emersion commented Jan 28, 2019 •

edited

Loading