Module linkage fixes for shared build #4169

kamrann · 2024-09-20T04:52:51Z

Following on from #4153:

In regards to constexpr, this was indeed a Clang bug, and anything already marked constexpr should be a non-issue; the problem has been fixed on Clang trunk with potential for it to be backported to the 19.x release.
A number of other linker errors specific to mixing #include with import were also being caused by a related Clang bug, also now fixed on trunk.
Despite this, a bunch of linker errors still remained when trying to consume the fmt module via a shared library build, due to the last bullet point in the linked issue.

This PR addresses those remaining issues by adding FMT_INLINE/FMT_CONSTEXPR to some member functions defined within the class definition. However, due to the nature of linker issues, problematic cases are only being picked up where a symbol is actually being used from an external dll/exe. Currently, the coverage for that is limited - fmt's own CI is not yet running modules builds I think (or anyway not with FMT_SHARED); the build2 fmt package CI also cannot stress test this as the very latest Clang is not available. Furthermore, I've so far only enabled a subset of the tests in the build2 package.

Fundamentally, I think this addition of inline or constexpr is essentially needed on every single in-class defined member function which is intended for public use (or alternatively FMT_API on the member/class), which is probably a lot more than what I've addressed in these commits and would be quite intrusive. So of course feel free to merge if you like, but it's possible some more thought is warranted in regards to

is every public member function in the headers actually intended to be usable directly by clients (this question of course also relates to what should be exported from the module, but for export, doing things in bulk is possible and less intrusive in terms of code changes)
is there a preference for exposing those that should be exposed as inline vs FMT_API

…texpr, to avoid missing external symbols when using fmt module with shared build due to modules not defaulting to implicit inline.

NOTE: Looks as if basic_format_args::get(string_view) could probably be made constexpr instead, but sticking with minimal change approach.

…+ std::string constexpr limitations.

kamrann · 2024-09-20T08:29:52Z

Linter seems to be complaining about formatting that I didn't change.

vitaut · 2024-09-21T14:47:12Z

Before we go into detail, could you link to where this assumption comes from?

inside a module TU, member functions defined within the class definition are not implicitly inline

This seems like a major incompatibility and I would be surprised if it was the case.

kamrann · 2024-09-22T03:40:52Z

Sure. The relevant revision to the standard was P1779. There's also some discussion regarding it in this SO post.

vitaut · 2024-09-22T15:28:44Z

include/fmt/base.h

+  FMT_CONSTEXPR explicit buffer_traits(size_t) {}
+  FMT_CONSTEXPR auto count() const -> size_t { return 0; }
+  FMT_CONSTEXPR auto limit(size_t size) -> size_t { return size; }


These can be constexpr. FMT_CONSTEXPR is only for cases when C++14 relaxed constexpr is needed.

Okay my bad, hadn't realized the distinction. I think all the changes are now constexpr where they can be.

vitaut · 2024-09-22T15:29:09Z

include/fmt/base.h

+  FMT_CONSTEXPR explicit fixed_buffer_traits(size_t limit) : limit_(limit) {}
+  FMT_CONSTEXPR auto count() const -> size_t { return count_; }
+  FMT_CONSTEXPR auto limit(size_t size) -> size_t {


Same here and in a few more places below.

vitaut · 2024-09-22T15:31:30Z

include/fmt/base.h

@@ -2223,7 +2223,7 @@ struct locale_ref {
 public:
  constexpr locale_ref() : locale_(nullptr) {}
  template <typename Locale> explicit locale_ref(const Locale& loc);
-  explicit operator bool() const noexcept { return locale_ != nullptr; }
+  FMT_INLINE explicit operator bool() const noexcept { return locale_ != nullptr; }


Here and elsewhere, please use inline instead of FMT_INLINE since the latter expands to always inline in optimized mode.

vitaut · 2024-09-22T15:32:58Z

include/fmt/chrono.h

+    FMT_INLINE dispatcher(std::time_t t) : time_(t) {}

-    auto run() -> bool {
+    FMT_INLINE auto run() -> bool {
      using namespace fmt::detail;
      return handle(localtime_r(&time_, &tm_));
    }

-    auto handle(std::tm* tm) -> bool { return tm != nullptr; }
+    FMT_INLINE auto handle(std::tm* tm) -> bool { return tm != nullptr; }

-    auto handle(detail::null<>) -> bool {
+    FMT_INLINE auto handle(detail::null<>) -> bool {
      using namespace fmt::detail;
      return fallback(localtime_s(&tm_, &time_));
    }

-    auto fallback(int res) -> bool { return res == 0; }
+    FMT_INLINE auto fallback(int res) -> bool { return res == 0; }

 #if !FMT_MSC_VERSION
-    auto fallback(detail::null<>) -> bool {
+    FMT_INLINE auto fallback(detail::null<>) -> bool {


This is not needed since the struct where these functions are defined is inaccessible to module consumers.

Although I may have branched out a little when making the changes, the vast majority of them were made on functions that were generating linker errors. I've verified again, these are necessary for the linker to succeed.

Essentially, the issue being addressed here is one of dll symbol export, not module name export, and so the visibility of names (which is what module export/detail namespaces relate to) I think has no bearing. Since this struct is in an inline function which is exposed, a consumer TU that calls this function (directly or transitively) will either need to inline the member functions of the struct, or emit an external symbol reference for them. So the same choice still applies.

include/fmt/chrono.h

vitaut · 2024-09-22T15:35:00Z

include/fmt/chrono.h

@@ -912,7 +912,7 @@ template <typename Derived> struct null_chrono_spec_handler {
 };

 struct tm_format_checker : null_chrono_spec_handler<tm_format_checker> {
-  FMT_NORETURN void unsupported() { FMT_THROW(format_error("no format")); }
+  FMT_NORETURN FMT_INLINE void unsupported() { FMT_THROW(format_error("no format")); }


Here and in other places in the detail namespace - inline is not needed since it is not exported from the module.

As above, detail namespace functions will still generate linker errors if neither inline nor dll-exported.

vitaut · 2024-09-25T15:21:52Z

OK, considering that the number of changes is manageable, we can merge it in but please apply clang-format first.

vitaut · 2024-09-26T14:54:01Z

Merged, thanks!

@timsong-cpp

- Update from version 11.0.2 to 11.1.3 - Update of rootfile - Changelog 11.1.3 - Fixed compilation on GCC 9.4 (fmtlib/fmt#4313). - Worked around an internal compiler error when using C++20 modules with GCC 14.2 and earlier (fmtlib/fmt#4295). - Worked around a bug in GCC 6 (fmtlib/fmt#4318). - Fixed an issue caused by instantiating `formatter<const T>` (fmtlib/fmt#4303, fmtlib/fmt#4325). Thanks @timsong-cpp. - Fixed formatting into `std::ostreambuf_iterator` when using format string compilation (fmtlib/fmt#4309, fmtlib/fmt#4312). Thanks @phprus. - Restored a constraint on the map formatter so that it correctly reports as unformattable when the element is (fmtlib/fmt#4326). Thanks @timsong-cpp. - Reduced the size of format specs (fmtlib/fmt#4298). - Readded `args()` to `fmt::format_context` (fmtlib/fmt#4307, fmtlib/fmt#4310). Thanks @Erroneous1. - Fixed a bogus MSVC warning (fmtlib/fmt#4314, fmtlib/fmt#4322). Thanks @ZehMatt. - Fixed a pedantic mode error in the CMake config (fmtlib/fmt#4327). Thanks @rlalik. 11.1.2 - Fixed ABI compatibility with earlier 11.x versions (fmtlib/fmt#4292). - Added `wchar_t` support to the `std::bitset` formatter (fmtlib/fmt#4285, fmtlib/fmt#4286, fmtlib/fmt#4289, fmtlib/fmt#4290). Thanks @phprus. - Prefixed CMake components with `fmt-` to simplify usage of {fmt} via `add_subdirectory` (fmtlib/fmt#4283). - Updated docs for meson (fmtlib/fmt#4291). Thanks @trim21. - Fixed a compilation error in chrono on nvcc (fmtlib/fmt#4297, fmtlib/fmt#4301). Thanks @breyerml. - Fixed various warnings (fmtlib/fmt#4288, fmtlib/fmt#4299). Thanks @GamesTrap and @edo9300. 11.1.1 - Fixed ABI compatibility with earlier 11.x versions (fmtlib/fmt#4278). - Defined CMake components (`core` and `doc`) to allow docs to be installed separately (fmtlib/fmt#4276). Thanks @carlsmedstad. 11.1.0 - Improved C++20 module support (fmtlib/fmt#4081, fmtlib/fmt#4083, fmtlib/fmt#4084, fmtlib/fmt#4152, fmtlib/fmt#4153, fmtlib/fmt#4169, fmtlib/fmt#4190, fmtlib/fmt#4234, fmtlib/fmt#4239). Thanks @kamrann and @Arghnews. - Reduced debug (unoptimized) binary code size and the number of template instantiations when passing formatting arguments. For example, unoptimized binary code size for `fmt::print("{}", 42)` was reduced by ~40% on GCC and ~60% on clang (x86-64). GCC: - Before: 161 instructions of which 105 are in reusable functions ([godbolt](https://www.godbolt.org/z/s9bGoo4ze)). - After: 116 instructions of which 60 are in reusable functions ([godbolt](https://www.godbolt.org/z/r7GGGxMs6)). Clang: - Before: 310 instructions of which 251 are in reusable functions ([godbolt](https://www.godbolt.org/z/Ts88b7M9o)). - After: 194 instructions of which 135 are in reusable functions ([godbolt](https://www.godbolt.org/z/vcrjP8ceW)). - Added an experimental `fmt::writer` API that can be used for writing to different destinations such as files or strings (fmtlib/fmt#2354). For example ([godbolt](https://www.godbolt.org/z/rWoKfbP7e)): ```c++ #include <fmt/os.h> void write_text(fmt::writer w) { w.print("The answer is {}.", 42); } int main() { // Write to FILE. write_text(stdout); // Write to fmt::ostream. auto f = fmt::output_file("myfile"); write_text(f); // Write to std::string. auto sb = fmt::string_buffer(); write_text(sb); std::string s = sb.str(); } ``` - Added width and alignment support to the formatter of `std::error_code`. - Made `std::expected<void, E>` formattable (fmtlib/fmt#4145, fmtlib/fmt#4148). For example ([godbolt](https://www.godbolt.org/z/hrj5c6G86)): ```c++ fmt::print("{}", std::expected<void, int>()); ``` prints ``` expected() ``` Thanks @phprus. - Made `fmt::is_formattable<void>` SFINAE-friendly (fmtlib/fmt#4147). - Added support for `_BitInt` formatting when using clang (fmtlib/fmt#4007, fmtlib/fmt#4072, fmtlib/fmt#4140, fmtlib/fmt#4173, fmtlib/fmt#4176). For example ([godbolt](https://www.godbolt.org/z/KWjbWec5z)): ```c++ using int42 = _BitInt(42); fmt::print("{}", int42(100)); ``` Thanks @Arghnews. - Added the `n` specifier for tuples and pairs (fmtlib/fmt#4107). Thanks @someonewithpc. - Added support for tuple-like types to `fmt::join` (fmtlib/fmt#4226, fmtlib/fmt#4230). Thanks @phprus. - Made more types formattable at compile time (fmtlib/fmt#4127). Thanks @AnthonyVH. - Implemented a more efficient compile-time `fmt::formatted_size` (fmtlib/fmt#4102, fmtlib/fmt#4103). Thanks @phprus. - Fixed compile-time formatting of some string types (fmtlib/fmt#4065). Thanks @torshepherd. - Made compiled version of `fmt::format_to` work with `std::back_insert_iterator<std::vector<char>>` (fmtlib/fmt#4206, fmtlib/fmt#4211). Thanks @phprus. - Added a formatter for `std::reference_wrapper` (fmtlib/fmt#4163, fmtlib/fmt#4164). Thanks @yfeldblum and @phprus. - Added experimental padding support (glibc `strftime` extension) to `%m`, `%j` and `%Y` (fmtlib/fmt#4161). Thanks @KKhanhH. - Made microseconds formatted as `us` instead of `µs` if the Unicode support is disabled (fmtlib/fmt#4088). - Fixed an unreleased regression in transcoding of surrogate pairs (fmtlib/fmt#4094, fmtlib/fmt#4095). Thanks @phprus. - Made `fmt::appender` satisfy `std::output_iterator` concept (fmtlib/fmt#4092, fmtlib/fmt#4093). Thanks @phprus. - Made `std::iterator_traits<fmt::appender>` standard-conforming (fmtlib/fmt#4185). Thanks @CaseyCarter. - Made it easier to reuse `fmt::formatter<std::string_view>` for types with an implicit conversion to `std::string_view` (fmtlib/fmt#4036, fmtlib/fmt#4055). Thanks @Arghnews. - Made it possible to disable `<filesystem>` use via `FMT_CPP_LIB_FILESYSTEM` for compatibility with some video game console SDKs, e.g. Nintendo Switch SDK (fmtlib/fmt#4257, fmtlib/fmt#4258, fmtlib/fmt#4259). Thanks @W4RH4WK and @phprus. - Fixed compatibility with platforms that use 80-bit `long double` (fmtlib/fmt#4245, fmtlib/fmt#4246). Thanks @jsirpoma. - Added support for UTF-32 code units greater than `0xFFFF` in fill (fmtlib/fmt#4201). - Fixed handling of legacy encodings on Windows with GCC (fmtlib/fmt#4162). - Made `fmt::to_string` take `fmt::basic_memory_buffer` by const reference (fmtlib/fmt#4261, fmtlib/fmt#4262). Thanks @sascha-devel. - Added `fmt::dynamic_format_arg_store::size` (fmtlib/fmt#4270). Thanks @hannes-harnisch. - Removed the ability to control locale usage via an undocumented `FMT_STATIC_THOUSANDS_SEPARATOR` in favor of `FMT_USE_LOCALE`. - Renamed `FMT_EXCEPTIONS` to `FMT_USE_EXCEPTIONS` for consistency with other similar macros. - Improved include directory ordering to reduce the chance of including incorrect headers when using multiple versions of {fmt} (fmtlib/fmt#4116). Thanks @cdzhan. - Made it possible to compile a subset of {fmt} without the C++ runtime. - Improved documentation and README (fmtlib/fmt#4066, fmtlib/fmt#4117, fmtlib/fmt#4203, fmtlib/fmt#4235). Thanks @zyctree and @nikola-sh. - Improved the documentation generator (fmtlib/fmt#4110, fmtlib/fmt#4115). Thanks @rturrado. - Improved CI (fmtlib/fmt#4155, fmtlib/fmt#4151). Thanks @phprus. - Fixed various warnings and compilation issues (fmtlib/fmt#2708, fmtlib/fmt#4091, fmtlib/fmt#4109, fmtlib/fmt#4113, fmtlib/fmt#4125, fmtlib/fmt#4129, fmtlib/fmt#4130, fmtlib/fmt#4131, fmtlib/fmt#4132, fmtlib/fmt#4133, fmtlib/fmt#4144, fmtlib/fmt#4150, fmtlib/fmt#4158, fmtlib/fmt#4159, fmtlib/fmt#4160, fmtlib/fmt#4170, fmtlib/fmt#4177, fmtlib/fmt#4187, fmtlib/fmt#4188, fmtlib/fmt#4194, fmtlib/fmt#4200, fmtlib/fmt#4205, fmtlib/fmt#4207, fmtlib/fmt#4208, fmtlib/fmt#4210, fmtlib/fmt#4220, fmtlib/fmt#4231, fmtlib/fmt#4232, fmtlib/fmt#4233, fmtlib/fmt#4236, fmtlib/fmt#4267, fmtlib/fmt#4271). Thanks @torsten48, @Arghnews, @tinfoilboy, @aminya, @Ottani, @zeroomega, @c4v4, @kongy, @vinayyadav3016, @sergio-nsk, @phprus and @YexuanXiao. Signed-off-by: Adolf Belka <[email protected]> Signed-off-by: Michael Tremer <[email protected]>

kamrann added 5 commits September 20, 2024 11:20

Mark some in-class defined member functions as explicitly inline/cons…

e1789cf

…texpr, to avoid missing external symbols when using fmt module with shared build due to modules not defaulting to implicit inline.

Merge branch 'master' into pr/module-linkage-fixes

27f9966

Switch constexpr to inline for context::arg(string_view).

9ca41fa

NOTE: Looks as if basic_format_args::get(string_view) could probably be made constexpr instead, but sticking with minimal change approach.

Work around apparent non-conformance of older MSVC compilers.

a67e251

Switch format_int::str() from constexpr to inline to satisfy libstdc+…

f9e99e5

…+ std::string constexpr limitations.

vitaut requested changes Sep 22, 2024

View reviewed changes

kamrann added 4 commits September 23, 2024 14:45

Replace usages of macros for constexpr/inline with keywords.

afb5f21

Merge branch 'master' into pr/module-linkage-fixes

8765197

Fix for locations requiring C++14 constexpr.

076afe3

Further minor constexpr tweaks.

7226c0e

kamrann force-pushed the pr/module-linkage-fixes branch from e30fadd to 7226c0e Compare September 23, 2024 06:28

Apply clang format

d6c7200

vitaut approved these changes Sep 26, 2024

View reviewed changes

vitaut merged commit 96dca56 into fmtlib:master Sep 26, 2024
45 checks passed

kamrann deleted the pr/module-linkage-fixes branch September 27, 2024 07:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Module linkage fixes for shared build #4169

Module linkage fixes for shared build #4169

kamrann commented Sep 20, 2024

kamrann commented Sep 20, 2024

vitaut commented Sep 21, 2024

kamrann commented Sep 22, 2024

vitaut Sep 22, 2024

kamrann Sep 23, 2024

vitaut Sep 22, 2024

vitaut Sep 22, 2024

vitaut Sep 22, 2024

kamrann Sep 23, 2024

vitaut Sep 22, 2024

kamrann Sep 23, 2024

vitaut commented Sep 25, 2024

vitaut commented Sep 26, 2024

Module linkage fixes for shared build #4169

Module linkage fixes for shared build #4169

Conversation

kamrann commented Sep 20, 2024

kamrann commented Sep 20, 2024

vitaut commented Sep 21, 2024

kamrann commented Sep 22, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vitaut commented Sep 25, 2024

vitaut commented Sep 26, 2024