GCC 14 optimizer doesn't like std::filesystem::path formatting (Wstringop-overflow) #4125
Comments
|
I'm trying to upgrade my project to fmt 11 and noticed the same gcc warning |
|
Looks like a compiler bug |
|
I investigated it. The difference here is GCC14 does one more inline. You can mimic the GCC14 behavior in GCC13 by adding With always inlining |
|
So I inlined all those functions manually. Found it keeps warning the following code snippets. https://godbolt.org/z/Y3E5fh9G6 if (num_chars_left > 0 && num_chars_left < block_size) {
char buf[2 * block_size - 1] = {};
{
auto _begin = p;
auto _out = buf;
while (_begin < p + num_chars_left) *_out++ = *_begin++; // warns here.
}Full Code Snippet
#include <fmt/format.h>
namespace fmt {
inline namespace v11 {
namespace detail {
auto write_escaped_string(basic_appender<char> out,
const basic_string_view<char> str) {
*out++ = '"';
const auto str_data = str.data();
const auto str_size = str.size();
auto begin = str_data, end = str_data + str_size;
do {
find_escape_result<char> escape;
auto decode = [&](const char* buf_ptr, const char* ptr) -> const char* {
auto cp = uint32_t();
auto error = 0;
auto end = utf8_decode(buf_ptr, &cp, &error);
if (needs_escape(error ? invalid_code_point : cp)) {
auto sv = string_view(ptr, error ? 1 : to_unsigned(end - buf_ptr));
escape = {sv.begin(), sv.end(), cp};
return nullptr;
}
return error ? buf_ptr + 1 : end;
};
auto p = begin;
const size_t block_size = 4; // utf8_decode always reads blocks of 4 chars.
if (p <= end - block_size) {
for (; p <= end - block_size;) {
p = decode(p, p);
if (!p) goto escape_found;
}
}
// invariant:
// 1. p >= end - block_size + 1
// 2. end - p < block_size
if (const auto num_chars_left = end - p;
num_chars_left > 0 && num_chars_left < block_size) {
// invariant: num_chars_left < block_size
char buf[2 * block_size - 1] = {};
{
auto _begin = p;
auto _out = buf;
while (_begin < p + num_chars_left) *_out++ = *_begin++;
}
const char* buf_ptr = buf;
do {
auto end2 = decode(buf_ptr, p);
if (!end2) goto escape_found;
p += end2 - buf_ptr;
buf_ptr = end2;
} while (buf_ptr - buf < num_chars_left);
}
escape_found:
out = copy<char>(begin, escape.begin, out);
begin = escape.end;
if (!begin) break;
auto c = escape.cp;
switch (escape.cp) {
case '\n':
*out++ = '\\';
c = 'n';
break;
case '\r':
*out++ = '\\';
c = 'r';
break;
case '\t':
*out++ = '\\';
c = 't';
break;
case '"':
FMT_FALLTHROUGH;
case '\'':
FMT_FALLTHROUGH;
case '\\':
*out++ = '\\';
break;
default:
if (escape.cp < 0x100) {
out = write_codepoint<2, char>(out, 'x', escape.cp);
continue;
}
if (escape.cp < 0x10000) {
out = write_codepoint<4, char>(out, 'u', escape.cp);
continue;
}
if (escape.cp < 0x110000) {
out = write_codepoint<8, char>(out, 'U', escape.cp);
continue;
}
for (char escape_char : basic_string_view<char>(
escape.begin, to_unsigned(escape.end - escape.begin))) {
out = write_codepoint<2, char>(
out, 'x', static_cast<uint32_t>(escape_char) & 0xFF);
}
continue;
}
*out++ = c;
} while (begin != end);
*out++ = '"';
return out;
}
} // namespace detail
} // namespace v11
} // namespace fmt
|
|
I carefully reads through the logic and decide it should be a false positive. Maybe we will need a hack to suppress that compiler warning. |
|
It does look like false positive so might be worth reporting to gcc but I applied a workaround in 0379bf3. |
Yeah, but it's a temporary workaround. The same warning happens if |
msimberg
added a commit
to msimberg/pika
that referenced
this issue
Aug 26, 2024
…gs in GCC 13 CI configuration fmtlib/fmt#4125
msimberg
added a commit
to msimberg/pika
that referenced
this issue
Aug 27, 2024
…gs in GCC 13/14 CI configurations fmtlib/fmt#4125
If you have a better solution, a PR would be welcome. |
mtremer
pushed a commit
to ipfire/ipfire-2.x
that referenced
this issue
Feb 22, 2025
- Update from version 11.0.2 to 11.1.3
- Update of rootfile
- Changelog
11.1.3
- Fixed compilation on GCC 9.4 (fmtlib/fmt#4313).
- Worked around an internal compiler error when using C++20 modules with GCC
14.2 and earlier (fmtlib/fmt#4295).
- Worked around a bug in GCC 6 (fmtlib/fmt#4318).
- Fixed an issue caused by instantiating `formatter<const T>`
(fmtlib/fmt#4303,
fmtlib/fmt#4325). Thanks @timsong-cpp.
- Fixed formatting into `std::ostreambuf_iterator` when using format string
compilation (fmtlib/fmt#4309,
fmtlib/fmt#4312). Thanks @phprus.
- Restored a constraint on the map formatter so that it correctly reports as
unformattable when the element is (fmtlib/fmt#4326).
Thanks @timsong-cpp.
- Reduced the size of format specs (fmtlib/fmt#4298).
- Readded `args()` to `fmt::format_context`
(fmtlib/fmt#4307,
fmtlib/fmt#4310). Thanks @Erroneous1.
- Fixed a bogus MSVC warning (fmtlib/fmt#4314,
fmtlib/fmt#4322). Thanks @ZehMatt.
- Fixed a pedantic mode error in the CMake config
(fmtlib/fmt#4327). Thanks @rlalik.
11.1.2
- Fixed ABI compatibility with earlier 11.x versions
(fmtlib/fmt#4292).
- Added `wchar_t` support to the `std::bitset` formatter
(fmtlib/fmt#4285,
fmtlib/fmt#4286,
fmtlib/fmt#4289,
fmtlib/fmt#4290). Thanks @phprus.
- Prefixed CMake components with `fmt-` to simplify usage of {fmt} via
`add_subdirectory` (fmtlib/fmt#4283).
- Updated docs for meson (fmtlib/fmt#4291).
Thanks @trim21.
- Fixed a compilation error in chrono on nvcc
(fmtlib/fmt#4297,
fmtlib/fmt#4301). Thanks @breyerml.
- Fixed various warnings
(fmtlib/fmt#4288,
fmtlib/fmt#4299). Thanks @GamesTrap and @edo9300.
11.1.1
- Fixed ABI compatibility with earlier 11.x versions
(fmtlib/fmt#4278).
- Defined CMake components (`core` and `doc`) to allow docs to be installed
separately (fmtlib/fmt#4276).
Thanks @carlsmedstad.
11.1.0
- Improved C++20 module support
(fmtlib/fmt#4081,
fmtlib/fmt#4083,
fmtlib/fmt#4084,
fmtlib/fmt#4152,
fmtlib/fmt#4153,
fmtlib/fmt#4169,
fmtlib/fmt#4190,
fmtlib/fmt#4234,
fmtlib/fmt#4239).
Thanks @kamrann and @Arghnews.
- Reduced debug (unoptimized) binary code size and the number of template
instantiations when passing formatting arguments. For example, unoptimized
binary code size for `fmt::print("{}", 42)` was reduced by ~40% on GCC and
~60% on clang (x86-64).
GCC:
- Before: 161 instructions of which 105 are in reusable functions
([godbolt](https://www.godbolt.org/z/s9bGoo4ze)).
- After: 116 instructions of which 60 are in reusable functions
([godbolt](https://www.godbolt.org/z/r7GGGxMs6)).
Clang:
- Before: 310 instructions of which 251 are in reusable functions
([godbolt](https://www.godbolt.org/z/Ts88b7M9o)).
- After: 194 instructions of which 135 are in reusable functions
([godbolt](https://www.godbolt.org/z/vcrjP8ceW)).
- Added an experimental `fmt::writer` API that can be used for writing to
different destinations such as files or strings
(fmtlib/fmt#2354).
For example ([godbolt](https://www.godbolt.org/z/rWoKfbP7e)):
```c++
#include <fmt/os.h>
void write_text(fmt::writer w) {
w.print("The answer is {}.", 42);
}
int main() {
// Write to FILE.
write_text(stdout);
// Write to fmt::ostream.
auto f = fmt::output_file("myfile");
write_text(f);
// Write to std::string.
auto sb = fmt::string_buffer();
write_text(sb);
std::string s = sb.str();
}
```
- Added width and alignment support to the formatter of `std::error_code`.
- Made `std::expected<void, E>` formattable
(fmtlib/fmt#4145,
fmtlib/fmt#4148).
For example ([godbolt](https://www.godbolt.org/z/hrj5c6G86)):
```c++
fmt::print("{}", std::expected<void, int>());
```
prints
```
expected()
```
Thanks @phprus.
- Made `fmt::is_formattable<void>` SFINAE-friendly
(fmtlib/fmt#4147).
- Added support for `_BitInt` formatting when using clang
(fmtlib/fmt#4007,
fmtlib/fmt#4072,
fmtlib/fmt#4140,
fmtlib/fmt#4173,
fmtlib/fmt#4176).
For example ([godbolt](https://www.godbolt.org/z/KWjbWec5z)):
```c++
using int42 = _BitInt(42);
fmt::print("{}", int42(100));
```
Thanks @Arghnews.
- Added the `n` specifier for tuples and pairs
(fmtlib/fmt#4107). Thanks @someonewithpc.
- Added support for tuple-like types to `fmt::join`
(fmtlib/fmt#4226,
fmtlib/fmt#4230). Thanks @phprus.
- Made more types formattable at compile time
(fmtlib/fmt#4127). Thanks @AnthonyVH.
- Implemented a more efficient compile-time `fmt::formatted_size`
(fmtlib/fmt#4102,
fmtlib/fmt#4103). Thanks @phprus.
- Fixed compile-time formatting of some string types
(fmtlib/fmt#4065). Thanks @torshepherd.
- Made compiled version of `fmt::format_to` work with
`std::back_insert_iterator<std::vector<char>>`
(fmtlib/fmt#4206,
fmtlib/fmt#4211). Thanks @phprus.
- Added a formatter for `std::reference_wrapper`
(fmtlib/fmt#4163,
fmtlib/fmt#4164). Thanks @yfeldblum and @phprus.
- Added experimental padding support (glibc `strftime` extension) to `%m`, `%j`
and `%Y` (fmtlib/fmt#4161). Thanks @KKhanhH.
- Made microseconds formatted as `us` instead of `µs` if the Unicode support is
disabled (fmtlib/fmt#4088).
- Fixed an unreleased regression in transcoding of surrogate pairs
(fmtlib/fmt#4094,
fmtlib/fmt#4095). Thanks @phprus.
- Made `fmt::appender` satisfy `std::output_iterator` concept
(fmtlib/fmt#4092,
fmtlib/fmt#4093). Thanks @phprus.
- Made `std::iterator_traits<fmt::appender>` standard-conforming
(fmtlib/fmt#4185). Thanks @CaseyCarter.
- Made it easier to reuse `fmt::formatter<std::string_view>` for types with
an implicit conversion to `std::string_view`
(fmtlib/fmt#4036,
fmtlib/fmt#4055). Thanks @Arghnews.
- Made it possible to disable `<filesystem>` use via `FMT_CPP_LIB_FILESYSTEM`
for compatibility with some video game console SDKs, e.g. Nintendo Switch SDK
(fmtlib/fmt#4257,
fmtlib/fmt#4258,
fmtlib/fmt#4259). Thanks @W4RH4WK and @phprus.
- Fixed compatibility with platforms that use 80-bit `long double`
(fmtlib/fmt#4245,
fmtlib/fmt#4246). Thanks @jsirpoma.
- Added support for UTF-32 code units greater than `0xFFFF` in fill
(fmtlib/fmt#4201).
- Fixed handling of legacy encodings on Windows with GCC
(fmtlib/fmt#4162).
- Made `fmt::to_string` take `fmt::basic_memory_buffer` by const reference
(fmtlib/fmt#4261,
fmtlib/fmt#4262). Thanks @sascha-devel.
- Added `fmt::dynamic_format_arg_store::size`
(fmtlib/fmt#4270). Thanks @hannes-harnisch.
- Removed the ability to control locale usage via an undocumented
`FMT_STATIC_THOUSANDS_SEPARATOR` in favor of `FMT_USE_LOCALE`.
- Renamed `FMT_EXCEPTIONS` to `FMT_USE_EXCEPTIONS` for consistency with other
similar macros.
- Improved include directory ordering to reduce the chance of including
incorrect headers when using multiple versions of {fmt}
(fmtlib/fmt#4116). Thanks @cdzhan.
- Made it possible to compile a subset of {fmt} without the C++ runtime.
- Improved documentation and README
(fmtlib/fmt#4066,
fmtlib/fmt#4117,
fmtlib/fmt#4203,
fmtlib/fmt#4235). Thanks @zyctree and @nikola-sh.
- Improved the documentation generator (fmtlib/fmt#4110,
fmtlib/fmt#4115). Thanks @rturrado.
- Improved CI (fmtlib/fmt#4155,
fmtlib/fmt#4151). Thanks @phprus.
- Fixed various warnings and compilation issues
(fmtlib/fmt#2708,
fmtlib/fmt#4091,
fmtlib/fmt#4109,
fmtlib/fmt#4113,
fmtlib/fmt#4125,
fmtlib/fmt#4129,
fmtlib/fmt#4130,
fmtlib/fmt#4131,
fmtlib/fmt#4132,
fmtlib/fmt#4133,
fmtlib/fmt#4144,
fmtlib/fmt#4150,
fmtlib/fmt#4158,
fmtlib/fmt#4159,
fmtlib/fmt#4160,
fmtlib/fmt#4170,
fmtlib/fmt#4177,
fmtlib/fmt#4187,
fmtlib/fmt#4188,
fmtlib/fmt#4194,
fmtlib/fmt#4200,
fmtlib/fmt#4205,
fmtlib/fmt#4207,
fmtlib/fmt#4208,
fmtlib/fmt#4210,
fmtlib/fmt#4220,
fmtlib/fmt#4231,
fmtlib/fmt#4232,
fmtlib/fmt#4233,
fmtlib/fmt#4236,
fmtlib/fmt#4267,
fmtlib/fmt#4271).
Thanks @torsten48, @Arghnews, @tinfoilboy, @aminya, @Ottani, @zeroomega,
@c4v4, @kongy, @vinayyadav3016, @sergio-nsk, @phprus and @YexuanXiao.
Signed-off-by: Adolf Belka <[email protected]>
Signed-off-by: Michael Tremer <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Starting with 11.0.2, when using GCC 14, particularly with link time optimization but also reproducible in some cases without, it seems that formatting std::filesystem::path (i.e.
fmt::to_string(std::filesystem::path {...})) produces stringop-overflow warnings. While this can be reproduced by just callingfmt::to_string<std::filesystem::path>in exploring I found that explicitly instantiating two particular templates seems to be enough to trigger it (one or the other is not enough). Using O3 optimizations seems to be required for this to trigger.https://godbolt.org/z/Gsvoqz71e
I've traced the change causing this to commit f29a7e7. Specifically just the removal of the copy overload there. Reverting that overload removal is one possible workaround.
Despite the functions that are instantiated there, the actual function in question seems to be
for_each_codepoint. Increasing the size of buf by up to 8 characters eliminates the warnings. I've also had success in getting rid of them by storing the start and end pointers for the string instead and removing theif (s.size() >= block_size)check to simplify the flow (might have needed an assume somewhere in there too). Ultimately this feels like a false positive to me, and I can report to GCC if agreed.I've tried various combinations of [[assume()]] without changing the code structure with no avail, so GCC seems particularly stubborn about this one.
The text was updated successfully, but these errors were encountered: