Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible use-after-move in join() #4231

Closed
mattgodbolt opened this issue Nov 8, 2024 · 5 comments · Fixed by #4236
Closed

Possible use-after-move in join() #4231

mattgodbolt opened this issue Nov 8, 2024 · 5 comments · Fixed by #4236

Comments

@mattgodbolt
Copy link

mattgodbolt commented Nov 8, 2024

The code at:

fmt/include/fmt/ranges.h

Lines 661 to 666 in 720da57

auto format(view_ref& value, FormatContext& ctx) const
-> decltype(ctx.out()) {
auto it = std::forward<view_ref>(value).begin;
auto out = ctx.out();
if (it == value.end) return out;
out = value_formatter_.format(*it, ctx);

Does a

auto it = std::forward<view_ref>(value).begin;

and later accesses value.end and value.sep.

In the case the iterator is not copy constructible then the view_ref is an rvalue-reference:

  using view_ref = conditional_t<std::is_copy_constructible<It>::value,
                                 const join_view<It, Sentinel, Char>&,
                                 join_view<It, Sentinel, Char>&&>;

which effectively turns the forward<> into a move.

clang-tidy warns on the accesses of end and sep as they are accesses on a moved-from object.

I'm not sure that this is really a problem in practice but it seems like something that should be avoided. I can't see an easy way to do so though. I'm not sure we can cast just the iterator easily? Taking a copy of sep works but for the same reasons as above we can't copy end out without a move either.

@mattgodbolt
Copy link
Author

mattgodbolt commented Nov 8, 2024

I'm a bit out of my depth but maybe something like:

    auto value_copy = std::forward<view_ref>(value);
    auto it = std::move(value_copy.begin);
    // rest of the code uses value_copy

though this forces a copy even in the good case.

@mattgodbolt
Copy link
Author

@Arghnews
Copy link
Contributor

Arghnews commented Nov 9, 2024

I had a look at this.

I believe this specialization is only used for exactly fmt:join_views.

If you try and pass an lvalue fmt::join_view, you hit a static assert saying we can only pass rvalue views (https://compiler-explorer.com/z/9jjPodzzo)

So now we can be sure that this code path only deals with receiving fmt::joins that are rvalues.

This format function is called from here, line 2224:

fmt/include/fmt/base.h

Lines 2214 to 2225 in 720da57

// Formats an argument of a custom type, such as a user-defined class.
template <typename T, typename Formatter>
static void format_custom(void* arg, parse_context<char_type>& parse_ctx,
Context& ctx) {
auto f = Formatter();
parse_ctx.advance_to(f.parse(parse_ctx));
using qualified_type =
conditional_t<has_formatter<const T, char_type>(), const T, T>;
// format must be const for compatibility with std::format and compilation.
const auto& cf = f;
ctx.advance_to(cf.format(*static_cast<qualified_type*>(arg), ctx));
}

qualified_type on line 2220 in that block is always non const (for a fmt::join_view). See:
https://compiler-explorer.com/z/cK3h3EzeP

This means that line 2224's first argument passed to format is:
*static_cast<qualified_type*>(arg) which will always deduce to fmt::join_view&

 

A minimal fix is to change fmt::join_view::format to only take lvalues, since as it stands, that's all it will ever receive, and change it to it&:

  template <typename FormatContext>
  auto format(join_view<It, Sentinel, Char>& value, FormatContext& ctx) const
      -> decltype(ctx.out()) {
    auto& it = value.begin;

This also saves a copy/move since we now take it by reference. The reference being into the original rvalue argument that is passed in the println call.

 

Note this code was originally put in in 10508a30ecd91e5d09a27e4c6c0a01a89fd4edc7 and changed to std::forward in d2473b7b73c0af2a3ed34c99e50ace0a1040581a

If I've missed anything or my assumptions or wrong, please point it out

 

I can put in a PR for this if vitaut agrees this is a good change to make
@vitaut

@vitaut
Copy link
Contributor

vitaut commented Nov 10, 2024

Thanks @mattgodbolt for reporting and @Arghnews for investigating the issue.

A minimal fix is to change fmt::join_view::format to only take lvalues, since as it stands, that's all it will ever receive, and change it to it&

I think this is a correct fix.

I can put in a PR for this if vitaut agrees this is a good change to make

Please do.

@mattgodbolt
Copy link
Author

Thank you both!

Arghnews added a commit to Arghnews/fmt that referenced this issue Nov 12, 2024
Should fix fmtlib#4231
Could instead use:
<!std::is_copy_constructible_v<It>, It&, It>;
Arghnews added a commit to Arghnews/fmt that referenced this issue Nov 14, 2024
vitaut pushed a commit that referenced this issue Nov 14, 2024
mtremer pushed a commit to ipfire/ipfire-2.x that referenced this issue Feb 22, 2025
- Update from version 11.0.2 to 11.1.3
- Update of rootfile
- Changelog
    11.1.3
	- Fixed compilation on GCC 9.4 (fmtlib/fmt#4313).
	- Worked around an internal compiler error when using C++20 modules with GCC
	  14.2 and earlier (fmtlib/fmt#4295).
	- Worked around a bug in GCC 6 (fmtlib/fmt#4318).
	- Fixed an issue caused by instantiating `formatter<const T>`
	  (fmtlib/fmt#4303,
	  fmtlib/fmt#4325). Thanks @timsong-cpp.
	- Fixed formatting into `std::ostreambuf_iterator` when using format string
	  compilation (fmtlib/fmt#4309,
	  fmtlib/fmt#4312). Thanks @phprus.
	- Restored a constraint on the map formatter so that it correctly reports as
	  unformattable when the element is (fmtlib/fmt#4326).
	  Thanks @timsong-cpp.
	- Reduced the size of format specs (fmtlib/fmt#4298).
	- Readded `args()` to `fmt::format_context`
	  (fmtlib/fmt#4307,
	  fmtlib/fmt#4310). Thanks @Erroneous1.
	- Fixed a bogus MSVC warning (fmtlib/fmt#4314,
	  fmtlib/fmt#4322). Thanks @ZehMatt.
	- Fixed a pedantic mode error in the CMake config
	  (fmtlib/fmt#4327). Thanks @rlalik.
    11.1.2
	- Fixed ABI compatibility with earlier 11.x versions
	  (fmtlib/fmt#4292).
	- Added `wchar_t` support to the `std::bitset` formatter
	  (fmtlib/fmt#4285,
	  fmtlib/fmt#4286,
	  fmtlib/fmt#4289,
	  fmtlib/fmt#4290). Thanks @phprus.
	- Prefixed CMake components with `fmt-` to simplify usage of {fmt} via
	  `add_subdirectory` (fmtlib/fmt#4283).
	- Updated docs for meson (fmtlib/fmt#4291).
	  Thanks @trim21.
	- Fixed a compilation error in chrono on nvcc
	  (fmtlib/fmt#4297,
	  fmtlib/fmt#4301). Thanks @breyerml.
	- Fixed various warnings
	  (fmtlib/fmt#4288,
	  fmtlib/fmt#4299). Thanks @GamesTrap and @edo9300.
    11.1.1
	- Fixed ABI compatibility with earlier 11.x versions
	  (fmtlib/fmt#4278).
	- Defined CMake components (`core` and `doc`) to allow docs to be installed
	  separately (fmtlib/fmt#4276).
	  Thanks @carlsmedstad.
    11.1.0
	- Improved C++20 module support
	  (fmtlib/fmt#4081,
	  fmtlib/fmt#4083,
	  fmtlib/fmt#4084,
	  fmtlib/fmt#4152,
	  fmtlib/fmt#4153,
	  fmtlib/fmt#4169,
	  fmtlib/fmt#4190,
	  fmtlib/fmt#4234,
	  fmtlib/fmt#4239).
	  Thanks @kamrann and @Arghnews.
	- Reduced debug (unoptimized) binary code size and the number of template
	  instantiations when passing formatting arguments. For example, unoptimized
	  binary code size for `fmt::print("{}", 42)` was reduced by ~40% on GCC and
	  ~60% on clang (x86-64).
	  GCC:
	  - Before: 161 instructions of which 105 are in reusable functions
	    ([godbolt](https://www.godbolt.org/z/s9bGoo4ze)).
	  - After: 116 instructions of which 60 are in reusable functions
	    ([godbolt](https://www.godbolt.org/z/r7GGGxMs6)).
	  Clang:
	  - Before: 310 instructions of which 251 are in reusable functions
	    ([godbolt](https://www.godbolt.org/z/Ts88b7M9o)).
	  - After: 194 instructions of which 135 are in reusable functions
	    ([godbolt](https://www.godbolt.org/z/vcrjP8ceW)).
	- Added an experimental `fmt::writer` API that can be used for writing to
	  different destinations such as files or strings
	  (fmtlib/fmt#2354).
	  For example ([godbolt](https://www.godbolt.org/z/rWoKfbP7e)):
		  ```c++
		  #include <fmt/os.h>
		  void write_text(fmt::writer w) {
		    w.print("The answer is {}.", 42);
		  }
		  int main() {
		    // Write to FILE.
		    write_text(stdout);
		    // Write to fmt::ostream.
		    auto f = fmt::output_file("myfile");
		    write_text(f);
		    // Write to std::string.
		    auto sb = fmt::string_buffer();
		    write_text(sb);
		    std::string s = sb.str();
		  }
		  ```
	- Added width and alignment support to the formatter of `std::error_code`.
	- Made `std::expected<void, E>` formattable
	  (fmtlib/fmt#4145,
	  fmtlib/fmt#4148).
	  For example ([godbolt](https://www.godbolt.org/z/hrj5c6G86)):
		  ```c++
		  fmt::print("{}", std::expected<void, int>());
		  ```
		  prints
		  ```
		  expected()
		  ```
	  Thanks @phprus.
	- Made `fmt::is_formattable<void>` SFINAE-friendly
	  (fmtlib/fmt#4147).
	- Added support for `_BitInt` formatting when using clang
	  (fmtlib/fmt#4007,
	  fmtlib/fmt#4072,
	  fmtlib/fmt#4140,
	  fmtlib/fmt#4173,
	  fmtlib/fmt#4176).
	  For example ([godbolt](https://www.godbolt.org/z/KWjbWec5z)):
		  ```c++
		  using int42 = _BitInt(42);
		  fmt::print("{}", int42(100));
		  ```
	  Thanks @Arghnews.
	- Added the `n` specifier for tuples and pairs
	  (fmtlib/fmt#4107). Thanks @someonewithpc.
	- Added support for tuple-like types to `fmt::join`
	  (fmtlib/fmt#4226,
	  fmtlib/fmt#4230). Thanks @phprus.
	- Made more types formattable at compile time
	  (fmtlib/fmt#4127). Thanks @AnthonyVH.
	- Implemented a more efficient compile-time `fmt::formatted_size`
	  (fmtlib/fmt#4102,
	  fmtlib/fmt#4103). Thanks @phprus.
	- Fixed compile-time formatting of some string types
	  (fmtlib/fmt#4065). Thanks @torshepherd.
	- Made compiled version of `fmt::format_to` work with
	  `std::back_insert_iterator<std::vector<char>>`
	  (fmtlib/fmt#4206,
	  fmtlib/fmt#4211). Thanks @phprus.
	- Added a formatter for `std::reference_wrapper`
	  (fmtlib/fmt#4163,
	  fmtlib/fmt#4164). Thanks @yfeldblum and @phprus.
	- Added experimental padding support (glibc `strftime` extension) to `%m`, `%j`
	  and `%Y` (fmtlib/fmt#4161). Thanks @KKhanhH.
	- Made microseconds formatted as `us` instead of `µs` if the Unicode support is
	  disabled (fmtlib/fmt#4088).
	- Fixed an unreleased regression in transcoding of surrogate pairs
	  (fmtlib/fmt#4094,
	  fmtlib/fmt#4095). Thanks @phprus.
	- Made `fmt::appender` satisfy `std::output_iterator` concept
	  (fmtlib/fmt#4092,
	  fmtlib/fmt#4093). Thanks @phprus.
	- Made `std::iterator_traits<fmt::appender>` standard-conforming
	  (fmtlib/fmt#4185). Thanks @CaseyCarter.
	- Made it easier to reuse `fmt::formatter<std::string_view>` for types with
	  an implicit conversion to `std::string_view`
	  (fmtlib/fmt#4036,
	  fmtlib/fmt#4055). Thanks @Arghnews.
	- Made it possible to disable `<filesystem>` use via `FMT_CPP_LIB_FILESYSTEM`
	  for compatibility with some video game console SDKs, e.g. Nintendo Switch SDK
	  (fmtlib/fmt#4257,
	  fmtlib/fmt#4258,
	  fmtlib/fmt#4259). Thanks @W4RH4WK and @phprus.
	- Fixed compatibility with platforms that use 80-bit `long double`
	  (fmtlib/fmt#4245,
	  fmtlib/fmt#4246). Thanks @jsirpoma.
	- Added support for UTF-32 code units greater than `0xFFFF` in fill
	  (fmtlib/fmt#4201).
	- Fixed handling of legacy encodings on Windows with GCC
	  (fmtlib/fmt#4162).
	- Made `fmt::to_string` take `fmt::basic_memory_buffer` by const reference
	  (fmtlib/fmt#4261,
	  fmtlib/fmt#4262). Thanks @sascha-devel.
	- Added `fmt::dynamic_format_arg_store::size`
	  (fmtlib/fmt#4270). Thanks @hannes-harnisch.
	- Removed the ability to control locale usage via an undocumented
	  `FMT_STATIC_THOUSANDS_SEPARATOR` in favor of `FMT_USE_LOCALE`.
	- Renamed `FMT_EXCEPTIONS` to `FMT_USE_EXCEPTIONS` for consistency with other
	  similar macros.
	- Improved include directory ordering to reduce the chance of including
	  incorrect headers when using multiple versions of {fmt}
	  (fmtlib/fmt#4116). Thanks @cdzhan.
	- Made it possible to compile a subset of {fmt} without the C++ runtime.
	- Improved documentation and README
	  (fmtlib/fmt#4066,
	  fmtlib/fmt#4117,
	  fmtlib/fmt#4203,
	  fmtlib/fmt#4235). Thanks @zyctree and @nikola-sh.
	- Improved the documentation generator (fmtlib/fmt#4110,
	  fmtlib/fmt#4115). Thanks @rturrado.
	- Improved CI (fmtlib/fmt#4155,
	  fmtlib/fmt#4151). Thanks @phprus.
	- Fixed various warnings and compilation issues
	  (fmtlib/fmt#2708,
	  fmtlib/fmt#4091,
	  fmtlib/fmt#4109,
	  fmtlib/fmt#4113,
	  fmtlib/fmt#4125,
	  fmtlib/fmt#4129,
	  fmtlib/fmt#4130,
	  fmtlib/fmt#4131,
	  fmtlib/fmt#4132,
	  fmtlib/fmt#4133,
	  fmtlib/fmt#4144,
	  fmtlib/fmt#4150,
	  fmtlib/fmt#4158,
	  fmtlib/fmt#4159,
	  fmtlib/fmt#4160,
	  fmtlib/fmt#4170,
	  fmtlib/fmt#4177,
	  fmtlib/fmt#4187,
	  fmtlib/fmt#4188,
	  fmtlib/fmt#4194,
	  fmtlib/fmt#4200,
	  fmtlib/fmt#4205,
	  fmtlib/fmt#4207,
	  fmtlib/fmt#4208,
	  fmtlib/fmt#4210,
	  fmtlib/fmt#4220,
	  fmtlib/fmt#4231,
	  fmtlib/fmt#4232,
	  fmtlib/fmt#4233,
	  fmtlib/fmt#4236,
	  fmtlib/fmt#4267,
	  fmtlib/fmt#4271).
	  Thanks @torsten48, @Arghnews, @tinfoilboy, @aminya, @Ottani, @zeroomega,
	  @c4v4, @kongy, @vinayyadav3016, @sergio-nsk, @phprus and @YexuanXiao.

Signed-off-by: Adolf Belka <[email protected]>
Signed-off-by: Michael Tremer <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants