Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

write_range_entry fails writing non-utf8 string #3038

Closed
stevenzzzz opened this issue Aug 16, 2022 · 7 comments
Closed

write_range_entry fails writing non-utf8 string #3038

stevenzzzz opened this issue Aug 16, 2022 · 7 comments

Comments

@stevenzzzz
Copy link

there seems to be a bug in "Inline auto write_range_entry(OutputIt out, const std::T& t)" would cause ASAN error if the string is a invalid-utf8 string.

using example: "\xf0\x28", it will cause an ASAN error.
@vitaut
Copy link
Contributor

vitaut commented Aug 16, 2022

Could you provide a self-contained repro (ideally on godbolt)?

@stevenzzzz
Copy link
Author

hmm, not sure if godbolt vm supports the asan sanitizer.

Could you try this?

#include <fmt/core.h>
#include <fmt/format.h>
#include <unordered_map>

int main(){

std::string invalid_utf8="\xf0\x28";

fmt::print("Foo Map: {}", invalid_utf8);

}

I actually got here from Envoy, which uses spdlog --> libfmt.

I have synced to 9.0.0, and "bazel test" on logging the afore mentioned string would end up with:

external/com_github_fmtlib_fmt/include/fmt/core.h:426: assertion failed: negative valuelibc++abi: terminating [2022-08-16 17:57:43.537][16][critical][backtrace] [./source/server/backtrace.h:104] Caught Aborted, suspect faulting address 0xfffe00000010
[2022-08-16 17:57:43.538][16][critical][backtrace] [./source/server/backtrace.h:91] Backtrace (use tools/stack_decode.py to get line numbers):
[2022-08-16 17:57:43.538][16][critical][backtrace] [./source/server/backtrace.h:92] Envoy version: 0/1.24.0-dev/test/DEBUG/BoringSSL
[2022-08-16 17:57:43.551][16][critical][backtrace] [./source/server/backtrace.h:96] #0: Envoy::SignalAction::sigHandler() [0x1517d9c]
[2022-08-16 17:57:43.551][16][critical][backtrace] [./source/server/backtrace.h:96] #1: _restore_rt [0x7f6475ecc420]
[2022-08-16 17:57:43.563][16][critical][backtrace] [./source/server/backtrace.h:96] #2: fmt::v8::detail::to_unsigned<>() [0xf4dbd4]
[2022-08-16 17:57:43.573][16][critical][backtrace] [./source/server/backtrace.h:96] #3: fmt::v8::detail::find_escape() [0xf6f2a7]
[2022-08-16 17:57:43.584][16][critical][backtrace] [./source/server/backtrace.h:96] #4: fmt::v8::detail::write_range_entry<>() [0xf6e757]
[2022-08-16 17:57:43.595][16][critical][backtrace] [./source/server/backtrace.h:96] #5: fmt::v8::detail::write_range_entry<>() [0xf6e65b]
[2022-08-16 17:57:43.606][16][critical][backtrace] [./source/server/backtrace.h:96] #6: fmt::v8::formatter<>::format<>() [0xf6e519]
[2022-08-16 17:57:43.617][16][critical][backtrace] [./source/server/backtrace.h:96] #7: fmt::v8::detail::value<>::format_custom_arg<>() [0xf6e367]
[2022-08-16 17:57:43.628][16][critical][backtrace] [./source/server/backtrace.h:96] #8: fmt::v8::basic_format_arg<>::handle::format() [0xf60a29]
[2022-08-16 17:57:43.639][16][critical][backtrace] [./source/server/backtrace.h:96] #9: fmt::v8::detail::default_arg_formatter<>::operator()() [0xf4d2ca]
[2022-08-16 17:57:43.650][16][critical][backtrace] [./source/server/backtrace.h:96] #10: fmt::v8::detail::parse_replacement_field<>() [0xf60fab]
[2022-08-16 17:57:43.661][16][critical][backtrace] [./source/server/backtrace.h:96] #11: fmt::v8::detail::vformat_to<>() [0xf4bc63]
[2022-08-16 17:57:43.672][16][critical][backtrace] [./source/server/backtrace.h:96] #12: spdlog::logger::log<>() [0xf4b22c]
[2022-08-16 17:57:43.683][16][critical][backtrace] [./source/server/backtrace.h:96] #13: spdlog::logger::log<>() [0xf401ff]

@vitaut
Copy link
Contributor

vitaut commented Aug 16, 2022

Did you mean to use map in your example? Strings don't go through ranges.

@stevenzzzz
Copy link
Author

Sorry, here is some more context: envoyproxy/envoy#22609

yes, it's actually when formatting a map, the detail::write_range_entry is called on every item, and the item.second value in our case contains an invalid string, like the one I pasted earlier.

that would cause an ASAN error.

I saw that in the most recent version there is quite a bit refactoring around detail::write_range_entry, I am not sure if that solved the issue, but I have synced to 9.0.0, and seems it didn't.

@dyfrgi
Copy link

dyfrgi commented Aug 17, 2022

Here is a repro on godbolt, showing the crash. I made this pretty similarly to the way spdlog uses fmt, thus the use of fmt::detail::vformat_to and fmt::make_format_args. This triggers the assert in to_unsigned when built without -DNDEBUG.

https://godbolt.org/z/zx4d4KTvE

@phprus phprus mentioned this issue Aug 18, 2022
Merged
@phprus
Copy link
Contributor

phprus commented Aug 18, 2022

@stevenzzzz @dyfrgi
Please check PR #3044

@vitaut
Copy link
Contributor

vitaut commented Aug 24, 2022

Fixed in #3044 (thanks @phprus).

@vitaut vitaut closed this as completed Aug 24, 2022
@phprus phprus mentioned this issue Aug 24, 2022
Closed
mtremer referenced this issue in ipfire/ipfire-2.x Nov 28, 2022
@pmu-ipf
fmt: Update to version 9.1.0
90d43e5 
- Update from version 9.0.0 to 9.1.0
- Update of rootfile
- Changelog
    9.1.0 - 2022-08-27
	* ``fmt::formatted_size`` now works at compile time
		  `#3026 <https://github.com/fmtlib/fmt/pull/3026>`_
			  For example (`godbolt <https://godbolt.org/z/1MW5rMdf8>`__):
			   .. code:: c++
			     #include <fmt/compile.h>
			     int main() {
			       using namespace fmt::literals;
			       constexpr size_t n = fmt::formatted_size("{}"_cf, 42);
			       fmt::print("{}\n", n); // prints 2
			     }
	* Fixed handling of invalid UTF-8
		  `#3038 <https://github.com/fmtlib/fmt/pull/3038>`_,
		  `#3044 <https://github.com/fmtlib/fmt/pull/3044>`_,
		  `#3056 <https://github.com/fmtlib/fmt/pull/3056>`_
	* Improved Unicode support in ``ostream`` overloads of ``print``
		  `#2994 <https://github.com/fmtlib/fmt/pull/2994>`_,
		  `#3001 <https://github.com/fmtlib/fmt/pull/3001>`_,
		  `#3025 <https://github.com/fmtlib/fmt/pull/3025>`_
	* Fixed handling of the sign specifier in localized formatting on systems with
	   32-bit ``wchar_t``
		  `#3041 <https://github.com/fmtlib/fmt/issues/3041>`_).
	* Added support for wide streams to ``fmt::streamed``
		  `#2994 <https://github.com/fmtlib/fmt/pull/2994>`_
	* Added the ``n`` specifier that disables the output of delimiters when
	   formatting ranges
		  `#2981 <https://github.com/fmtlib/fmt/pull/2981>`_,
		  `#2983 <https://github.com/fmtlib/fmt/pull/2983>`_
			  For example (`godbolt <https://godbolt.org/z/roKqGdj8c>`__):
			   .. code:: c++
			     #include <fmt/ranges.h>
			     #include <vector>
			     int main() {
			       auto v = std::vector{1, 2, 3};
			       fmt::print("{:n}\n", v); // prints 1, 2, 3
			     }
	* Worked around problematic ``std::string_view`` constructors introduced in C++23
		  `#3030 <https://github.com/fmtlib/fmt/issues/3030>`_,
		  `#3050 <https://github.com/fmtlib/fmt/issues/3050>`_
	* Improve handling (exclusion) of recursive ranges
		  `#2968 <https://github.com/fmtlib/fmt/issues/2968>`_,
		  `#2974 <https://github.com/fmtlib/fmt/pull/2974>`_
	* Improved error reporting in format string compilation
		  `#3055 <https://github.com/fmtlib/fmt/issues/3055>`_
	* Improved the implementation of
		  `Dragonbox <https://github.com/jk-jeon/dragonbox>`_, the algorithm used for
		   the default floating-point formatting
		  `#2984 <https://github.com/fmtlib/fmt/pull/2984>`_
	* Fixed issues with floating-point formatting on exotic platforms.
	* Improved the implementation of chrono formatting
		  `#3010 <https://github.com/fmtlib/fmt/pull/3010>`_
	* Improved documentation
		  `#2966 <https://github.com/fmtlib/fmt/pull/2966>`_,
		  `#3009 <https://github.com/fmtlib/fmt/pull/3009>`_,
		  `#3020 <https://github.com/fmtlib/fmt/issues/3020>`_,
		  `#3037 <https://github.com/fmtlib/fmt/pull/3037>`_
	* Improved build configuration
		  `#2991 <https://github.com/fmtlib/fmt/pull/2991>`_,
		  `#2995 <https://github.com/fmtlib/fmt/pull/2995>`_,
		  `#3004 <https://github.com/fmtlib/fmt/issues/3004>`_,
		  `#3007 <https://github.com/fmtlib/fmt/pull/3007>`_,
		  `#3040 <https://github.com/fmtlib/fmt/pull/3040>`_
	* Fixed various warnings and compilation issues
		  `#2969 <https://github.com/fmtlib/fmt/issues/2969>`_,
		  `#2971 <https://github.com/fmtlib/fmt/pull/2971>`_,
		  `#2975 <https://github.com/fmtlib/fmt/issues/2975>`_,
		  `#2982 <https://github.com/fmtlib/fmt/pull/2982>`_,
		  `#2985 <https://github.com/fmtlib/fmt/pull/2985>`_,
		  `#2988 <https://github.com/fmtlib/fmt/issues/2988>`_,
		  `#3000 <https://github.com/fmtlib/fmt/issues/3000>`_,
		  `#3006 <https://github.com/fmtlib/fmt/issues/3006>`_,
		  `#3014 <https://github.com/fmtlib/fmt/issues/3014>`_,
		  `#3015 <https://github.com/fmtlib/fmt/issues/3015>`_,
		  `#3021 <https://github.com/fmtlib/fmt/pull/3021>`_,
		  `#3023 <https://github.com/fmtlib/fmt/issues/3023>`_,
		  `#3024 <https://github.com/fmtlib/fmt/pull/3024>`_,
		  `#3029 <https://github.com/fmtlib/fmt/pull/3029>`_,
		  `#3043 <https://github.com/fmtlib/fmt/pull/3043>`_,
		  `#3052 <https://github.com/fmtlib/fmt/issues/3052>`_,
		  `#3053 <https://github.com/fmtlib/fmt/pull/3053>`_,
		  `#3054 <https://github.com/fmtlib/fmt/pull/3054>`_

Signed-off-by: Adolf Belka <[email protected]>
Reviewed-by: Michael Tremer <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@phprus @vitaut @dyfrgi @stevenzzzz