-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Improve debug checks (prelude to STL Hardening) #5270
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve debug checks (prelude to STL Hardening) #5270
Conversation
…d will be removed.
This identifies that we're dealing with array<T, 0>, and that the call is permanently bogus, following the example of "array<T, 0>::front() invalid".
This is consistent with vector. mdspan's message was very opaque; starting with "mdspan subscript out of range" provides clarity. Additionally, "I" was previously used without being defined; I've replaced it with its definition from /2. ranges::view_interface now refers to itself with qualification.
deque now mentions whether pop_front() or pop_back() was called. optional now mentions whether operator*() or operator->() was called. ranges::view_interface now refers to itself with qualification.
… subspan<Offset, Count>().
…uffix(). Spell the function names literally for improved searchability. Mention the short version of the class name too. Adjust "longer" to "larger" for style.
This splits "expected stores an error, not a value" into: "operator->() called on a std::expected that stores an error, not a value" "operator*() called on a std::expected that stores an error, not a value" And changes "expected stores a value, not an error" into: "error() called on a std::expected that stores a value, not an error" This mentions which member function is being called. Because "expected" doesn't really sound like a noun, I've added "std::" qualification and made the phrases less terse.
These weren't CDL-sensitive at all, and were properly guarded with `IDL != 0`.
…ynamic width/precision. `<format>` isn't CDL-sensitive. Negative dynamic width/precision are required to throw format_error; we don't use "death tests" for exceptions. For clarity and consistency, add positive and negative test cases for both. This may slightly duplicate existing coverage, but negative dynamic precision wasn't being properly tested AFAICT.
VSO-847348 "c1xx ICEs deducing array size inside a temploid" was fixed at some point during the last 6 years (verified with prod/fe x86chk).
…TL_ASSERT` In `<memory_resource>`, we always call `_Prepare_oversized`, but the `_STL_REPORT_ERROR` was guarded by `#ifdef _DEBUG`. Convert this to `_STL_ASSERT`, which is consistent with the `_STL_ASSERT` that appears immediately below. In `<numeric>` lcm(), extract `const bool _Overflow` so the `_STL_VERIFY` has no side effects.
In `<mdspan>`, add `return 1;` in `stride()` to support upcoming changes for continue-on-error. In `<regex>`, unconditionally `return false;` for the same reason.
This doesn't change behavior (it remains a debug-only check), it just removes an escape hatch that was never used.
We can always extract _Newsize for clarity.
…ERIFY. This is especially noticeable when _STL_VERIFY already follows.
We'll continue right into an infinite loop, but at least we won't fall off the end of a non-void function.
…through to seq_cst. We were already doing this in other places.
We don't really care about IDL exactly equal to 1, but these should clearly be checked in IDL=1 release, consistent with the following checks.
These conditions are already guaranteed by `duration` and `ratio`. They don't need to be checked for users, especially at runtime.
It's silly to assert that _Myptr is non-null when we're about to dereference it (or return it for imminent dereferencing). We purged this category of asserts from the STL aeons ago, but must have missed auto_ptr. The idea is that null dereferences are just as obvious in the debugger. Note that this makes the code consistent with unique_ptr and shared_ptr.
First, a cleanup: _Verify_offset does something for IDL >= 1 only, so we can guard the callsite in operator+=. In operator-=, the pathological "integer overflow" check was being performed unconditionally. Guard it for IDL >= 1, and the following _Verify_offset call.
See: https://godbolt.org/z/b95sjPPGo And: https://github.com/microsoft/STL/blob/fc15609a0f2ae2a134c34e7c9a13977994f37367/stl/inc/atomic#L103-L105 `do { ... } while (false)` has debug codegen costs. We don't need this because we ALWAYS use braces for control flow. `if (cond) { } else { ... }` also has debug codegen costs. We control the conditions, and the Standard's specification for boolean-testable has improved, so we shouldn't need this. We can drop `-Wassume` ("assumption is ignored because it contains (potential) side-effects") suppression for Clang in `_STL_VERIFY`. Clang doesn't support `/analyze`, but VSO_0938757_attribute_order is special - it runs Clang with `/D_PREFAST_` to verify that we're ordering standard attributes before non-standard attributes. Because this is the only test that's activating `_Analysis_assume_` for Clang, compiling it with `-Wno-assume` is better than modifying the product code.
…UGH` in `<atomic>`. For `_ATOMIC_LOAD_ARM64` and `_ATOMIC_POST_LOAD_BARRIER_AS_NEEDED`, use `_Check_load_memory_order` and arrange the control flow to preserve the fallthrough behavior. Because `_Check_load_memory_order` takes a real enum, we need to compare against the enumerators and avoid `static_cast<unsigned int>(_Order)`. For `_ATOMIC_STORE_PREFIX`, expand this **absolutely hideous** macro at each point of usage, then perform a similar transformation. As a bonus, we get to drop a bunch of early `return;`s.
davidmrdavid
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've completed a first pass. Thanks a lot, again, for the pristine commit history, it made this a lot easier. I've left some comments, most are non-blockers and others reveal bits I don't fully grok yet. I'll look to resolve these by tomorrow so I can unblock you sooner rather than later.
Excited for hardening! 💎 💎 💎
|
I'm mirroring this to the MSVC-internal repo - please notify me if any further changes are pushed. |
🗺️ Overview
Works towards #5090. For ease of review, this PR contains all of the cleanups and enhancements that I noticed along the way. The following PR will then be able to focus on removing
_CONTAINER_DEBUG_LEVELand adding STL Hardening without mixing in worthwhile-but-distracting changes.Notably, this PR has audited the STL for "Continue on Error" (CoE) compatibility, allowing users to define
_STL_CRT_SECURE_INVALID_PARAMETERto something that returns. (Throwing an exception is still super ultra forbidden.) Automated test coverage for CoE may follow in a future PR (it would inherently involve UB, so I need to investigate).This also improves debug codegen for the STL's debug checks, which were emitting unnecessary branches.
💬 Comment cleanups
_STL_VERIFYover multiple lines.In general, we comment our
#elseand#endifdirectives to clarify where "regions" begin and end. However, for very short regions that aren't densely nested in other preprocessor directives, we're willing to omit the commits as they cause more clutter than clarity.Here, removing the comments will make it easier to edit the conditions later. (All CDL conditions will be replaced with either the STL Hardening pattern, or IDL != 0.)
📜 Documentation cleanups
docs/import_library.md: Don't mention CDL - it doesn't affect ABI, and will be removed.📄 Debug message improvements
"array<T, 0> subscript is invalid"array<T, 0>, and that the call is permanently bogus, following the example of"array<T, 0>::front() is invalid"."MEOW subscript out of range"vector.mdspan's message was very opaque; starting with"mdspan subscript out of range"provides clarity. Additionally, "I" was previously used without being defined; I've replaced it with its definition from /2.ranges::view_interfacenow refers to itself with qualification."MEOW() called on empty WOOF"dequenow mentions whetherpop_front()orpop_back()was called.optionalnow mentions whetheroperator*()oroperator->()was called.ranges::view_interfacenow refers to itself with qualification.span'sfirst<Count>(),last<Count>(),subspan<Offset, Count>().string_viewremove_prefix(),remove_suffix().std::expected."expected stores an error, not a value"into:"operator->() called on a std::expected that contains an error, not a value""operator*() called on a std::expected that contains an error, not a value""expected stores a value, not an error"into:"error() called on a std::expected that contains a value, not an error""expected"doesn't really sound like a noun, I've added"std::"qualification and made the phrases less terse. And as @sivadeilra suggested, "stores" has connotations of "memory writes", so "contains" is clearer.<format>'s implementation and couldn't resist fixing it.☑️ Test cleanups
<ranges>tests.IDL != 0.<format>test, properly test negative dynamic width/precision.<format>isn't CDL-sensitive.format_error; we don't use "death tests" for exceptions. For clarity and consistency, add positive and negative test cases for both. This may slightly duplicate existing coverage, but negative dynamic precision wasn't being properly tested AFAICT.if constexprinstead of tag dispatch and plainif.🪄 Code cleanups
const, unwrap "strengthened"._STL_REPORT_ERROR=>_STL_VERIFYor_STL_ASSERT<memory_resource>, we always call_Prepare_oversized, but the_STL_REPORT_ERRORwas guarded by#ifdef _DEBUG. Convert this to_STL_ASSERT, which is consistent with the_STL_ASSERTthat appears immediately below.<numeric>gcd(), we're using De Morgan's laws, the STL maintainer's best friend.<numeric>lcm(), extractconst bool _Overflowso the_STL_VERIFYhas no side effects._STL_VERIFY(false)=>_STL_REPORT_ERROR<mdspan>, addreturn 1;instride()to support upcoming changes for continue-on-error.<regex>, unconditionallyreturn false;for the same reason.casebeforedefault._Check_alignmentintoatomic_refctor._ATOMIC_REF_CHECK_ALIGNMENTinto_STL_ASSERT._Offand_Count. Addconstto_Moved._STD copy()call._Newsizefor clarity._STL_ASSERTshould be_STL_VERIFY._STL_VERIFYalready follows._Can_memcpyfor clarity, simplifying following CoE fix._Check_MEOW_memory_orderto avoid some_FALLTHROUGHin<atomic>._ATOMIC_LOAD_ARM64and_ATOMIC_POST_LOAD_BARRIER_AS_NEEDED, use_Check_load_memory_orderand arrange the control flow to preserve the fallthrough behavior._Check_load_memory_ordertakes a real enum, we need to compare against the enumerators and avoidstatic_cast<unsigned int>(_Order)._ATOMIC_STORE_PREFIX, expand this absolutely hideous macro 🤮 at each point of usage, then perform a similar transformation. As a bonus, we get to drop a bunch of earlyreturn;s.🚀 Enhancements
uninitialized_meow's logic._INVALID_MEMORY_ORDER, consistently fallthrough toseq_cst._STL_ASSERT=>_STL_VERIFYfor checks within IDL >= 1._STL_VERIFYbelow._STL_ASSERTwith_STL_INTERNAL_STATIC_ASSERT.durationandratio. They don't need to be checked for users, especially at runtime.auto_ptrderef/arrow._Myptris non-null when we're about to dereference it (or return it for imminent dereferencing). We purged this category of asserts from the STL aeons ago, but must have missedauto_ptr. The idea is that null dereferences are just as obvious in the debugger. Note that this makes the code consistent withunique_ptrandshared_ptr._Span_iterator::operator-=was unconditionally checking._Verify_offsetdoes something for IDL >= 1 only, so we can guard the callsite inoperator+=.operator-=, the pathological "integer overflow" check was being performed unconditionally. Guard it for IDL >= 1, and the following_Verify_offsetcall._STL_REPORT_ERROR,_STL_VERIFYcodegen.STL/stl/inc/atomic
Lines 103 to 105 in fc15609
do { ... } while (false)has debug codegen costs. We don't need this because we ALWAYS use braces for control flow.if (cond) { } else { ... }also has debug codegen costs. We control the conditions, and the Standard's specification for boolean-testable has improved, so we shouldn't need this.-Wassume("assumption is ignored because it contains (potential) side-effects") suppression for Clang in_STL_VERIFY. Clang doesn't support/analyze, butVSO_0938757_attribute_orderis special - it runs Clang with/D_PREFAST_to verify that we're ordering standard attributes before non-standard attributes. Because this is the only test that's activating_Analysis_assume_for Clang, compiling it with-Wno-assumeis better than modifying the product code.