Skip to content

[libc++] Add __new_at_least - #215813

Open
philnik777 wants to merge 2 commits into
llvm:mainfrom
philnik777:allocate_at_least
Open

philnik777 wants to merge 2 commits into
llvm:mainfrom
philnik777:allocate_at_least

Conversation

@philnik777

@philnik777 philnik777 commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

This adds a variant for operator new (spelled as __new_at_least, since this is a library-internal extension) that returns the number of objects actually allocated. It is being used in std::allocator::allocate_at_least, which then allows vector and string to make use of any over-allocations.

Fixes #212790

Comment thread libcxx/include/__configuration/availability.h Outdated
Comment thread libcxx/include/__new/allocate.h
Comment thread libcxx/src/support/new.ipp Outdated
Comment thread libcxx/src/support/new.ipp Outdated
@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown

✅ With the latest revision this PR passed the C/C++ code formatter.

@philnik777
philnik777 force-pushed the allocate_at_least branch 4 times, most recently from fc95ec0 to 211353a Compare August 24, 2026 10:48
@philnik777 philnik777 self-assigned this Aug 24, 2026
@philnik777
philnik777 marked this pull request as ready for review August 24, 2026 14:29
@philnik777
philnik777 requested a review from a team as a code owner August 24, 2026 14:29
@llvmorg-github-actions llvmorg-github-actions Bot added the libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi. label Aug 24, 2026
@llvmorg-github-actions

Copy link
Copy Markdown

@llvm/pr-subscribers-libcxx

Author: Nikolas Klauser (philnik777)

Changes

Full diff: https://github.com/llvm/llvm-project/pull/215813.diff

12 Files Affected:

  • (modified) libcxx/include/CMakeLists.txt (+1)
  • (modified) libcxx/include/__configuration/attributes.h (+6)
  • (modified) libcxx/include/__configuration/availability.h (+10)
  • (modified) libcxx/include/__memory/allocate_at_least.h (+12-10)
  • (modified) libcxx/include/__memory/allocator.h (+6-1)
  • (modified) libcxx/include/__new/allocate.h (+19)
  • (added) libcxx/include/__new/new_at_least.h (+45)
  • (modified) libcxx/include/module.modulemap.in (+1)
  • (modified) libcxx/lib/abi/CHANGELOG.TXT (+6)
  • (modified) libcxx/lib/abi/x86_64-unknown-linux-gnu.libcxxabi.v1.stable.exceptions.nonew.abilist (+1)
  • (modified) libcxx/src/support/new.ipp (+95-12)
  • (added) libcxx/test/libcxx/language.support/support.dynamic/libcpp_allocate_at_least.pass.cpp (+90)
diff --git a/libcxx/include/CMakeLists.txt b/libcxx/include/CMakeLists.txt
index d717a7354c62b..7b9f45e106a9f 100644
--- a/libcxx/include/CMakeLists.txt
+++ b/libcxx/include/CMakeLists.txt
@@ -655,6 +655,7 @@ set(files
   __new/global_new_delete.h
   __new/interference_size.h
   __new/launder.h
+  __new/new_at_least.h
   __new/new_handler.h
   __new/nothrow_t.h
   __new/placement_new_delete.h
diff --git a/libcxx/include/__configuration/attributes.h b/libcxx/include/__configuration/attributes.h
index 2f8841fce0a07..2ae819213534c 100644
--- a/libcxx/include/__configuration/attributes.h
+++ b/libcxx/include/__configuration/attributes.h
@@ -473,6 +473,12 @@
 #  define _LIBCPP_DISABLE_POINTER_FIELD_PROTECTION
 #endif
 
+#if __has_cpp_attribute(_Clang::__malloc_span__)
+#  define _LIBCPP_MALLOC_SPAN [[_Clang::__malloc_span__]]
+#else
+#  define _LIBCPP_MALLOC_SPAN
+#endif
+
 // TODO(LLVM 25): Remove this escape hatch
 #ifndef _LIBCPP_DISABLE_UNUSED_STRUCT_WARNINGS
 #  define _LIBCPP_WARN_UNUSED [[__gnu__::__warn_unused__]]
diff --git a/libcxx/include/__configuration/availability.h b/libcxx/include/__configuration/availability.h
index 0709deff47ea6..bccbc907ccfcc 100644
--- a/libcxx/include/__configuration/availability.h
+++ b/libcxx/include/__configuration/availability.h
@@ -39,6 +39,9 @@
 // in all versions of the library are available.
 #if !_LIBCPP_HAS_VENDOR_AVAILABILITY_ANNOTATIONS
 
+#  define _LIBCPP_INTRODUCED_IN_LLVM_24 1
+#  define _LIBCPP_INTRODUCED_IN_LLVM_24_ATTRIBUTE /* nothing */
+
 #  define _LIBCPP_INTRODUCED_IN_LLVM_23 1
 #  define _LIBCPP_INTRODUCED_IN_LLVM_23_ATTRIBUTE /* nothing */
 
@@ -73,6 +76,11 @@
 
 // clang-format off
 
+// LLVM 24
+// TODO: Fill this in
+#  define _LIBCPP_INTRODUCED_IN_LLVM_24 0
+#  define _LIBCPP_INTRODUCED_IN_LLVM_24_ATTRIBUTE __attribute__((unavailable))
+
 // LLVM 23
 // TODO: Fill this in
 #  define _LIBCPP_INTRODUCED_IN_LLVM_23 0
@@ -239,6 +247,8 @@
 
 #endif
 
+#define _LIBCPP_AVAILABILITY_HAS_NEW_AT_LEAST _LIBCPP_INTRODUCED_IN_LLVM_24
+
 // This determines whether we assume that the internal std::__bad_variant_access_with_msg class
 // (which carries a message describing the cause of the failure in bad_variant_access::what())
 // provides a key function in the dylib. This allows centralizing its vtable and typeinfo instead
diff --git a/libcxx/include/__memory/allocate_at_least.h b/libcxx/include/__memory/allocate_at_least.h
index 72140d0de27af..50be5d664a91c 100644
--- a/libcxx/include/__memory/allocate_at_least.h
+++ b/libcxx/include/__memory/allocate_at_least.h
@@ -11,7 +11,10 @@
 
 #include <__config>
 #include <__cstddef/size_t.h>
+#include <__fwd/memory.h>
 #include <__memory/allocator_traits.h>
+#include <__new/allocate.h>
+#include <__type_traits/is_constant_evaluated.h>
 
 #if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
 #  pragma GCC system_header
@@ -19,16 +22,6 @@
 
 _LIBCPP_BEGIN_NAMESPACE_STD
 
-template <class _Pointer, class _SizeT = size_t>
-struct __allocation_result {
-  _Pointer ptr;
-  _SizeT count;
-
-  _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR __allocation_result(_Pointer __ptr, _SizeT __count)
-      : ptr(__ptr), count(__count) {}
-};
-_LIBCPP_CTAD_SUPPORTED_FOR_TYPE(__allocation_result);
-
 #if _LIBCPP_STD_VER >= 23
 
 template <class _Alloc>
@@ -46,6 +39,15 @@ __allocate_at_least(_Alloc& __alloc, size_t __n) {
   return __allocation_result<typename _Traits::pointer, typename _Traits::size_type>(__alloc.allocate(__n), __n);
 }
 
+template <class _Tp>
+[[__nodiscard__]] _LIBCPP_CONSTEXPR __allocation_result<_Tp*> __allocate_at_least(allocator<_Tp>& __alloc, size_t __n) {
+  if (__libcpp_is_constant_evaluated()) {
+    return __allocation_result<_Tp*>(__alloc.allocate(__n), __n);
+  } else {
+    return std::__libcpp_allocate_at_least<_Tp>(__element_count(__n));
+  }
+}
+
 #endif // _LIBCPP_STD_VER >= 23
 
 _LIBCPP_END_NAMESPACE_STD
diff --git a/libcxx/include/__memory/allocator.h b/libcxx/include/__memory/allocator.h
index dd8c8647c178a..cffc65d19e913 100644
--- a/libcxx/include/__memory/allocator.h
+++ b/libcxx/include/__memory/allocator.h
@@ -96,7 +96,12 @@ class allocator
 #if _LIBCPP_STD_VER >= 23
   [[nodiscard]] _LIBCPP_HIDE_FROM_ABI constexpr allocation_result<_Tp*> allocate_at_least(size_t __n) {
     static_assert(sizeof(_Tp) >= 0, "cannot allocate memory for an incomplete type");
-    return {allocate(__n), __n};
+    if consteval {
+      return {allocate(__n), __n};
+    } else {
+      auto [__ptr, __count] = std::__libcpp_allocate_at_least<_Tp>(__element_count(__n));
+      return {__ptr, __count};
+    }
   }
 #endif
 
diff --git a/libcxx/include/__new/allocate.h b/libcxx/include/__new/allocate.h
index b9bc2e1a50710..dd4b859535775 100644
--- a/libcxx/include/__new/allocate.h
+++ b/libcxx/include/__new/allocate.h
@@ -13,6 +13,7 @@
 #include <__cstddef/max_align_t.h>
 #include <__cstddef/size_t.h>
 #include <__new/align_val_t.h>
+#include <__new/new_at_least.h>
 #include <__type_traits/type_identity.h>
 #include <__utility/element_count.h>
 
@@ -42,6 +43,24 @@ __libcpp_allocate(__element_count __n, [[__maybe_unused__]] size_t __align = _LI
   return static_cast<_Tp*>(__builtin_operator_new(__size));
 }
 
+template <class _Tp>
+_LIBCPP_NO_CFI __allocation_result<_Tp*>
+__libcpp_allocate_at_least(__element_count __n, [[__maybe_unused__]] size_t __align = _LIBCPP_ALIGNOF(_Tp)) {
+#if _LIBCPP_AVAILABILITY_HAS_NEW_AT_LEAST
+  size_t __size = static_cast<size_t>(__n) * sizeof(_Tp);
+#  if _LIBCPP_HAS_ALIGNED_ALLOCATION
+  if (__is_overaligned_for_new(__align)) {
+    auto [__ptr, __count] = std::__new_at_least(__size, static_cast<align_val_t>(__align));
+    return {static_cast<_Tp*>(__ptr), __count / sizeof(_Tp)};
+  }
+#  endif
+  auto [__ptr, __count] = std::__new_at_least(__size);
+  return __allocation_result<_Tp*>(static_cast<_Tp*>(__ptr), __count / sizeof(_Tp));
+#else
+  return __allocation_result<_Tp*>(std::__libcpp_allocate<_Tp>(__n, __align), __n);
+#endif
+}
+
 #if defined(__cpp_sized_deallocation) && __cpp_sized_deallocation >= 201309L
 #  define _LIBCPP_ONLY_IF_SIZED_DEALLOCATION(...) __VA_ARGS__
 #else
diff --git a/libcxx/include/__new/new_at_least.h b/libcxx/include/__new/new_at_least.h
new file mode 100644
index 0000000000000..181dc38d85d37
--- /dev/null
+++ b/libcxx/include/__new/new_at_least.h
@@ -0,0 +1,45 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _LIBCPP___NEW_NEW_AT_LEAST_H
+#define _LIBCPP___NEW_NEW_AT_LEAST_H
+
+#include <__config>
+#include <__cstddef/size_t.h>
+#include <__new/align_val_t.h>
+
+#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
+#  pragma GCC system_header
+#endif
+
+_LIBCPP_BEGIN_NAMESPACE_STD
+
+template <class _Pointer, class _SizeT = size_t>
+struct __allocation_result {
+  _Pointer ptr;
+  _SizeT count;
+
+  _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR __allocation_result(_Pointer __ptr, _SizeT __count)
+      : ptr(__ptr), count(__count) {}
+};
+_LIBCPP_CTAD_SUPPORTED_FOR_TYPE(__allocation_result);
+
+#if _LIBCPP_AVAILABILITY_HAS_NEW_AT_LEAST
+_LIBCPP_BEGIN_EXPLICIT_ABI_ANNOTATIONS
+
+_LIBCPP_MALLOC_SPAN _LIBCPP_EXPORTED_FROM_ABI __allocation_result<void*> __new_at_least(size_t);
+#  if _LIBCPP_HAS_LIBRARY_ALIGNED_ALLOCATION
+_LIBCPP_MALLOC_SPAN _LIBCPP_EXPORTED_FROM_ABI __allocation_result<void*> __new_at_least(size_t, align_val_t);
+#  endif
+
+_LIBCPP_END_EXPLICIT_ABI_ANNOTATIONS
+#endif
+
+_LIBCPP_END_NAMESPACE_STD
+
+#endif // _LIBCPP___NEW_NEW_AT_LEAST_H
diff --git a/libcxx/include/module.modulemap.in b/libcxx/include/module.modulemap.in
index 78f4e08d9c282..55ca002b6b6b9 100644
--- a/libcxx/include/module.modulemap.in
+++ b/libcxx/include/module.modulemap.in
@@ -1794,6 +1794,7 @@ module std {
     }
     module interference_size    { header "__new/interference_size.h" }
     module launder              { header "__new/launder.h" }
+    module new_at_least         { header "__new/new_at_least.h" }
     module new_handler          { header "__new/new_handler.h" }
     module nothrow_t            { header "__new/nothrow_t.h" }
     module placement_new_delete { header "__new/placement_new_delete.h" }
diff --git a/libcxx/lib/abi/CHANGELOG.TXT b/libcxx/lib/abi/CHANGELOG.TXT
index 30bb3d6254965..61b6e8869c8c3 100644
--- a/libcxx/lib/abi/CHANGELOG.TXT
+++ b/libcxx/lib/abi/CHANGELOG.TXT
@@ -16,6 +16,12 @@ New entries should be added directly below the "Version" header.
 Version 24.0
 ------------
 
+* [libc++] Add __new_at_least
+
+  All platforms
+  -------------
+  Symbol added: _ZNSt3__114__new_at_leastEm
+
 * [libc++] Remove locale::id::__next_id from the ABI
 
   `__next_id` is only ever used inside the dylib, so we can avoid making it public.
diff --git a/libcxx/lib/abi/x86_64-unknown-linux-gnu.libcxxabi.v1.stable.exceptions.nonew.abilist b/libcxx/lib/abi/x86_64-unknown-linux-gnu.libcxxabi.v1.stable.exceptions.nonew.abilist
index c727c028d21e6..25a006ec26b59 100644
--- a/libcxx/lib/abi/x86_64-unknown-linux-gnu.libcxxabi.v1.stable.exceptions.nonew.abilist
+++ b/libcxx/lib/abi/x86_64-unknown-linux-gnu.libcxxabi.v1.stable.exceptions.nonew.abilist
@@ -9,6 +9,7 @@
 {'is_defined': False, 'name': '_ZNSt16invalid_argumentD1Ev', 'type': 'FUNC'}
 {'is_defined': False, 'name': '_ZNSt20bad_array_new_lengthC1Ev', 'type': 'FUNC'}
 {'is_defined': False, 'name': '_ZNSt20bad_array_new_lengthD1Ev', 'type': 'FUNC'}
+{'is_defined': False, 'name': '_ZNSt3__114__new_at_leastEm', 'type': 'FUNC'}
 {'is_defined': False, 'name': '_ZNSt8bad_castC1Ev', 'type': 'FUNC'}
 {'is_defined': False, 'name': '_ZNSt8bad_castD1Ev', 'type': 'FUNC'}
 {'is_defined': False, 'name': '_ZNSt8bad_castD2Ev', 'type': 'FUNC'}
diff --git a/libcxx/src/support/new.ipp b/libcxx/src/support/new.ipp
index 80be2d904af4c..f07fe2559ea48 100644
--- a/libcxx/src/support/new.ipp
+++ b/libcxx/src/support/new.ipp
@@ -6,6 +6,7 @@
 //
 //===----------------------------------------------------------------------===//
 
+#include <__memory/allocate_at_least.h>
 #include <cstddef>
 #include <cstdlib>
 #include <new>
@@ -23,6 +24,12 @@ void __throw_bad_alloc_shim();
 #  define _LIBCPP_ASSERT_SHIM
 #endif
 
+enum class on_failure {
+  return_null,
+  throw_bad_alloc,
+};
+
+template <on_failure failure_mode>
 static void* operator_new_impl(std::size_t size) {
   if (size == 0)
     size = 1;
@@ -36,14 +43,13 @@ static void* operator_new_impl(std::size_t size) {
     else
       break;
   }
+  if (failure_mode == on_failure::throw_bad_alloc && !p)
+    __throw_bad_alloc_shim();
   return p;
 }
 
 OVERRIDABLE_FUNCTION void* operator new(std::size_t size) _THROW_BAD_ALLOC {
-  void* p = operator_new_impl(size);
-  if (p == nullptr)
-    __throw_bad_alloc_shim();
-  return p;
+  return operator_new_impl<on_failure::throw_bad_alloc>(size);
 }
 
 [[gnu::weak]] void* operator new(size_t size, const std::nothrow_t&) noexcept {
@@ -59,7 +65,7 @@ OVERRIDABLE_FUNCTION void* operator new(std::size_t size) _THROW_BAD_ALLOC {
       "`operator new(size_t, nothrow_t)` as well.");
 #  endif
 
-  return operator_new_impl(size);
+  return operator_new_impl<on_failure::return_null>(size);
 #else
   void* p = nullptr;
   try {
@@ -85,7 +91,7 @@ OVERRIDABLE_FUNCTION void* operator new[](size_t size) _THROW_BAD_ALLOC { return
       "`operator new[](size_t, nothrow_t)` as well.");
 #  endif
 
-  return operator_new_impl(size);
+  return operator_new_impl<on_failure::return_null>(size);
 #else
   void* p = nullptr;
   try {
@@ -110,6 +116,7 @@ OVERRIDABLE_FUNCTION void* operator new[](size_t size) _THROW_BAD_ALLOC { return
 
 #if _LIBCPP_HAS_LIBRARY_ALIGNED_ALLOCATION
 
+template <on_failure failure_mode>
 static void* operator_new_aligned_impl(std::size_t size, std::align_val_t alignment) {
   if (size == 0)
     size = 1;
@@ -127,14 +134,13 @@ static void* operator_new_aligned_impl(std::size_t size, std::align_val_t alignm
     else
       break;
   }
+  if (failure_mode == on_failure::throw_bad_alloc && !p)
+    __throw_bad_alloc_shim();
   return p;
 }
 
 OVERRIDABLE_FUNCTION void* operator new(std::size_t size, std::align_val_t alignment) _THROW_BAD_ALLOC {
-  void* p = operator_new_aligned_impl(size, alignment);
-  if (p == nullptr)
-    __throw_bad_alloc_shim();
-  return p;
+  return operator_new_aligned_impl<on_failure::throw_bad_alloc>(size, alignment);
 }
 
 [[gnu::weak]] void* operator new(size_t size, std::align_val_t alignment, const std::nothrow_t&) noexcept {
@@ -150,7 +156,7 @@ OVERRIDABLE_FUNCTION void* operator new(std::size_t size, std::align_val_t align
       "`operator new(size_t, align_val_t, nothrow_t)` as well.");
 #    endif
 
-  return operator_new_aligned_impl(size, alignment);
+  return operator_new_aligned_impl<on_failure::return_null>(size, alignment);
 #  else
   void* p = nullptr;
   try {
@@ -178,7 +184,7 @@ OVERRIDABLE_FUNCTION void* operator new[](size_t size, std::align_val_t alignmen
       "override `operator new[](size_t, align_val_t, nothrow_t)` as well.");
 #    endif
 
-  return operator_new_aligned_impl(size, alignment);
+  return operator_new_aligned_impl<on_failure::return_null>(size, alignment);
 #  else
   void* p = nullptr;
   try {
@@ -211,3 +217,80 @@ OVERRIDABLE_FUNCTION void* operator new[](size_t size, std::align_val_t alignmen
   ::operator delete[](ptr, alignment);
 }
 #endif // _LIBCPP_HAS_LIBRARY_ALIGNED_ALLOCATION
+
+#ifdef __APPLE__
+#  include <malloc/malloc.h>
+#elifdef __FreeBSD__
+#  include <malloc_np.h>
+#endif
+
+// FIXME: Clang should really accept functions in [[gnu::ifunc]] (or possibly [[clang::ifunc]])
+using std::__allocation_result;
+
+using new_t         = void*(std::size_t);
+using new_aligned_t = void*(std::size_t, std::align_val_t);
+
+using new_at_least_t         = __allocation_result<void*>(std::size_t);
+using new_at_least_aligned_t = __allocation_result<void*>(std::size_t, std::align_val_t);
+
+[[maybe_unused]] static new_at_least_t* new_at_least_resolver() {
+  if (std::__is_function_overridden < new_t, operator new>()) {
+    return [](std::size_t size) -> __allocation_result<void*> { return {::operator new(size), size}; };
+  } else {
+    return [](std::size_t size) -> __allocation_result<void*> {
+#ifdef __APPLE__
+      auto good_size = ::malloc_good_size(size);
+      return {operator_new_impl<on_failure::throw_bad_alloc>(good_size), good_size};
+#elifdef __FreeBSD__
+      auto good_size = ::nallocx(size, 0);
+      return {operator_new_impl<on_failure::throw_bad_alloc>(good_size), good_size};
+#else
+      // Other platforms should specialize this for their system allocator
+      return {operator_new_impl<on_failure::throw_bad_alloc>(size), size};
+#endif
+    };
+  }
+}
+
+[[maybe_unused]] static new_at_least_aligned_t* new_at_least_aligned_resolver() {
+  if (std::__is_function_overridden < new_aligned_t, operator new>()) {
+    return [](std::size_t size, std::align_val_t align) -> __allocation_result<void*> {
+      return {::operator new(size, align), size};
+    };
+  } else {
+    return [](std::size_t size, std::align_val_t align) -> __allocation_result<void*> {
+#ifdef __APPLE__
+      auto good_size = malloc_good_size(size);
+      return {operator_new_aligned_impl<on_failure::throw_bad_alloc>(good_size, align), good_size};
+#elifdef __FreeBSD__
+      auto good_size = ::nallocx(size, MALLOCX_ALIGN(static_cast<size_t>(align)));
+      return {operator_new_aligned_impl<on_failure::throw_bad_alloc>(good_size), good_size};
+#else
+      return {operator_new_aligned_impl<on_failure::throw_bad_alloc>(size, align), size};
+#endif
+    };
+  }
+}
+
+_LIBCPP_BEGIN_NAMESPACE_STD
+_LIBCPP_BEGIN_EXPLICIT_ABI_ANNOTATIONS
+
+#if __has_cpp_attribute(gnu::ifunc)
+
+[[gnu::ifunc("_ZL21new_at_least_resolverv")]] new_at_least_t __new_at_least;
+[[gnu::ifunc("_ZL29new_at_least_aligned_resolverv")]] new_at_least_aligned_t __new_at_least;
+
+#else
+
+std::__allocation_result<void*> __new_at_least(std::size_t size) { return {::operator new(size), size}; }
+
+#  if _LIBCPP_HAS_LIBRARY_ALIGNED_ALLOCATION
+std::__allocation_result<void*> __new_at_least(std::size_t size, std::align_val_t align) {
+  return {::operator new(size, align), size};
+}
+#  endif
+
+#endif
+
+_LIBCPP_END_EXPLICIT_ABI_ANNOTATIONS
+_LIBCPP_END_NAMESPACE_STD
diff --git a/libcxx/test/libcxx/language.support/support.dynamic/libcpp_allocate_at_least.pass.cpp b/libcxx/test/libcxx/language.support/support.dynamic/libcpp_allocate_at_least.pass.cpp
new file mode 100644
index 0000000000000..62a0cd07acfd3
--- /dev/null
+++ b/libcxx/test/libcxx/language.support/support.dynamic/libcpp_allocate_at_least.pass.cpp
@@ -0,0 +1,90 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11
+
+// Check that __libcpp_allocate_at_least works as expected
+
+#include <__memory/is_sufficiently_aligned.h>
+#include <__new/allocate.h>
+#include <cassert>
+#include <limits>
+#include <new>
+
+#include "test_macros.h"
+
+int new_handler_called = 0;
+
+void my_new_handler() {
+  ++new_handler_called;
+  std::set_new_handler(nullptr);
+}
+
+int main(int, char**) {
+  { // Check that a simple call works as expected
+    auto result = std::__libcpp_allocate_at_least<char>(std::__element_count(1));
+    assert(result.ptr);
+    assert(result.count >= 1);
+#if defined(_LIBCPP_AVAILABILITY_HAS_NEW_AT_LEAST) && (defined(__APPLE__) || defined(__FreeBSD__))
+    // Check that on platforms where we have a better implementation than the generic one we will always get more than
+    // one byte. This is technically not guaranteed, but allocators are in general not capable of allocating individual
+    // bytes efficiently.
+    assert(result.count > 1);
+#endif
+    operator delete(result.ptr, result.count);
+  }
+
+#ifndef TEST_HAS_NO_ALIGNED_ALLOCATION
+  { // Check that the aligned version is called with big alignment
+    auto result = std::__libcpp_allocate_at_least<char>(std::__element_count(1), __STDCPP_DEFAULT_NEW_ALIGNMENT__ * 2);
+    assert(result.ptr);
+    assert(std::__is_sufficiently_aligned<__STDCPP_DEFAULT_NEW_ALIGNMENT__ * 2>(result.ptr));
+    assert(result.count >= 1);
+#  if defined(_LIBCPP_AVAILABILITY_HAS_NEW_AT_LEAST) && (defined(__APPLE__) || defined(__FreeBSD__))
+    // Check that on platforms where we have a better implementation than the generic one we will always get more than
+    // one byte. This is technically not guaranteed, but allocators are in general not capable of allocating individual
+    // bytes efficiently.
+    assert(result.count > 1);
+#  endif
+    operator delete(result.ptr, result.count);
+  }
+#endif // TEST_HAS_NO_ALIGNED_ALLOCATION
+
+  { // Test that the new handler is called if allocation fails
+#ifndef TEST_HAS_NO_EXCEPTIONS
+    new_handler_called = 0;
+    std::set_new_handler(my_new_handler);
+    try {
+      (void)std::__libcpp_allocate_at_least<char>(std::__element_count(std::numeric_limits<std::size_t>::max()));
+      assert(false);
+    } catch (std::bad_alloc const&) {
+      assert(new_handler_called == 1);
+    } catch (...) {
+      assert(false);
+    }
+#endif
+  }
+
+  { // Test that the new handler is called if allocation fails (aligned overload)
+#if !defined(TEST_HAS_NO_EXCEPTIONS) && !defined(TEST_HAS_NO_ALIGNED_ALLOCATION)
+    new_handler_called = 0;
+    std::set_new_handler(my_new_handler);
+    try {
+      (void)std::__libcpp_allocate_at_least<char>(
+          std::__element_count(std::numeric_limits<std::size_t>::max()), __STDCPP_DEFAULT_NEW_ALIGNMENT__ * 2);
+      assert(false);
+    } catch (std::bad_alloc const&) {
+      assert(new_handler_called == 1);
+    } catch (...) {
+      assert(false);
+    }
+#endif
+  }
+
+  return 0;
+}

Comment thread libcxx/include/__configuration/availability.h
Comment thread libcxx/include/__memory/allocate_at_least.h
Comment thread libcxx/include/__memory/allocate_at_least.h
Comment thread libcxx/include/__new/new_at_least.h
Comment thread libcxx/include/__new/new_at_least.h Outdated
Comment thread libcxx/src/support/new.ipp Outdated
Comment thread libcxx/src/support/new.ipp
Comment thread libcxx/src/support/new.ipp

@ldionne ldionne left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking really good.

We had a very long discussion about the state of things with respect to operator new, where it lives and how various platforms handle libc++abi.dylib and libc++.dylib. Since that is quite a mess, we agreed to try and see if we can simplify the situation by always putting operator new inside libc++abi.dylib, which seems to only not be the case on FreeBSD right now.


// UNSUPPORTED: c++03, c++11

// Check that __libcpp_allocate_at_least falls back to a user-provided operator new if provided.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Check that __libcpp_allocate_at_least falls back to a user-provided operator new if provided.
// Check that __libcpp_allocate_at_least falls back to a user-provided operator new if operator new is overridden. This test checks the unaligned variant of __libcpp_allocate_at_least.


// operator new(size_t, align_val_t) isn't overridden, so we still use the special implementation.
#ifndef TEST_HAS_NO_ALIGNED_ALLOCATION
{ // Check that the aligned version is called with when using an alignment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{ // Check that the aligned version is called with when using an alignment
{ // Check that the aligned version is called when using an alignment


// UNSUPPORTED: c++03, c++11

// Check that __libcpp_allocate_at_least falls back to a user-provided operator new if provided.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Check that __libcpp_allocate_at_least falls back to a user-provided operator new if provided.
// Check that __libcpp_allocate_at_least falls back to a user-provided operator new if operator new is overridden. This test checks the aligned variant of __libcpp_allocate_at_least.

@@ -0,0 +1,55 @@
//===----------------------------------------------------------------------===//

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you confirm that running this test locally while removing the __is_function_overridden check in the library actually produces a failure? Just cause it's easy to write these kinds of tests in a way that doesn't actually test what we're intending to test for subtle reasons.

Comment thread libcxx/src/support/new.ipp
// `__new_at_least` acts like an overload of `operator new` which takes a size (and possibly alignment) and returns
// a pointer as well as the actually allocated amount of memory. If the user replaces the relevant `operator new`
// overload this will fall back to calling that. Otherwise it tries to allocate in a way to get the actually allocated
// size, depending on what the platform provides.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should document that this CANNOT be used in places where we would want to call operator[], since it doesn't check whether operator[] has been overridden.

Comment on lines +56 to +60
# if __has_builtin(__builtin_assume)
__builtin_assume(__ptr);
__builtin_assume((reinterpret_cast<uintptr_t>(__ptr) & (__align - 1)) == 0);
__builtin_assume(__count >= __size);
# endif

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should redefine _LIBCPP_ASSUME to be __builtin_assume (when it exists), however we should drop _LIBCPP_ASSUME from the assertion checks in __assert by default (see this comment):

// WARNING: __builtin_assume can currently inhibit optimizations. Only add assumptions with a clear
// optimization intent. See https://discourse.llvm.org/t/llvm-assume-blocks-optimization/71609 for a
// discussion.

Oh and we should probably define that in __utility/assume.h or something similar.

#if _LIBCPP_AVAILABILITY_HAS_NEW_AT_LEAST
size_t __size = static_cast<size_t>(__n) * sizeof(_Tp);
# if _LIBCPP_HAS_ALIGNED_ALLOCATION
if (__is_overaligned_for_new(__align)) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can simplify to:

#if _LIBCPP_HAS_ALIGNED_ALLOCATION
    auto [__ptr, __count] = __is_overaligned_for_new(__align)
        ? std::__new_at_least(__size, static_cast<align_val_t>(__align))
        : std::__new_at_least(__size);
#else
    auto [__ptr, __count] = std::__new_at_least(__size);
#endif

_LIBCPP_ASSUME(...);
_LIBCPP_ASSUME(...);
_LIBCPP_ASSUME(...);

return __allocation_result<_Tp*>(static_cast<_Tp*>(__ptr), __count / sizeof(_Tp));

#endif
}

OVERRIDABLE_FUNCTION void* operator new[](size_t size) _THROW_BAD_ALLOC { return ::operator new(size); }

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to document what we just found out about overridable_function.h. First, since #208330, I think __is_function_overridden only works within a single TU: in other words, you must define the function you're checking for an override in the same TU where you are defining that function with the OVERRIDABLE_FUNCTION attribute.

I'm not certain this is actually something we want to require since it seems very brittle, but we should have a discussion and document it. Can you open a separate PR?

@@ -6,6 +6,8 @@
//

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we would need to put new_at_least in libcxxabi/lib/new-delete.exp so it gets re-exported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi. performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[libc++] Consider adding an allocate_at_least-like function to the new-family of functions

2 participants