Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Typos corrections to Overview.md #1

Closed
wants to merge 1 commit into from
Closed

Typos corrections to Overview.md #1

wants to merge 1 commit into from

Conversation

chrisaycock
Copy link
Contributor

Some typos corrections, and collapsed the two descriptions of Bits.h into one.

@ghost ghost assigned andrewjcg Jun 3, 2012
@tudor tudor closed this Jun 5, 2012
@tudor
Copy link
Contributor

tudor commented Jun 5, 2012

Merged manually, thanks!

@chrisaycock
Copy link
Contributor Author

@tudor Glad I could be of help!

unmeshvrije pushed a commit to unmeshvrije/C-- that referenced this pull request Feb 16, 2014
…o one.

Summary: facebook/folly#1

Test Plan: N/A.

Reviewed By: [email protected]

FB internal diff: D485738
@WenlinWang WenlinWang mentioned this pull request Feb 3, 2015
ghost pushed a commit that referenced this pull request Sep 15, 2015
Summary: avoid sillyness:

```
$ _build/opt/folly/io/async/test/async_test --gtest_list_tests
ASAN:SIGSEGV
=================================================================
==3245135==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000010 (pc 0x000000583444 sp 0x7fff17ba0c40 bp 0x7fff17ba0c80 T0)
    #0 0x583443 in std::_Rb_tree<int, std::pair<int const, folly::SSLContext::SSLLockType>, std::_Select1st<std::pair<int const, folly::SSLContext::SSLLockType> >, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > >::operator=(std::_Rb_tree<int, std::pair<int const, folly::SSLContext::SSLLockType>, std::_Select1st<std::pair<int const, folly::SSLContext::SSLLockType> >, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > > const&) third-party2/libgcc/4.9.x/gcc-4.9-glibc-2.20-fb/024dbc3/include/c++/4.9.x/bits/stl_tree.h:1138
    #1 0x583443 in std::map<int, folly::SSLContext::SSLLockType, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > >::operator=(std::map<int, folly::SSLContext::SSLLockType, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > > const&) third-party2/libgcc/4.9.x/gcc-4.9-glibc-2.20-fb/024dbc3/include/c++/4.9.x/bits/stl_map.h:295
    #2 0x583443 in folly::SSLContext::setSSLLockTypes(std::map<int, folly::SSLContext::SSLLockType, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > >) folly/io/async/SSLContext.cpp:682
    #3 0x40e028 in Initializer folly/io/async/test/AsyncSSLSocketTest2.cpp:143
    #4 0x40e028 in __static_initialization_and_destruction_0 folly/io/async/test/AsyncSSLSocketTest2.cpp:146
    #5 0x40e028 in _GLOBAL__sub_I__ZN5folly47AsyncSSLSocketTest2_AttachDetachSSLContext_Test10test_info_E folly/io/async/test/AsyncSSLSocketTest2.cpp:147
    #6 0x66cf2e in __libc_csu_init /home/engshare/third-party2/glibc/2.20/src/glibc-2.20/csu/elf-init.c:88
    #7 0x7f7145600084 in __libc_start_main (/usr/local/fbcode/gcc-4.9-glibc-2.20-fb/lib/libc.so.6+0x20084)
    #8 0x410be5 (/data/users/lucian/fbcode2/_build/opt/folly/io/async/test/async_test+0x410be5)

AddressSanitizer can not provide additional info.
AAAAAAASUMMARY: AddressSanitizer: SEGV third-party2/libgcc/4.9.x/gcc-4.9-glibc-2.20-fb/024dbc3/include/c++/4.9.x/bits/stl_tree.h:1138 std::_Rb_tree<int, std::pair<int const, folly::SSLContext::SSLLockType>, std::_Select1st<std::pair<int const, folly::SSLContext::SSLLockType> >, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > >::operator=(std::_Rb_tree<int, std::pair<int const, folly::SSLContext::SSLLockType>, std::_Select1st<std::pair<int const, folly::SSLContext::SSLLockType> >, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > > const&)
==3245135==ABORTING
```

Reviewed By: @philippv

Differential Revision: D2440796
korovkin pushed a commit to korovkin/folly that referenced this pull request Oct 1, 2015
Summary: avoid sillyness:

```
$ _build/opt/folly/io/async/test/async_test --gtest_list_tests
ASAN:SIGSEGV
=================================================================
==3245135==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000010 (pc 0x000000583444 sp 0x7fff17ba0c40 bp 0x7fff17ba0c80 T0)
    #0 0x583443 in std::_Rb_tree<int, std::pair<int const, folly::SSLContext::SSLLockType>, std::_Select1st<std::pair<int const, folly::SSLContext::SSLLockType> >, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > >::operator=(std::_Rb_tree<int, std::pair<int const, folly::SSLContext::SSLLockType>, std::_Select1st<std::pair<int const, folly::SSLContext::SSLLockType> >, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > > const&) third-party2/libgcc/4.9.x/gcc-4.9-glibc-2.20-fb/024dbc3/include/c++/4.9.x/bits/stl_tree.h:1138
    facebook#1 0x583443 in std::map<int, folly::SSLContext::SSLLockType, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > >::operator=(std::map<int, folly::SSLContext::SSLLockType, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > > const&) third-party2/libgcc/4.9.x/gcc-4.9-glibc-2.20-fb/024dbc3/include/c++/4.9.x/bits/stl_map.h:295
    facebook#2 0x583443 in folly::SSLContext::setSSLLockTypes(std::map<int, folly::SSLContext::SSLLockType, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > >) folly/io/async/SSLContext.cpp:682
    facebook#3 0x40e028 in Initializer folly/io/async/test/AsyncSSLSocketTest2.cpp:143
    facebook#4 0x40e028 in __static_initialization_and_destruction_0 folly/io/async/test/AsyncSSLSocketTest2.cpp:146
    facebook#5 0x40e028 in _GLOBAL__sub_I__ZN5folly47AsyncSSLSocketTest2_AttachDetachSSLContext_Test10test_info_E folly/io/async/test/AsyncSSLSocketTest2.cpp:147
    facebook#6 0x66cf2e in __libc_csu_init /home/engshare/third-party2/glibc/2.20/src/glibc-2.20/csu/elf-init.c:88
    facebook#7 0x7f7145600084 in __libc_start_main (/usr/local/fbcode/gcc-4.9-glibc-2.20-fb/lib/libc.so.6+0x20084)
    facebook#8 0x410be5 (/data/users/lucian/fbcode2/_build/opt/folly/io/async/test/async_test+0x410be5)

AddressSanitizer can not provide additional info.
AAAAAAASUMMARY: AddressSanitizer: SEGV third-party2/libgcc/4.9.x/gcc-4.9-glibc-2.20-fb/024dbc3/include/c++/4.9.x/bits/stl_tree.h:1138 std::_Rb_tree<int, std::pair<int const, folly::SSLContext::SSLLockType>, std::_Select1st<std::pair<int const, folly::SSLContext::SSLLockType> >, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > >::operator=(std::_Rb_tree<int, std::pair<int const, folly::SSLContext::SSLLockType>, std::_Select1st<std::pair<int const, folly::SSLContext::SSLLockType> >, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > > const&)
==3245135==ABORTING
```

Reviewed By: @philippv

Differential Revision: D2440796
ghost pushed a commit that referenced this pull request Apr 28, 2016
Summary:
Use tag dispatching instead of `enable_if`: it is clearer, it
sidesteps the GCC mangling bug, and more importantly the conditional
doesn't leak into the symbol, making stack traces and profiles more
readable.

Testing on a compilation unit with 1000 `Function`s from simple lambdas.
Before:
```
folly::impl::Function<int (), false>::Function<main::{lambda()#1}, {lambda()#1}>(main::{lambda()#1}&&, std::enable_if<std::integral_constant<bool, ((sizeof (std::decay<main::{lambda()#1}>::type))<=(sizeof folly::detail::function::Data::small))&&std::is_nothrow_move_constructible<std::decay<main::{lambda()#1}> >::value>::value, folly::detail::Tag>::type)::Ops::call(folly::detail::function&)
```
After:
```
folly::impl::Function<int (), false>::OpsSmall<main::{lambda()#1}>::call(folly::detail::function::Data&)
```
Note that the function type is repeated 5 times before, and only once after. This makes a large difference with long namespaces.

Binary size is almost unaffected, compile times slightly better:
Before:
GCC opt: 22.3 seconds, 4435232 bytes
Clang dev: 7.7 seconds, 5257344 bytes

After:
GCC opt: 18.6 seconds, 4493920 bytes
Clang dev: 7.2 seconds, 5469136 bytes

Reviewed By: ericniebler

Differential Revision: D3231530

fb-gh-sync-id: 6aa76e7f780a8afdbfed8a378f257ceb86dce704
fbshipit-source-id: 6aa76e7f780a8afdbfed8a378f257ceb86dce704
ghost pushed a commit that referenced this pull request Apr 28, 2016
Summary:
Currently you need to depend on the destructor of `ReadHolder` (using closures as in code block #1 below or empty assignment as in code block #2 below) to ensure that a `ReadHolder` goes out of scope (and unlocks) in order to subsequently acquire a write lock via `WriteHolder` without deadlocking.
This diff introduces a way of unlocking a `ReadHolder` while it's still in scope such that a `WriteHolder` can be acquired. This makes the code more straight forward (reducing the risk of deadlock due to a programmer's misunderstanding of how `SharedMutex` / the holder framework works) => see code block # 3 below
Also add some documentation about why `WriteHolder::WriteHolder(ReadHolder&&)` doesn't exist

Code Block #1 : Use of closures
```
class foo {
 public:
  std::string getMemoizedData() {
    {
      folly::SharedMutex::ReadHolder readHolder(lock_);
      if (!data_.empty()) {
        // important to return by value, otherwise caller might access
        // data_ after we release the read lock
        return data_;
      }
    }
    {
      // try again with a write lock
      folly::SharedMutex::WriteHolder writeHolder(lock_);
      if (data_.empty()) {
        data_ = "my awesome string";
      }
      return data_;
    }
  }

 private:
  folly::SharedMutex lock_;
  std::string data_;
};
```

Code Block #2 : Use of empty assignment
```
class foo {
 public:
  std::string getMemoizedData() {
    folly::SharedMutex::ReadHolder readHolder(lock_);
    if (!data_.empty()) {
      // important to return by value, otherwise caller might access
      // data_ after we release the read lock
      return data_;
    }
    readHolder = {};

    // try again with a write lock
    folly::SharedMutex::WriteHolder writeHolder(lock_);
    if (data_.empty()) {
      data_ = "my awesome string";
    }
    return data_;
  }

 private:
  folly::SharedMutex lock_;
  std::string data_;
};
```

Code Block #3 : Use of unlock()
```
class foo {
 public:
  std::string getMemoizedData() {
    folly::SharedMutex::ReadHolder readHolder(lock_);
    if (!data_.empty()) {
      // important to return by value, otherwise caller might access
      // data_ after we release the read lock
      return data_;
    }
    readHolder->unlock();

    // try again with a write lock
    folly::SharedMutex::WriteHolder writeHolder(lock_);
    if (data_.empty()) {
      data_ = "my awesome string";
    }
    return data_;
  }

 private:
  folly::SharedMutex lock_;
  std::string data_;
};
```

Reviewed By: yfeldblum

Differential Revision: D3176025

fb-gh-sync-id: c7d47ca71df08673c7c1f1fd5ed9e01a663c1797
fbshipit-source-id: c7d47ca71df08673c7c1f1fd5ed9e01a663c1797
ghost pushed a commit that referenced this pull request May 19, 2016
Summary:
```
buck-out/gen/folly/__default_headers__#default,headers/folly/FBString.h:365:7: runtime error: null pointer passed as argument 2, which is declared to never be null
third-party-buck/build/glibc/include/string.h:70:33: note: nonnull attribute specified here
    #0 0x433a39 in _ZZN5folly13fbstring_coreIcEC1EPKcmbENKUlvE_clEv ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////buck-out/gen/folly/__default_headers__#default,headers/folly/FBString.h:365:7
    #1 0x4335a9 in _ZN5folly14ScopeGuardImplIZNS_13fbstring_coreIcEC1EPKcmbEUlvE_E7executeEv ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////buck-out/gen/folly/__default_headers__#default,headers/folly/ScopeGuard.h:153:29
    #2 0x4335a9 in _ZN5folly14ScopeGuardImplIZNS_13fbstring_coreIcEC1EPKcmbEUlvE_ED2Ev ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////buck-out/gen/folly/__default_headers__#default,headers/folly/ScopeGuard.h:130
    #3 0x4335a9 in folly::fbstring_core<char>::fbstring_core(char const*, unsigned long, bool) ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////buck-out/gen/folly/__default_headers__#default,headers/folly/FBString.h:427
    #4 0x4353fa in folly::basic_fbstring<char, std::char_traits<char>, std::allocator<char>, folly::fbstring_core<char> >::basic_fbstring(char const*, unsigned long, std::allocator<char> const&) ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////buck-out/gen/folly/__default_headers__#default,headers/folly/FBString.h:1038:9
```

Reviewed By: Gownta

Differential Revision: D3321266

fbshipit-source-id: 28d5aef16e91a98066a1de6bab95403fbc63eaab
@tian-yuan tian-yuan mentioned this pull request Jul 23, 2016
@tdauth tdauth mentioned this pull request Mar 3, 2017
facebook-github-bot pushed a commit that referenced this pull request Mar 31, 2017
Summary:
ubsan flags this problem:

```
buck-out/opt-ubsan/gen/folly/__default_headers__#header-mode-symlink-tree-with-header-map,headers/folly/experimental/StringKeyedCommon.h:31:18: runtime error: null pointer passed as argument 2, which is declared to never be null
third-party-buck/gcc-4.9-glibc-2.20-fb/build/glibc/include/string.h:47:28: note: nonnull attribute specified here
    #0 0x5cb88f in std::pair<std::__detail::_Node_iterator<std::pair<folly::Range<char const*> const, facebook::eden::GitIgnore>, false, true>, bool> folly::StringKeyedUnorderedMap<facebook::eden::GitIgnore, folly::Hash, std::equal_to<folly::Range<char const*> >, std::allocator<std::pair<folly::Range<char const*> const, facebook::eden::GitIgnore> > >::emplace<facebook::eden::GitIgnore>(folly::Range<char const*>, facebook::eden::GitIgnore&&) buck-out/opt-ubsan/gen/folly/__default_headers__#header-mode-symlink-tree-with-header-map,headers/folly/experimental/StringKeyedCommon.h:31
    #1 0x5c6652 in facebook::eden::(anonymous namespace)::IgnoreChecker::isIgnored(facebook::eden::detail::RelativePathBase<folly::Range<char const*> >) eden/fs/inodes/Dirstate.cpp:226
    #2 0x5c6037 in facebook::eden::(anonymous namespace)::IgnoreChecker::isIgnored(facebook::eden::detail::RelativePathBase<folly::Range<char const*> >) eden/fs/inodes/Dirstate.cpp:180
    #3 0x5c5173 in facebook::eden::Dirstate::getStatusForExistingDirectory(facebook::eden::detail::RelativePathBase<folly::Range<char const*> >) const eden/fs/inodes/Dirstate.cpp:372
    #4 0x5c4509 in facebook::eden::Dirstate::getStatus() const eden/fs/inodes/Dirstate.cpp:272
    #5 0x50a448 in verifyExpectedDirstate(facebook::eden::Dirstate const*, std::unordered_map<std::basic_fbstring<char, std::char_traits<char>, std::allocator<char>, std::fbstring_core<char> >, facebook::eden::StatusCode, std::hash<std::basic_fbstring<char, std::char_traits<char>, std::allocator<char>, std::fbstring_core<char> > >, std::equal_to<std::basic_fbstring<char, std::char_traits<char>, std::allocator<char>, std::fbstring_core<char> > >, std::allocator<std::pair<std::basic_fbstring<char, std::char_traits<char>, std::allocator<char>, std::fbstring_core<char> > const, facebook::eden::StatusCode> > >&&) eden/fs/inodes/test/DirstateTest.cpp:48
    #6 0x50c70c in Dirstate_addDirectoriesWithMixOfFiles_Test::TestBody() eden/fs/inodes/test/DirstateTest.cpp:220
    #7 0xb45857 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/engshare/third-party2/gtest/1.7.0/src/gtest-1.7.0/./src/gtest.cc:2364
    #8 0xb36784 in testing::Test::Run() /home/engshare/third-party2/gtest/1.7.0/src/gtest-1.7.0/./src/gtest.cc:2437
    #9 0xb36957 in testing::TestInfo::Run() [clone .part.558] /home/engshare/third-party2/gtest/1.7.0/src/gtest-1.7.0/./src/gtest.cc:2612
    #10 0xb36b74 in testing::TestCase::Run() [clone .part.559] /home/engshare/third-party2/gtest/1.7.0/src/gtest-1.7.0/./src/gtest.cc:2587
    #11 0xb3806e in testing::internal::UnitTestImpl::RunAllTests() [clone .part.561] /home/engshare/third-party2/gtest/1.7.0/src/gtest-1.7.0/./src/gtest.cc:4571
    #12 0xb382d9 in testing::UnitTest::Run() /home/engshare/third-party2/gtest/1.7.0/src/gtest-1.7.0/./src/gtest.cc:4519
    #13 0x861ca7 in main third-party-buck/gcc-4.9-glibc-2.20-fb/build/gtest/include/gtest/gtest.h:2326
    #14 0x7f20be0740f5 in __libc_start_main /home/engshare/third-party2/glibc/2.20/src/glibc-2.20/csu/libc-start.c:289
    #15 0x4a552c in _start /home/engshare/third-party2/glibc/2.20/src/glibc-2.20/csu/../sysdeps/x86_64/start.S:122

UndefinedBehaviorSanitizer: undefined-behavior buck-out/opt-ubsan/gen/folly/__default_headers__#header-mode-symlink-tree-with-header-map,headers/folly/experimental/StringKeyedCommon.h:31:18
```

The issue is that `StringPiece` default constructs to `{nullptr, nullptr}` as a valid representation of an empty string.
This is tripping up UBSAN in this case.

The fix is a trivial nullptr check

Reviewed By: igorsugak

Differential Revision: D4791015

fbshipit-source-id: dec7484b29ecb29c17b8dd6a9b0e8093f07d63cb
facebook-github-bot pushed a commit that referenced this pull request Jun 1, 2017
Summary:
Test output before the fix is applied:
```
folly/experimental/Bits.h:247:59: runtime error: signed integer overflow: -2147483648 - 1 cannot be represented in type 'int'
     #0 folly/experimental/Bits.h:247 folly::Bits<...>::set(int*, unsigned long, unsigned long, int)
     #1 folly/experimental/test/BitsTest.cpp:228 std::enable_if<...>::type (anonymous namespace)::testSet<...>(unsigned char*, unsigned long, unsigned long, int)
     #2 folly/experimental/test/BitsTest.cpp:263 Bits_Boundaries_Test::TestBody()
    #17 folly/experimental/test/BitsTest.cpp:381 main
```

Reviewed By: philippv

Differential Revision: D5160789

fbshipit-source-id: 43f1926d58f1a5c019d4f8794d10a7a80a5c4749
facebook-github-bot pushed a commit that referenced this pull request Oct 26, 2017
Summary:
Data race reported by TSAN:

```
WARNING: ThreadSanitizer: data race (pid=608219)
  Read of size 1 at 0x7b5800000c29 by thread T314:
    #0 0x60b3441 in folly::Codel::overloaded(std::chrono::duration<long, std::ratio<1l, 1000000000l> >) ./folly/executors/Codel.cpp:76
    #1 0x5c1222 in apache::thrift::concurrency::ThreadManager::ImplT<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::Worker<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::run() ./thrift/lib/cpp/concurrency/ThreadManager.tcc:119
    #2 0x5d803e7 in apache::thrift::concurrency::PthreadThread::threadMain(void*) ./thrift/lib/cpp/concurrency/PosixThreadFactory.cpp:200
    #3 0x619739d in __tsan_thread_start_func crtstuff.c:?

  Previous write of size 1 at 0x7b5800000c29 by thread T315:
    #0 0x60b33e4 in folly::Codel::overloaded(std::chrono::duration<long, std::ratio<1l, 1000000000l> >) ??:?
    #1 0x5c1222 in apache::thrift::concurrency::ThreadManager::ImplT<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::Worker<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::run() ./thrift/lib/cpp/concurrency/ThreadManager.tcc:119
    #2 0x5d803e7 in apache::thrift::concurrency::PthreadThread::threadMain(void*) ./thrift/lib/cpp/concurrency/PosixThreadFactory.cpp:200
    #3 0x619739d in __tsan_thread_start_func crtstuff.c:?

  Location is heap block of size 768 at 0x7b5800000c00 allocated by main thread:
    #0 0x616ab83 in operator new(unsigned long) ??:?
    #1 0x53cb92 in __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<apache::thrift::concurrency::SimpleThreadManager<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >, std::allocator<apache::thrift::concurrency::SimpleThreadManager<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > > >, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long, void const*)
    ...
```

It looks like there are multiple threads reading and writing the `overloaded_` bool. To fix it, wrap it in a `std::atomic`.

Reviewed By: yfeldblum, meyering

Differential Revision: D6149766

fbshipit-source-id: 605b29fa2f602d2ed2dfc22e46b739ef169f914e
@wumuzi520 wumuzi520 mentioned this pull request Feb 27, 2018
facebook-github-bot pushed a commit that referenced this pull request Oct 4, 2018
Summary:
TSAN flagged the following lock inversion below. The reason for this lock inversion
is because of the following example usage:

1. set request context B, save old one A (lock B, lock A)
2. later on, reset request context back to A, get the old one back B (lock A, lock B)

In the two calls, we always locked the the new context first, resulting
in a lock inversion. The fix is to always acquire them in a consistent
order using `folly::acquireLocked`

TSAN report:

```
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=1574307)
  Cycle in lock order graph: M159 (0x7b1400000138) => M157 (0x7b14000000e8) => M159

  Mutex M157 acquired here while holding mutex M159 in main thread:
    #0 0x48f18c in AnnotateRWLockAcquired ??:?
    #1 0x315495 in folly::SharedMutexImpl<false, void, std::atomic, false>::lock_shared() ./folly/SharedMutex.h:385
    #2 0x1b0eb8 in folly::detail::LockTraitsImpl<folly::SharedMutexImpl<false, void, std::atomic, false>, (folly::detail::MutexLevel)1, false>::lock_shared(folly::SharedMutexImpl<false, void, std::atomic, false>&) ./folly/LockTraits.h:157
    #3 0x1b0e68 in std::integral_constant<bool, true> folly::LockPolicyShared::lock<folly::SharedMutexImpl<false, void, std::atomic, false> >(folly::SharedMutexImpl<false, void, std::atomic, false>&) ./folly/LockTraits.h:493
    #4 0x1d63a9 in LockedPtrBase ./folly/Synchronized.h:1026
    #5 0x1d6258 in LockedPtr ./folly/Synchronized.h:1326
    #6 0x1c9a17 in folly::SynchronizedBase<folly::Synchronized<folly::RequestContext::State, folly::SharedMutexImpl<false, void, std::atomic, false> >, (folly::detail::MutexLevel)1>::rlock() const ./folly/Synchronized.h:123
    #7 0x1c7530 in folly::RequestContext::setContext(std::shared_ptr<folly::RequestContext>) ./folly/io/async/Request.cpp:243
    #8 0x36a7b in folly::RequestContext::create() ./folly/io/async/Request.h:119
    #9 0x34916 in RequestContextScopeGuard ./folly/io/async/Request.h:269

  Mutex M159 previously acquired by the same thread here:
    #0 0x48f18c in AnnotateRWLockAcquired ??:?
    #1 0x315495 in folly::SharedMutexImpl<false, void, std::atomic, false>::lock_shared() ./folly/SharedMutex.h:385
    #2 0x1b0eb8 in folly::detail::LockTraitsImpl<folly::SharedMutexImpl<false, void, std::atomic, false>, (folly::detail::MutexLevel)1, false>::lock_shared(folly::SharedMutexImpl<false, void, std::atomic, false>&) ./folly/LockTraits.h:157
    #3 0x1b0e68 in std::integral_constant<bool, true> folly::LockPolicyShared::lock<folly::SharedMutexImpl<false, void, std::atomic, false> >(folly::SharedMutexImpl<false, void, std::atomic, false>&) ./folly/LockTraits.h:493
    #4 0x1d63a9 in LockedPtrBase ./folly/Synchronized.h:1026
    #5 0x1d6258 in LockedPtr ./folly/Synchronized.h:1326
    #6 0x1c9a17 in folly::SynchronizedBase<folly::Synchronized<folly::RequestContext::State, folly::SharedMutexImpl<false, void, std::atomic, false> >, (folly::detail::MutexLevel)1>::rlock() const ./folly/Synchronized.h:123
    #7 0x1c7516 in folly::RequestContext::setContext(std::shared_ptr<folly::RequestContext>) ./folly/io/async/Request.cpp:242
    #8 0x36a7b in folly::RequestContext::create() ./folly/io/async/Request.h:119
    #9 0x34916 in RequestContextScopeGuard ./folly/io/async/Request.h:269

  Mutex M159 acquired here while holding mutex M157 in main thread:
    #0 0x48f18c in AnnotateRWLockAcquired ??:?
    #1 0x315495 in folly::SharedMutexImpl<false, void, std::atomic, false>::lock_shared() ./folly/SharedMutex.h:385
    #2 0x1b0eb8 in folly::detail::LockTraitsImpl<folly::SharedMutexImpl<false, void, std::atomic, false>, (folly::detail::MutexLevel)1, false>::lock_shared(folly::SharedMutexImpl<false, void, std::atomic, false>&) ./folly/LockTraits.h:157
    #3 0x1b0e68 in std::integral_constant<bool, true> folly::LockPolicyShared::lock<folly::SharedMutexImpl<false, void, std::atomic, false> >(folly::SharedMutexImpl<false, void, std::atomic, false>&) ./folly/LockTraits.h:493
    #4 0x1d63a9 in LockedPtrBase ./folly/Synchronized.h:1026
    #5 0x1d6258 in LockedPtr ./folly/Synchronized.h:1326
    #6 0x1c9a17 in folly::SynchronizedBase<folly::Synchronized<folly::RequestContext::State, folly::SharedMutexImpl<false, void, std::atomic, false> >, (folly::detail::MutexLevel)1>::rlock() const ./folly/Synchronized.h:123
    #7 0x1c7530 in folly::RequestContext::setContext(std::shared_ptr<folly::RequestContext>) ./folly/io/async/Request.cpp:243
    #8 0x34b99 in ~RequestContextScopeGuard ./folly/io/async/Request.h:278

  Mutex M157 previously acquired by the same thread here:
    #0 0x48f18c in AnnotateRWLockAcquired ??:?
    #1 0x315495 in folly::SharedMutexImpl<false, void, std::atomic, false>::lock_shared() ./folly/SharedMutex.h:385
    #2 0x1b0eb8 in folly::detail::LockTraitsImpl<folly::SharedMutexImpl<false, void, std::atomic, false>, (folly::detail::MutexLevel)1, false>::lock_shared(folly::SharedMutexImpl<false, void, std::atomic, false>&) ./folly/LockTraits.h:157
    #3 0x1b0e68 in std::integral_constant<bool, true> folly::LockPolicyShared::lock<folly::SharedMutexImpl<false, void, std::atomic, false> >(folly::SharedMutexImpl<false, void, std::atomic, false>&) ./folly/LockTraits.h:493
    #4 0x1d63a9 in LockedPtrBase ./folly/Synchronized.h:1026
    #5 0x1d6258 in LockedPtr ./folly/Synchronized.h:1326
    #6 0x1c9a17 in folly::SynchronizedBase<folly::Synchronized<folly::RequestContext::State, folly::SharedMutexImpl<false, void, std::atomic, false> >, (folly::detail::MutexLevel)1>::rlock() const ./folly/Synchronized.h:123
    #7 0x1c7516 in folly::RequestContext::setContext(std::shared_ptr<folly::RequestContext>) ./folly/io/async/Request.cpp:242
    #8 0x34b99 in ~RequestContextScopeGuard ./folly/io/async/Request.h:278

```

Reviewed By: igorsugak

Differential Revision: D9797715

fbshipit-source-id: a0f991ae0ec8f3e9d33af6e05db49eceaf3a5174
facebook-github-bot pushed a commit that referenced this pull request Jan 18, 2019
Summary:
This is mostly a POC. It will probably fail in some cases, but it's better than nothing.
Sample output:
  (gdb) co_bt this
  0x292d70 <zero()>
  0x293f50 <one()>
  0x295050 <two()>
  0x296150 <three()>
  0x297250 <folly::coro::TaskWithExecutor<int>::start() &&::{lambda(folly::Promise<int>, folly::coro::TaskWithExecutor<int>)#1}::operator()(folly::Promise<int>, folly::coro::TaskWithExecutor<int>) const>

Reviewed By: jwiepert

Differential Revision: D13727139

fbshipit-source-id: bff98eb4f5eb2ebd73c880d3b525172782f87511
facebook-github-bot pushed a commit that referenced this pull request Oct 23, 2019
Summary:
MemorySanitizer was failing due to uninitialized memory in the stack.

```
==596112==WARNING: MemorySanitizer: use-of-uninitialized-value
[testserver][INFO]: IOBuff received. Len = 128
    #0 0x9d4b21 in folly::SocketAddress::updateUnixAddressLength(unsigned int) xplat/folly/SocketAddress.cpp:710:7
    #1 0x9d4740 in folly::SocketAddress::setFromSockaddr(sockaddr_un const*, unsigned int) xplat/folly/SocketAddress.cpp:338:3

  Uninitialized value was created by an allocation of 'addrStorage' in the stack frame of function '_ZN5folly17AsyncServerSocket12handler
ReadyEtNS_13NetworkSocketEt'
    #0 0x7de840 in folly::AsyncServerSocket::handlerReady(unsigned short, folly::NetworkSocket, unsigned short) xplat/folly/io/async/Asyn
cServerSocket.cpp:835
```

Differential Revision: D18086738

fbshipit-source-id: f36013b157d7b34d3b29c08d4533f01be1a1a8c9
facebook-github-bot pushed a commit that referenced this pull request Feb 1, 2020
Summary:
Until C++17 types with extended alignment need special handling
in eg. std::vector and also std::unique_ptr. CPUThreadPoolExecutor has a
default queue that requires extended alignment, but also allows the user
to provide their own blocking queue. To handle this we move the private
member to a shared_ptr which supported type erased destructors, and then
use allocate_shared for the default queue type with a custom allocator
that will satisfy alignment constraints.

```
.../unique_ptr.h:825:34: runtime error: constructor call on misaligned address 0x613000000040 for type 'folly::UnboundedBlockingQueue<folly::CPUThreadPoolExecutor::CPUTask>', which requires 128 byte alignment
0x613000000040: note: pointer points here
 02 00 00 1c  be be be be be be be be  be be be be be be be be  be be be be be be be be  be be be be
              ^
    #0 0x75b35e in std::_MakeUniq<folly::UnboundedBlockingQueue<folly::CPUThreadPoolExecutor::CPUTask> >::__single_object std::make_unique<folly::UnboundedBlockingQueue<folly::CPUThreadPoolExecutor::CPUTask> >() .../unique_ptr.h:825:30
    #1 0x75602f in folly::CPUThreadPoolExecutor::CPUThreadPoolExecutor(unsigned long, std::shared_ptr<folly::ThreadFactory>) xplat/folly/executors/CPUThreadPoolExecutor.cpp:66:18
    #2 0x756c6c in folly::CPUThreadPoolExecutor::CPUThreadPoolExecutor(unsigned long) xplat/folly/executors/CPUThreadPoolExecutor.cpp:84:7
```

Differential Revision: D19669832

fbshipit-source-id: 7fb60a06dbb6a04edddb6b619af7a8cef4995fd2
facebook-github-bot pushed a commit that referenced this pull request Jul 9, 2020
Summary:
Use sized-deallocation (`sdallocx`) if possible in `folly::SysAllocator` and `folly::Arena`.

`Arena` has always allocated two types of blocks:
1. Normal (fixed-sized): size is the "goodSize adjusted" `minBlockSize`
2. Large (variable-sized): when #1 is too small

Type #2 makes sized-deallocation tricky -- we need somewhere to remember the allocated sizes.

The old code used a single type `Block` and kept a single list. Here I change to have two types and two lists.  The `LargeBlock` has an additional `allocSize` data member.

This makes the Arena object itself 16B larger, but seems better than adding a 4B `allocSize` to each and every block, regardless of type.

Note that, prior to this change, it was possible to `merge()` arenas with different `minBlockSize`.  This is no longer possible.

Reviewed By: yfeldblum

Differential Revision: D22189916

fbshipit-source-id: e6fba48eaae0b5cc8456b856b02d2cfc71c03834
facebook-github-bot pushed a commit that referenced this pull request Jul 10, 2020
Summary:
Use sized-deallocation (`sdallocx`) if possible in `folly::SysAllocator` and `folly::Arena`.

`Arena` has always allocated two types of blocks:
1. Normal (fixed-sized): size is the "goodSize adjusted" `minBlockSize`
2. Large (variable-sized): when #1 is too small

Type #2 makes sized-deallocation tricky -- we need somewhere to remember the allocated sizes.

The old code used a single type `Block` and kept a single list. Here I change to have two types and two lists.  The `LargeBlock` has an additional `allocSize` data member.

This makes the Arena object itself 16B larger, but seems better than adding a 4B `allocSize` to each and every block, regardless of type.

Note that, prior to this change, it was possible to `merge()` arenas with different `minBlockSize`.  This is no longer possible.

Reviewed By: davidtgoldblatt

Differential Revision: D22466906

fbshipit-source-id: 719f357a0a1f6cfcda3208391837195c3859ab69
machogallo2 pushed a commit to machogallo2/folly that referenced this pull request Sep 5, 2020
dotconnor pushed a commit to 5448C8/folly that referenced this pull request Mar 19, 2021
Summary:
I have been encountering

```
AddressSanitizer:DEADLYSIGNAL
=================================================================
==937752==ERROR: AddressSanitizer: SEGV on unknown address 0x24a12000e4f18 (pc 0x7f3902a6e696 bp 0x7f37bc5e4380 sp 0x7f37bc5e42a0 T107)
==937752==The signal is caused by a READ memory access.
SCARINESS: 20 (wild-addr-read)

    #0 0x7f3902a6e695 in folly::ThreadLocalPtr<folly::SingletonThreadLocal<folly::hazptr_tc<std::atomic>, void, folly::detail::DefaultMake<folly::hazptr_tc<std::atomic> >, void>::Wrapper, void, void>::get() const folly/ThreadLocal.h:160
    facebook#1 0x7f3902a6d600 in get folly/ThreadLocal.h:69
    facebook#2 0x7f3902a6d600 in folly::ThreadLocal<folly::SingletonThreadLocal<folly::hazptr_tc<std::atomic>, void, folly::detail::DefaultMake<folly::hazptr_tc<std::atomic> >, void>::Wrapper, void, void>::operator*() const folly/ThreadLocal.h:78
    facebook#3 0x7f3902a6ca1b in folly::SingletonThreadLocal<folly::hazptr_tc<std::atomic>, void, folly::detail::DefaultMake<folly::hazptr_tc<std::atomic> >, void>::getWrapper() folly/SingletonThreadLocal.h:149
    facebook#4 0x7f3902a6cbe5 in folly::SingletonThreadLocal<folly::hazptr_tc<std::atomic>, void, folly::detail::DefaultMake<folly::hazptr_tc<std::atomic> >, void>::LocalLifetime::~LocalLifetime() folly/SingletonThreadLocal.h:121
    facebook#5 0x7f38f04d9fd5 in (anonymous namespace)::run(void*) /data/users/kirkshoop/gcc/trunk/libstdc++-v3/libsupc++/atexit_thread.cc:75:16
    facebook#6 0x7f38ef9ee551 in __nptl_deallocate_tsd.part.8 /home/engshare/third-party2/glibc/2.26/src/glibc-2.26/nptl/pthread_create.c:301:8
    facebook#7 0x7f38ef9ef7c8 in __nptl_deallocate_tsd /home/engshare/third-party2/glibc/2.26/src/glibc-2.26/sysdeps/nptl/futex-internal.h:200:3
    facebook#8 0x7f38ef9ef7c8 in start_thread /home/engshare/third-party2/glibc/2.26/src/glibc-2.26/nptl/pthread_create.c:475:3
```

I tried using a unique Tag for the SingletonThreadLocal for debugging purposes and this seemed to fix the issue.

Reviewed By: magedm

Differential Revision: D19356179

fbshipit-source-id: a18c6b02f0a40e0509753075614eeff71d859bf7
dotconnor pushed a commit to 5448C8/folly that referenced this pull request Mar 19, 2021
Summary:
Until C++17 types with extended alignment need special handling
in eg. std::vector and also std::unique_ptr. CPUThreadPoolExecutor has a
default queue that requires extended alignment, but also allows the user
to provide their own blocking queue. To handle this we move the private
member to a shared_ptr which supported type erased destructors, and then
use allocate_shared for the default queue type with a custom allocator
that will satisfy alignment constraints.

```
.../unique_ptr.h:825:34: runtime error: constructor call on misaligned address 0x613000000040 for type 'folly::UnboundedBlockingQueue<folly::CPUThreadPoolExecutor::CPUTask>', which requires 128 byte alignment
0x613000000040: note: pointer points here
 02 00 00 1c  be be be be be be be be  be be be be be be be be  be be be be be be be be  be be be be
              ^
    #0 0x75b35e in std::_MakeUniq<folly::UnboundedBlockingQueue<folly::CPUThreadPoolExecutor::CPUTask> >::__single_object std::make_unique<folly::UnboundedBlockingQueue<folly::CPUThreadPoolExecutor::CPUTask> >() .../unique_ptr.h:825:30
    facebook#1 0x75602f in folly::CPUThreadPoolExecutor::CPUThreadPoolExecutor(unsigned long, std::shared_ptr<folly::ThreadFactory>) xplat/folly/executors/CPUThreadPoolExecutor.cpp:66:18
    facebook#2 0x756c6c in folly::CPUThreadPoolExecutor::CPUThreadPoolExecutor(unsigned long) xplat/folly/executors/CPUThreadPoolExecutor.cpp:84:7
```

Differential Revision: D19237477

fbshipit-source-id: c88b26e2ca17e168f124ba27c0989f787b1ce4fd
dotconnor pushed a commit to 5448C8/folly that referenced this pull request Mar 19, 2021
Summary:
The default allocator for `new` does not apparently respect the extended alignment requirement for `hasptr_rec<Atom>`. We use folly's AlignedSysAllocator for these cases, which will then pass UBSan checks for alignment.

```
xplat/folly/synchronization/HazptrDomain.h:572:20: runtime error: constructor call on misaligned address 0x60c0000004c0 for type 'hazptr_rec<atomic>', which requires 128 byte alignment
0x60c0000004c0: note: pointer points here
 02 00 00 7f  be be be be be be be be  be be be be be be be be  be be be be be be be be  be be be be
              ^
    #0 0x77949e in folly::hazptr_domain<std::atomic>::acquire_new_hprec() xplat/folly/synchronization/HazptrDomain.h:572:16
    facebook#1 0x7791cc in folly::hazptr_domain<std::atomic>::hprec_acquire() xplat/folly/synchronization/HazptrDomain.h:227:35
    facebook#2 0x766a7f in folly::hazptr_holder<std::atomic>::hazptr_holder(folly::hazptr_domain<std::atomic>&) xplat/folly/synchronization/HazptrHolder.h:71:21
    facebook#3 0x766a7f in void folly::UnboundedQueue<folly::CPUThreadPoolExecutor::CPUTask, false, false, false, 6ul, 7ul, std::atomic>::enqueueImpl<folly::CPUThreadPoolExecutor::CPUTask>(folly::CPUThreadPoolExecutor::CPUTask&&) xplat/folly/concurrency/UnboundedQueue.h:374
```

Reviewed By: magedm

Differential Revision: D19237973

fbshipit-source-id: 0edea12bb3f028d21830689d52f7e0290d187ff9
dotconnor pushed a commit to 5448C8/folly that referenced this pull request Mar 19, 2021
Summary:
Until C++17 types with extended alignment need special handling
in eg. std::vector and also std::unique_ptr. CPUThreadPoolExecutor has a
default queue that requires extended alignment, but also allows the user
to provide their own blocking queue. To handle this we move the private
member to a shared_ptr which supported type erased destructors, and then
use allocate_shared for the default queue type with a custom allocator
that will satisfy alignment constraints.

```
.../unique_ptr.h:825:34: runtime error: constructor call on misaligned address 0x613000000040 for type 'folly::UnboundedBlockingQueue<folly::CPUThreadPoolExecutor::CPUTask>', which requires 128 byte alignment
0x613000000040: note: pointer points here
 02 00 00 1c  be be be be be be be be  be be be be be be be be  be be be be be be be be  be be be be
              ^
    #0 0x75b35e in std::_MakeUniq<folly::UnboundedBlockingQueue<folly::CPUThreadPoolExecutor::CPUTask> >::__single_object std::make_unique<folly::UnboundedBlockingQueue<folly::CPUThreadPoolExecutor::CPUTask> >() .../unique_ptr.h:825:30
    facebook#1 0x75602f in folly::CPUThreadPoolExecutor::CPUThreadPoolExecutor(unsigned long, std::shared_ptr<folly::ThreadFactory>) xplat/folly/executors/CPUThreadPoolExecutor.cpp:66:18
    facebook#2 0x756c6c in folly::CPUThreadPoolExecutor::CPUThreadPoolExecutor(unsigned long) xplat/folly/executors/CPUThreadPoolExecutor.cpp:84:7
```

Differential Revision: D19669832

fbshipit-source-id: 7fb60a06dbb6a04edddb6b619af7a8cef4995fd2
dotconnor pushed a commit to 5448C8/folly that referenced this pull request Mar 19, 2021
Summary:
Use sized-deallocation (`sdallocx`) if possible in `folly::SysAllocator` and `folly::Arena`.

`Arena` has always allocated two types of blocks:
1. Normal (fixed-sized): size is the "goodSize adjusted" `minBlockSize`
2. Large (variable-sized): when facebook#1 is too small

Type facebook#2 makes sized-deallocation tricky -- we need somewhere to remember the allocated sizes.

The old code used a single type `Block` and kept a single list. Here I change to have two types and two lists.  The `LargeBlock` has an additional `allocSize` data member.

This makes the Arena object itself 16B larger, but seems better than adding a 4B `allocSize` to each and every block, regardless of type.

Note that, prior to this change, it was possible to `merge()` arenas with different `minBlockSize`.  This is no longer possible.

Reviewed By: yfeldblum

Differential Revision: D22189916

fbshipit-source-id: e6fba48eaae0b5cc8456b856b02d2cfc71c03834
dotconnor pushed a commit to 5448C8/folly that referenced this pull request Mar 19, 2021
Summary:
Use sized-deallocation (`sdallocx`) if possible in `folly::SysAllocator` and `folly::Arena`.

`Arena` has always allocated two types of blocks:
1. Normal (fixed-sized): size is the "goodSize adjusted" `minBlockSize`
2. Large (variable-sized): when facebook#1 is too small

Type facebook#2 makes sized-deallocation tricky -- we need somewhere to remember the allocated sizes.

The old code used a single type `Block` and kept a single list. Here I change to have two types and two lists.  The `LargeBlock` has an additional `allocSize` data member.

This makes the Arena object itself 16B larger, but seems better than adding a 4B `allocSize` to each and every block, regardless of type.

Note that, prior to this change, it was possible to `merge()` arenas with different `minBlockSize`.  This is no longer possible.

Reviewed By: davidtgoldblatt

Differential Revision: D22466906

fbshipit-source-id: 719f357a0a1f6cfcda3208391837195c3859ab69
facebook-github-bot pushed a commit that referenced this pull request May 10, 2022
Summary:
TSAN aborts on a data-race of the form:
```
WARNING: ThreadSanitizer: data race (pid=978596)
  Write of size 8 at 0x7b0400000b40 by thread T2:
    #0 operator delete(void*, unsigned long) <null>
    #1 folly::coro::timed_wait</*...*/>(/*...*/)::'lambda'(/*...*/)::operator()</*...*/>(/*...*/) const folly/experimental/coro/TimedWait.h:51
  Previous atomic write of size 1 at 0x7b0400000b40 by thread T1:
    #0 __tsan_atomic8_exchange <null>
    #1 std::atomic<bool>::exchange(bool, std::memory_order) libgcc/include/c++/bits/atomic_base.h:499
    #2 folly::coro::timed_wait</*...*/>(/*...*/)::'lambda'(/*...*/)::operator()</*...*/>(/*...*/) const folly/experimental/coro/TimedWait.h:64
```

Reviewed By: davidtgoldblatt, iahs

Differential Revision: D36159450

fbshipit-source-id: 860fcbbc1f689f9e166934f75a2ef60abdcad421
markwkern added a commit to markwkern/folly that referenced this pull request Aug 23, 2022
markwkern added a commit to markwkern/folly that referenced this pull request Aug 23, 2022
markwkern added a commit to markwkern/folly that referenced this pull request Aug 23, 2022
@zhngs zhngs mentioned this pull request Nov 21, 2022
facebook-github-bot pushed a commit that referenced this pull request Mar 8, 2023
Summary:
AsyncUDPSocket test cases are crashing when running under llvm15. During
debugging it seems that the issue is the fact that the code tries to allocated 0
size array. Changing the code to prevent such allocation.

This is not very clean why to fix, but I am not sure if there is better one.
Please let me know if you have any suggestions.

Sample crash:
```
$ buck test //folly/io/async/test:async_udp_socket_test
...
stdout:
Note: Google Test filter = AsyncUDPSocketTest.TestDetachAttach
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from AsyncUDPSocketTest
[ RUN      ] AsyncUDPSocketTest.TestDetachAttach

stderr:
fbcode/folly/io/async/AsyncUDPSocket.cpp:699:10: runtime error: variable length array bound evaluates to non-positive value 0
    #0 0x7f4d8ed93704 in folly::AsyncUDPSocket::writev(folly::SocketAddress const&, iovec const*, unsigned long, int) fbcode/folly/io/async/AsyncUDPSocket.cpp:698
    #1 0x7f4d8ed9081f in folly::AsyncUDPSocket::writeGSO(folly::SocketAddress const&, std::unique_ptr<folly::IOBuf, std::default_delete<folly::IOBuf>> const&, int) fbcode/folly/io/async/AsyncUDPSocket.cpp:528
    #2 0x7f4d8ed900b2 in folly::AsyncUDPSocket::write(folly::SocketAddress const&, std::unique_ptr<folly::IOBuf, std::default_delete<folly::IOBuf>> const&) fbcode/folly/io/async/AsyncUDPSocket.cpp:660
    #3 0x350a05 in AsyncUDPSocketTest_TestDetachAttach_Test::TestBody() fbcode/folly/io/async/test/AsyncUDPSocketTest.cpp:914
    #4 0x7f4d90dd1ad5 in testing::Test::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2682:50
    #5 0x7f4d90dd1ad5 in testing::Test::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2672:6
    #6 0x7f4d90dd1c64 in testing::TestInfo::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2861:14
    #7 0x7f4d90dd1c64 in testing::TestInfo::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2833:6
    #8 0x7f4d90dd2321 in testing::TestSuite::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:3015:31
    #9 0x7f4d90dd2321 in testing::TestSuite::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2993:6
    #10 0x7f4d90dd2b1e in testing::internal::UnitTestImpl::RunAllTests() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:5855:47
    #11 0x7f4d90dd1d87 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2665:29
    #12 0x7f4d90dd1d87 in testing::UnitTest::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:5438:55
    #13 0x7f4d90d5c990 in RUN_ALL_TESTS() fbcode/third-party-buck/platform010/build/googletest/include/gtest/gtest.h:2490
    #14 0x7f4d90d5c618 in main fbcode/common/gtest/LightMain.cpp:20
    #15 0x7f4d8ea2c656 in __libc_start_call_main /home/engshare/third-party2/glibc/2.34/src/glibc-2.34/csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #16 0x7f4d8ea2c717 in __libc_start_main@GLIBC_2.2.5 /home/engshare/third-party2/glibc/2.34/src/glibc-2.34/csu/../csu/libc-start.c:409:3
    #17 0x33ea60 in _start /home/engshare/third-party2/glibc/2.34/src/glibc-2.34/csu/../sysdeps/x86_64/start.S:116
...
```

Reviewed By: russoue, dmm-fb

Differential Revision: D43858875

fbshipit-source-id: 93749bab17027b6dfc0dbc01b6c183e501a5494c
facebook-github-bot pushed a commit that referenced this pull request Apr 27, 2024
Summary:
ThreadLocalDetailTest.MultiThreadedTest had a data race on std::vector because multiple threads were accessing and modifying the std::vector without a mutex. I update the code to use indexes such that the vector is not updated concurrently.

Previous run failure that this fixes:

```
stderr:
==================
WARNING: ThreadSanitizer: data race (pid=1249005)
  Write of size 8 at 0x7fff195936d0 by main thread:
    #0 std::vector<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>, std::allocator<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>>>::pop_back() fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/stl_vector.h:1228 (thread_local_detail_test+0x116c28)
    #1 folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody() fbcode/folly/detail/test/ThreadLocalDetailTest.cpp:121 (thread_local_detail_test+0x101aba)
    #2 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) fbsource/src/gtest.cc:2675 (libthird-party_googletest_1.14.0_gtest.so+0xacf3c)
    #3 testing::Test::Run() fbsource/src/gtest.cc:2692 (libthird-party_googletest_1.14.0_gtest.so+0xacb9c)
    #4 testing::TestInfo::Run() fbsource/src/gtest.cc:2841 (libthird-party_googletest_1.14.0_gtest.so+0xafa4a)
    #5 testing::TestSuite::Run() fbsource/src/gtest.cc:3020 (libthird-party_googletest_1.14.0_gtest.so+0xb4303)
    #6 testing::internal::UnitTestImpl::RunAllTests() fbsource/src/gtest.cc:5925 (libthird-party_googletest_1.14.0_gtest.so+0xd105e)
    #7 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) fbsource/src/gtest.cc:2675 (libthird-party_googletest_1.14.0_gtest.so+0xd08f1)
    #8 testing::UnitTest::Run() fbsource/src/gtest.cc:5489 (libthird-party_googletest_1.14.0_gtest.so+0xd037e)
    #9 RUN_ALL_TESTS() fbsource/gtest/gtest.h:2317 (libcommon_gtest_light_main.so+0x22a7)
    #10 main fbcode/common/gtest/LightMain.cpp:20 (libcommon_gtest_light_main.so+0x2131)

  Previous read of size 8 at 0x7fff195936d0 by thread T123:
    #0 std::vector<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>, std::allocator<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>>>::size() const fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/stl_vector.h:919 (thread_local_detail_test+0x11b0bd)
    #1 std::vector<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>, std::allocator<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>>>::operator[](unsigned long) fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/stl_vector.h:1045 (thread_local_detail_test+0x11d101)
    #2 folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0::operator()() const fbcode/folly/detail/test/ThreadLocalDetailTest.cpp:97 (thread_local_detail_test+0x11d08c)
    #3 void std::__invoke_impl<void, folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0>(std::__invoke_other, folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0&&) fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/invoke.h:61 (thread_local_detail_test+0x11cf95)
    #4 std::__invoke_result<folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0>::type std::__invoke<folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0>(folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0&&) fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/invoke.h:96 (thread_local_detail_test+0x11cf05)
    #5 void std::thread::_Invoker<std::tuple<folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0>>::_M_invoke<0ul>(std::_Index_tuple<0ul>) fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/std_thread.h:253 (thread_local_detail_test+0x11cebd)
    #6 std::thread::_Invoker<std::tuple<folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0>>::operator()() fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/std_thread.h:260 (thread_local_detail_test+0x11ce65)
    #7 std::thread::_State_impl<std::thread::_Invoker<std::tuple<folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0>>>::_M_run() fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/std_thread.h:211 (thread_local_detail_test+0x11cd79)
    #8 execute_native_thread_routine /home/engshare/third-party2/libgcc/11.x/src/gcc-11.x/x86_64-facebook-linux/libstdc++-v3/src/c++11/../../../.././libstdc++-v3/src/c++11/thread.cc:82:18 (libstdc++.so.6+0xdf4e4) (BuildId: 452d1cdae868baeeb2fdf1ab140f1c219bf50c6e)

  Location is stack of main thread.

  Location is global '??' at 0x7fff19575000 ([stack]+0x1e6d0)

  Thread T123 (tid=1249233, running) created by main thread at:
    #0 pthread_create <null> (thread_local_detail_test+0x156bef)
    #1 __gthread_create /home/engshare/third-party2/libgcc/11.x/src/gcc-11.x/x86_64-facebook-linux/libstdc++-v3/include/x86_64-facebook-linux/bits/gthr-default.h:663:35 (libstdc++.so.6+0xdf80e) (BuildId: 452d1cdae868baeeb2fdf1ab140f1c219bf50c6e)
    #2 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State>>, void (*)()) /home/engshare/third-party2/libgcc/11.x/src/gcc-11.x/x86_64-facebook-linux/libstdc++-v3/src/c++11/../../../.././libstdc++-v3/src/c++11/thread.cc:147:37 (libstdc++.so.6+0xdf80e)
    #3 folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody() fbcode/folly/detail/test/ThreadLocalDetailTest.cpp:93 (thread_local_detail_test+0x1015cd)
    #4 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) fbsource/src/gtest.cc:2675 (libthird-party_googletest_1.14.0_gtest.so+0xacf3c)
    #5 testing::Test::Run() fbsource/src/gtest.cc:2692 (libthird-party_googletest_1.14.0_gtest.so+0xacb9c)
    #6 testing::TestInfo::Run() fbsource/src/gtest.cc:2841 (libthird-party_googletest_1.14.0_gtest.so+0xafa4a)
    #7 testing::TestSuite::Run() fbsource/src/gtest.cc:3020 (libthird-party_googletest_1.14.0_gtest.so+0xb4303)
    #8 testing::internal::UnitTestImpl::RunAllTests() fbsource/src/gtest.cc:5925 (libthird-party_googletest_1.14.0_gtest.so+0xd105e)
    #9 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) fbsource/src/gtest.cc:2675 (libthird-party_googletest_1.14.0_gtest.so+0xd08f1)
    #10 testing::UnitTest::Run() fbsource/src/gtest.cc:5489 (libthird-party_googletest_1.14.0_gtest.so+0xd037e)
    #11 RUN_ALL_TESTS() fbsource/gtest/gtest.h:2317 (libcommon_gtest_light_main.so+0x22a7)
    #12 main fbcode/common/gtest/LightMain.cpp:20 (libcommon_gtest_light_main.so+0x2131)

ThreadSanitizer: data race fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/stl_vector.h:1228 in std::vector<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>, std::allocator<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>>>::pop_back()
==================
ThreadSanitizer: reported 1 warnings
```

Reviewed By: yfeldblum

Differential Revision: D56643303

fbshipit-source-id: cc364817ae79bc6418cc07faa35e1bcd21830c66
DenisYaroshevskiy added a commit to DenisYaroshevskiy/folly that referenced this pull request Jul 11, 2024
Summary:
Pull Request resolved: facebook#2249

We have a need for some common concepts that are used in many places.

This is a proposal, please let me know what you think.

Differential Revision: D59396081
DenisYaroshevskiy added a commit to DenisYaroshevskiy/folly that referenced this pull request Jul 12, 2024
Summary:
Pull Request resolved: facebook#2249

We have a need for some common concepts that are used in many places.

This is a proposal, please let me know what you think.

Differential Revision: D59396081
DenisYaroshevskiy added a commit to DenisYaroshevskiy/folly that referenced this pull request Jul 18, 2024
Summary:
Pull Request resolved: facebook#2249

We have a need for some common concepts that are used in many places.

This is a proposal, please let me know what you think.

Differential Revision: D59396081
DenisYaroshevskiy added a commit to DenisYaroshevskiy/folly that referenced this pull request Jul 23, 2024
Summary:
Pull Request resolved: facebook#2249

We have a need for some common concepts that are used in many places.

This is a proposal, please let me know what you think.

Differential Revision: D59396081
DenisYaroshevskiy added a commit to DenisYaroshevskiy/folly that referenced this pull request Jul 30, 2024
Summary:
Pull Request resolved: facebook#2249

We have a need for some common concepts that are used in many places.

This is a proposal, please let me know what you think.

Differential Revision: D59396081
DenisYaroshevskiy added a commit to DenisYaroshevskiy/folly that referenced this pull request Aug 1, 2024
Summary:
Pull Request resolved: facebook#2249

We have a need for some common concepts that are used in many places.

This is a proposal, please let me know what you think.

Differential Revision: D59396081
DenisYaroshevskiy added a commit to DenisYaroshevskiy/folly that referenced this pull request Aug 1, 2024
Summary:
Pull Request resolved: facebook#2249

We have a need for some common concepts that are used in many places.

This is a proposal, please let me know what you think.

Differential Revision: D59396081
DenisYaroshevskiy added a commit to DenisYaroshevskiy/folly that referenced this pull request Aug 2, 2024
Summary:
Pull Request resolved: facebook#2249

We have a need for some common concepts that are used in many places.

This is a proposal, please let me know what you think.

Differential Revision: D59396081
facebook-github-bot pushed a commit that referenced this pull request Aug 2, 2024
Summary:
Moving simdForEachAligning into folly/algorithm/simd.

Step 1. Just move the files.

Reviewed By: dmm-fb

Differential Revision: D59680837

fbshipit-source-id: ad04d5a881bf8dbda186d1fd5cb943dae011e7ac
facebook-github-bot pushed a commit that referenced this pull request Aug 5, 2024
Summary:
Pull Request resolved: #2249

We have a need for some common concepts that are used in many places.

This is a proposal, please let me know what you think.

Usages example:

`uncvref_same_as`:

https://www.internalfb.com/code/fbsource/[a71ce59323a15fd7559d6132c1f8e2d0256ca7cb]/fbcode/multifeed/entrepot/lib/zdb_manager/ZdbUtil.h?lines=237

`uncvref_instantiated_from`:

https://www.internalfb.com/code/fbsource/[b366209d2c108133a0545edc7943b9461323623b]/fbcode/multifeed/mfaction/MFHashSerializer.h?lines=38

Reviewed By: yfeldblum

Differential Revision: D59396081

fbshipit-source-id: 4912c2f9fe683f4d3b37fce43466c9281d03ce13
facebook-github-bot pushed a commit that referenced this pull request Aug 12, 2024
Summary:
Ucache with io_uring is crashing with this stack trace:

```
(lldb) bt
* thread #1, name = 'ucache', stop reason = signal SIGSEGV: address not mapped to object
  * frame #0: 0x0000000000000000
    frame #1: 0x00007f517ba44560 libc.so.6`__restore_rt at libc_sigaction.c:13
    frame #2: 0x000000000a16e249 ucache`folly::AsyncIoUringSocket::ReadSqe::callback(this=0x00007f498c5955e0, cqe=<unavailable>) at AsyncIoUringSocket.cpp:639
    frame #3: 0x000000000964196a ucache`folly::IoUringBackend::getActiveEvents(folly::IoUringBackend::WaitForEventsMode) [inlined] folly::IoSqeBase::internalCallback(this=<unavailable>, cqe=<unavailable>) at IoUringBackend.cpp:0
    frame #4: 0x000000000964195c ucache`folly::IoUringBackend::getActiveEvents(folly::IoUringBackend::WaitForEventsMode) [inlined] folly::IoUringBackend::internalProcessCqe(this=0x00007f4a8f989c00, maxGet=4294967295, mode=NORMAL) at IoUringBackend.cpp:1556
    frame #5: 0x000000000964190f ucache`folly::IoUringBackend::getActiveEvents(this=0x00007f4a8f989c00, waitForEvents=<unavailable>) at IoUringBackend.cpp:1527
    frame #6: 0x0000000009640ac9 ucache`folly::IoUringBackend::eb_event_base_loop(this=0x00007f4a8f989c00, flags=1) at IoUringBackend.cpp:1178
    frame #7: 0x000000001b61547c ucache`folly::EventBase::loopMain(this=0x00007f4a8f97af00, flags=0, options=<unavailable>) at EventBase.cpp:0
    frame #8: 0x000000001bab9a2d ucache`folly::EventBase::loopBody(this=0x00007f4a8f97af00, flags=0, options=(ignoreKeepAlive = false, allowSuspension = false)) at EventBase.cpp:513
    frame #9: 0x000000000a3a6c90 ucache`folly::EventBase::loop(this=<unavailable>) at EventBase.cpp:474
    frame #10: 0x000000000a3a75f7 ucache`folly::EventBase::loopForever(this=<unavailable>) at EventBase.cpp:781
    frame #11: 0x000000000a436907 ucache`folly::IOThreadPoolExecutor::threadRun(this=0x00007f4da3fce1c0, thread=<unavailable>) at IOThreadPoolExecutor.cpp:240
    frame #12: 0x0000000009ed68b8 ucache`void std::__invoke_impl<void, void (folly::ThreadPoolExecutor::*&)(std::shared_ptr<folly::ThreadPoolExecutor::Thread>), folly::ThreadPoolExecutor*&, std::shared_ptr<folly::ThreadPoolExecutor::Thread>&>((null)=<unavailable>, __f=<unavailable>, __t=<unavailable>, __args=<unavailable>) at invoke.h:74
    frame #13: 0x00007f517b6df4e5 libstdc++.so.6`std::execute_native_thread_routine(__p=0x00007f4da01bdbe0) at thread.cc:82:18
    frame #14: 0x00007f517ba9abc9 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:434:8
    frame #15: 0x00007f517bb2cf5c libc.so.6`__clone3 at clone3.S:81
```

In frame 2 the offending function is `ReadSqe::callback()`: https://fburl.com/code/4fb46hdq

The problem is that `readCallback_` is null:

```
(lldb) fr v readCallback_
(folly::AsyncReader::ReadCallback *) readCallback_ = nullptr
```

However, `readCallback_` is checked earlier in the the function `ReadSqe::callback()`: https://fburl.com/code/dhnndk29

`AsyncIoUringSocket::readError()` now fails all pending writes and calls `writeErr()`, whose implementation can then close out a socket and call `AsyncIoUringSocket::setReadCB(nullptr)` to change the callback.

Move the call to `AsyncIoUringSocket::readError()` after the call to `readCallback_`.

Reviewed By: dmm-fb

Differential Revision: D60880377

fbshipit-source-id: 42d2f430cd3dc3f4787d0e4c29da7c8ef31ba9a8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants