Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[YSQL][PgstatCollector][TSan] ThreadSanitizer: data race in StatementCancelHandler #11701

Open
def- opened this issue Mar 9, 2022 · 0 comments
Assignees
Labels
area/ysql Yugabyte SQL (YSQL) kind/bug This issue is a bug priority/medium Medium priority issue qa_automation Bugs identified via itest-system, LST, Stress automation or causing automation failures

Comments

@def-
Copy link
Contributor

def- commented Mar 9, 2022

Jira Link: DB-824

Description

Many occurences of this, might be related to #11700 :

WARNING: ThreadSanitizer: data race (pid=14855)
  Write of size 1 at 0x000001caaa20 by thread T1:
    #0 StatementCancelHandler <null> (postgres+0xa482d9)
    #1 __tsan::CallUserSignalHandler(__tsan::ThreadState*, bool, bool, bool, int, __sanitizer::__sanitizer_siginfo*, void*) <null> (postgres+0x4ca7b0)
    #2 boost::asio::detail::epoll_reactor::run(long, boost::asio::detail::op_queue<boost::asio::detail::scheduler_operation>&) <null> (libyrpc.so+0x1aeec5)
    #3 non-virtual thunk to boost::asio::detail::epoll_reactor::run(long, boost::asio::detail::op_queue<boost::asio::detail::scheduler_operation>&) <null> (libyrpc.so+0x1af0dd)
    #4 boost::asio::detail::scheduler::do_run_one(boost::asio::detail::conditionally_enabled_mutex::scoped_lock&, boost::asio::detail::scheduler_thread_info&, boost::system::error_code const&) <null> (libyrpc.so+0x1b1445)
    #5 boost::asio::detail::scheduler::run(boost::system::error_code&) <null> (libyrpc.so+0x1b1104)
    #6 yb::rpc::IoThreadPool::Impl::Execute() <null> (libyrpc.so+0x1b3ada)
    #7 malloc <null> (libyrpc.so+0x1ac551)
    #8 main <null> (libyrpc.so+0x1b3088)
    #9 malloc <null> (libyrpc.so+0x1b2f8d)
    #10 std::__1::function<void ()>::operator()() const <null> (libserver_process.so+0x96004)
    #11 malloc <null> (libyb_util.so+0x3d7389)
    #12 PgstatCollectorMain <null> (postgres+0x4c4052)

  Previous read of size 1 at 0x000001caaa20 by main thread:
    [failed to restore the stack]

  Location is global 'InterruptPending' of size 1 at 0x000001caaa20 (postgres+0x000001caaa20)

  Thread T1 'iotp_pggate_ybc' (tid=14862, running) created by main thread at:
    #0 pthread_create <null> (postgres+0x4c40d5)
    #1 yb::Thread::StartThread(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::function<void ()>, scoped_refptr<yb::Thread>*) <null> (libyb_util.so+0x3d6b93)
    #2 yb::Result<scoped_refptr<yb::Thread> > yb::Thread::Make<std::__1::__bind<void (yb::rpc::IoThreadPool::Impl::*)(), yb::rpc::IoThreadPool::Impl*> >(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::__bind<void (yb::rpc::IoThreadPool::Impl::*)(), yb::rpc::IoThreadPool::Impl*>&&) <null> (libyrpc.so+0x1ac3c0)
    #3 yb::rpc::IoThreadPool::Impl::Impl(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, unsigned long) <null> (libyrpc.so+0x1abd8c)
    #4 yb::rpc::IoThreadPool::IoThreadPool(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, unsigned long) <null> (libyrpc.so+0x1abb07)
    #5 yb::rpc::Messenger::Messenger(yb::rpc::MessengerBuilder const&) <null> (libyrpc.so+0x1b7e57)
    #6 yb::rpc::MessengerBuilder::Build() <null> (libyrpc.so+0x1b4a4a)
    #7 PostgresServerProcessMain <null> (libyb_client.so+0x3e97dc)
    #8 yb::pggate::(anonymous namespace)::BuildMessenger(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, scoped_refptr<yb::MetricEntity> const&, std::__1::shared_ptr<yb::MemTracker> const&) <null> (libyb_pggate.so+0xcc931)
    #9 yb::pggate::PgApiContext::PgApiContext() <null> (libyb_pggate.so+0xcc5f7)
    #10 yb::pggate::YBCInitPgGateEx(PgTypeEntity const*, int, PgCallbacks, yb::pggate::PgApiContext*) <null> (libyb_pggate.so+0xba798)
    #11 YBCInitPgGate <null> (libyb_pggate.so+0xbaa37)
    #12 YBInitPostgresBackend <null> (postgres+0xc2b3ec)
    #13 InitPostgres <null> (postgres+0xc08f5c)
    #14 PostgresMain <null> (postgres+0xa499aa)
    #15 BackendRun <null> (postgres+0x9804b5)
    #16 BackendStartup <null> (postgres+0x97f8b6)
    #17 ServerLoop <null> (postgres+0x97e6b0)
    #18 PostmasterMain <null> (postgres+0x97b693)
    #19 PostgresServerProcessMain <null> (postgres+0x88b017)
    #20 main <null> (postgres+0x88b421)

Another one:

WARNING: ThreadSanitizer: data race (pid=14855)
  Read of size 4 at 0x7fe21a843074 by thread T1:
    #0 SetLatch <null> (postgres+0xa0dccf)
    #1 StatementCancelHandler <null> (postgres+0xa48307)
    #2 __tsan::CallUserSignalHandler(__tsan::ThreadState*, bool, bool, bool, int, __sanitizer::__sanitizer_siginfo*, void*) <null> (postgres+0x4ca7b0)
    #3 boost::asio::detail::epoll_reactor::run(long, boost::asio::detail::op_queue<boost::asio::detail::scheduler_operation>&) <null> (libyrpc.so+0x1aeec5)
    #4 non-virtual thunk to boost::asio::detail::epoll_reactor::run(long, boost::asio::detail::op_queue<boost::asio::detail::scheduler_operation>&) <null> (libyrpc.so+0x1af0dd)
    #5 boost::asio::detail::scheduler::do_run_one(boost::asio::detail::conditionally_enabled_mutex::scoped_lock&, boost::asio::detail::scheduler_thread_info&, boost::system::error_code const&) <null> (libyrpc.so+0x1b1445)
    #6 boost::asio::detail::scheduler::run(boost::system::error_code&) <null> (libyrpc.so+0x1b1104)
    #7 boost::asio::io_context::run(boost::system::error_code&) <null> (libyrpc.so+0x1b3ada)
    #8 yb::rpc::IoThreadPool::Impl::Execute() <null> (libyrpc.so+0x1ac551)
    #9 boost::asio::detail::scheduler::run(boost::system::error_code&) <null> (libyrpc.so+0x1b3088)
    #10 std::__1::__function::__func<std::__1::__bind<std::__1::__bind<void (yb::rpc::IoThreadPool::Impl::*)(), yb::rpc::IoThreadPool::Impl*>>, std::__1::allocator<std::__1::__bind<std::__1::__bind<void (yb::rpc::IoThreadPool::Impl::*)(), yb::rpc::IoThreadPool::Impl*>> >, void ()>::operator()() <null> (libyrpc.so+0x1b2f8d)
    #11 std::__1::function<void ()>::operator()() const <null> (libserver_process.so+0x96004)
    #12 yb::Thread::SuperviseThread(void*) <null> (libyb_util.so+0x3d7389)
    #13 ServerLoop <null> (postgres+0x4c4052)

  Previous write of size 4 at 0x7fe21a843074 by main thread:
    [failed to restore the stack]

  Location is global 'yb::rpc::Messenger::Messenger(yb::rpc::MessengerBuilder const&)' at 0x7fe2116cf000 (zero (deleted)+0x000009174074)

  Thread T1 'iotp_pggate_ybc' (tid=14862, running) created by main thread at:
    #0 pthread_create <null> (postgres+0x4c40d5)
    #1 PostgresServerProcessMain <null> (libyb_util.so+0x3d6b93)
    #2 main <null> (libyrpc.so+0x1ac3c0)
    #3 free <null> (libyrpc.so+0x1abd8c)
    #4 __fopen_internal <null> (libyrpc.so+0x1abb07)
    #5 pgstat_read_statsfiles <null> (libyrpc.so+0x1b7e57)
    #6 PgstatCollectorMain <null> (libyrpc.so+0x1b4a4a)
    #7 yb::pggate::PgApiContext::PgApiContext() <null> (libyb_client.so+0x3e97dc)
    #8 pgstat_start <null> (libyb_pggate.so+0xcc931)
    #9 yb::pggate::YBCInitPgGateEx(PgTypeEntity const*, int, PgCallbacks, yb::pggate::PgApiContext*) <null> (libyb_pggate.so+0xcc5f7)
    #10 boost::asio::detail::epoll_reactor::run(long, boost::asio::detail::op_queue<boost::asio::detail::scheduler_operation>&) <null> (libyb_pggate.so+0xba798)
    #11 YBCInitPgGate <null> (libyb_pggate.so+0xbaa37)
    #12 YBInitPostgresBackend <null> (postgres+0xc2b3ec)
    #13 InitPostgres <null> (postgres+0xc08f5c)
    #14 PostgresMain <null> (postgres+0xa499aa)
    #15 BackendRun <null> (postgres+0x9804b5)
    #16 BackendStartup <null> (postgres+0x97f8b6)
    #17 ServerLoop <null> (postgres+0x97e6b0)
    #18 PostmasterMain <null> (postgres+0x97b693)
    #19 PostgresServerProcessMain <null> (postgres+0x88b017)
    #20 main <null> (postgres+0x88b421)
@def- def- added kind/bug This issue is a bug area/ysql Yugabyte SQL (YSQL) labels Mar 9, 2022
@yugabyte-ci yugabyte-ci added the priority/medium Medium priority issue label Jun 8, 2022
@kripasreenivasan kripasreenivasan added the qa_automation Bugs identified via itest-system, LST, Stress automation or causing automation failures label Sep 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/ysql Yugabyte SQL (YSQL) kind/bug This issue is a bug priority/medium Medium priority issue qa_automation Bugs identified via itest-system, LST, Stress automation or causing automation failures
Projects
None yet
Development

No branches or pull requests

4 participants