Skip to content

Commit

Permalink
Fix: notification-thread: handling event from a removed tracer event src
Browse files Browse the repository at this point in the history
Issue
=====
The issue is caused by a race condition where the `lttng_poll_wait()`
returns a _REMOVE_TRACER_EVENT_SOURCE event followed by an actual
notification event on the removed event source fd.

This causes the notification thread to remove the fd from the potential
notification sources list and later fail to find that same fd in the
next iteration.

This race condition can lead to the notification thread to hang
indefinitely or to failed assertions within the `fini_thread_state()`
function.

Fix
===
When removing an tracer event source, force the notification thread
`lttng_poll_wait()` loop to restart to ignore events from the removed
fd.

Use the `restart_poll` for that purpose (see note below).

Reproducer
==========
It's easy to reproduce this issue by adding a `usleep(5000)` just before
the `lttng_poll_wait()` call in the notification thread.

Note
====
It's the second time that I fix this issue.

It was first fixed by this commit by adding the `restart_poll` flag:
  commit 8b52406
  Author: Francis Deslauriers <[email protected]>
  Date:   Thu Dec 10 15:41:29 2020 -0500

    notification-thread: drain all tracer notification on removal

and later, that other commit refactored that code but accidently removed
the use of the `restart_poll`:
  commit 34bf4f6
  Author: Francis Deslauriers <[email protected]>
  Date:   Wed May 26 16:05:16 2021 -0400

    notification-thread: remove fd from pollset on LPOLLHUP and friends

Signed-off-by: Francis Deslauriers <[email protected]>
Signed-off-by: Jérémie Galarneau <[email protected]>
Change-Id: I6da0ed4374b612934adc72fb88d5c142505c5d53
  • Loading branch information
frdeso authored and jgalar committed Oct 18, 2021
1 parent 0c0793a commit 10c54de
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions src/bin/lttng-sessiond/notification-thread-events.c
Original file line number Diff line number Diff line change
Expand Up @@ -2153,6 +2153,12 @@ int remove_tracer_event_source_from_pollset(

source_element->is_fd_in_poll_set = false;

/*
* Force the notification thread to restart the poll() loop to ensure
* that any events from the removed fd are removed.
*/
state->restart_poll = true;

ret = drain_event_notifier_notification_pipe(state, source_element->fd,
source_element->domain);
if (ret) {
Expand Down

0 comments on commit 10c54de

Please sign in to comment.