Skip to content

Deadlock in Subscriber impl of Drop #1515

@ardouglas

Description

@ardouglas

Hello, I encountered a deadlock while toying around with the Subscriber API. The scenario is as follows:

One thread performing inserts on a Tree.

One or more threads subscribed to updates on the Tree and forwarding those events to another thread over a channel. If the subscriber's end of the channel is closed, I break out of the subscriber loop and fall through to the end of the function (or so I thought).

What I'm actually seeing is that the impl of Drop gets deadlocked on the call to self.home.write(). To test this I added some prints to the drop method like:

impl Drop for Subscriber {
    fn drop(&mut self) {
        println!("entering drop for subscriber");
        let mut w_senders = self.home.write();
        w_senders.remove(&self.id);
        println!("exiting drop for subscriber");
    }
}

When I have two subscribers I will often see one of them make it through both printlns, but the other will not. Furthermore, the thread performing the inserts is no longer able to make forward progress in this scenario.

I found two ways around this:

  1. force the subscribers to continue reading until the inserts are complete
  2. edit the impl of Drop and drop the rx field of Subscriber prior to calling self.home.write() like:
impl Drop for Subscriber {
    fn drop(&mut self) {
        let (_tx, mut rx) = sync_channel(1);
        std::mem::swap(&mut rx, &mut self.rx);
        drop(rx);
        let mut w_senders = self.home.write();
        w_senders.remove(&self.id);
    }
}

The above code does seem to alleviate the deadlock though I'm not sure if there might be a better way to remove the cycle between the two fields.

Expected Result

Dropping a Subscriber does not impact writing threads

Actual Result

Dropping a Subscriber deadlocks the subscriber and the writer threads

Sled Version

0.34.7

Rustc Version

1.83.0 (90b35a623 2024-11-26)

Operating System

Rocky Linux 8.5

Minimal Code Sample

I don't immediately have one I can submit but if you're willing to accept a PR I will try to make a minimal test case.

logs, panic messages, stack traces

There are none

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions