Skip to content

Convert SWIMActor.sendPing to use Swift Concurrency (to avoid NIO shutdown errors) #1051

@ktoso

Description

@ktoso

It is a bit painful to prevent a periodically sending things actor mid-operation if a shutdown happens during it's execution.

Specifically code like this in SWIMActor:

    internal func sendPingRequests(_ directive: SWIMInstance.SendPingRequestDirective) async {
        // We are only interested in successful pings, as a single success tells us the node is
        // still alive. Therefore we propagate only the first success, but no failures.
        // The failure case is handled through the timeout of the whole operation.
        let eventLoop = self.actorSystem._eventLoopGroup.next()
        let firstSuccessful = eventLoop.makePromise(of: SWIM.PingResponse<SWIMActor, SWIMActor>.self)
        let pingTimeout = directive.timeout
        let peerToPing = directive.target

        let startedSendingPingRequestsSentAt: DispatchTime = .now()
        let pingRequestResponseTimeFirstTimer = self.swim.metrics.shell.pingRequestResponseTimeFirst
        firstSuccessful.futureResult.whenComplete { result in
            switch result {
            case .success: pingRequestResponseTimeFirstTimer.recordInterval(since: startedSendingPingRequestsSentAt)
            case .failure: ()
            }
        }

is prone to hitting Cannot schedule tasks on an EventLoop that has already shut down. from NIO when we hit shutdown() on the system -- which we do a lot in tests.

We should use Swift concurrency for this task and this way won't run into this issue; as we shut down, the tasks will be cancelled and the SWIM will stop its work anyway.

Metadata

Metadata

Assignees

Labels

1 - triagedTicket makes sense and is well defined, ready to be worked on2 - pick nextIf looking for work, pick this ticket :-)size/SSmall task. (A couple of hours of work.)

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions