Skip to content

Commit

Permalink
fix(go-grpc-server): always close resultChan
Browse files Browse the repository at this point in the history
By not closing the channel, if a server not implementing PredictStream
receives a client call would hang indefinetly as would wait for
resultChan to be consumed.

If the prediction stream returns we close the channel now and we wait
for the goroutine to finish.
  • Loading branch information
mudler committed Oct 4, 2024
1 parent 2553de0 commit fce2031
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions pkg/grpc/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,8 @@ func (s *server) PredictStream(in *pb.PredictOptions, stream pb.Backend_PredictS
}()

err := s.llm.PredictStream(in, resultChan)
// close the channel, so if resultChan is not closed by the LLM (maybe because does not implement PredictStream), the client will not hang
close(resultChan)
<-done

return err
Expand Down

0 comments on commit fce2031

Please sign in to comment.