WS connections refused with 1006, even though token is correct

[Twiggeh]

• I have seen https://docs.wire.com/ and https://github.com/wireapp/wire-server-deploy - the documentation there does not answer my question.

My question:
After running the production installation of wire server for ~ 20+ days, the ws connections are failing the upgrade with error code 1006, I have tried restarting cannon and nginz (because it was giving http 420, which I assume is a rate limit), but nothing helped.

Apart from re-deploying the entire backend I have not found a way to fix this behaviour, any Ideas on what I should check / do.

Currently I have failed over the misbehaving cluster to a "fresh one".

The normal requests that also require authorization are resolved with no issue with the same token (e.g. /notifications /list-connections)

[akshaymankar]

I haven't seen this error yet. I think the client logs might help with understanding the error. Do you have any logs?

[Twiggeh]

Hi @akshaymankar !

I do have logs, but I don't know from which pod you would like to have the logs from, e.g. cannon returns an empty file
kubectl logs $(kubectl get pods | grep cannon | awk '{print $1}') -c cannon

Here the console output for the ws upgrade

isTrusted: true
bubbles: false
cancelBubble: false
cancelable: false
code: 1006
composed: false
currentTarget: 
WebSocket {url: 'wss://nginz-ssl.<nda>/await?access_token=d5…a64.c=5720203841942841103&client=4e29e84c76b93db0', readyState: 3, bufferedAmount: 0, onopen: ƒ, onerror: ƒ, …}
defaultPrevented: false
eventPhase: 0
reason: ""
returnValue: true
srcElement: WebSocket {url: 'wss://nginz-ssl.<nda>/await?access_token=d5…a64.c=5720203841942841103&client=4e29e84c76b93db0', readyState: 3, bufferedAmount: 0, onopen: ƒ, onerror: ƒ, …}
target: WebSocket {url: 'wss://nginz-ssl.<nda>/await?access_token=d5…a64.c=5720203841942841103&client=4e29e84c76b93db0', readyState: 3, bufferedAmount: 0, onopen: ƒ, onerror: ƒ, …}
timeStamp: 72322.09999999963
type: "close"
wasClean: false
[[Prototype]]: CloseEvent

Hallo

Hallo mehr :D