You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
BSP agent crashes (after some retries) when there is any issue in producing BS proofs
From the last conversation a while back, we didn't have any monitoring, hence crashing the bsp-agent helped devs to be aware of some issue and help them debugging
But now that we have the metrics for successful block specimens, it would be nice to properly catch the errors, and avoid crashing the application
Specially recently we have been facing alot of crashes due to chain congestion
Some error logs recently:
bsp-agent-chorusone | 2024-04-14T12:47:17.858870813Z time="2024-04-14T12:47:17Z" level=info msg="Process pending streams at: 2024-04-14T12:47:17Z" function=consumePendingEvents line=120
bsp-agent-chorusone | 2024-04-14T12:47:19.621061918Z time="2024-04-14T12:47:19Z" level=error msg="error sending tx to deployed contract: submit transaction to pool failed: Ok(ImmediatelyDropped)" function=executeWithRetry line=153
bsp-agent-chorusone | 2024-04-14T12:47:19.621102596Z time="2024-04-14T12:47:19Z" level=error msg="failed to avro encode, prove and upload block-result segment with err: failed to prove & upload block-replica segment event: 1-19653865-replica" function=processStream line=155
bsp-agent-chorusone | 2024-04-14T12:47:19.623321289Z panic: failed to prove & upload block-replica segment event: 1-19653865-replica
bsp-agent-chorusone | 2024-04-14T12:47:19.623383788Z
bsp-agent-chorusone | 2024-04-14T12:47:19.623388126Z goroutine 7320 [running]:
bsp-agent-chorusone | 2024-04-14T12:47:19.623391242Z github.com/covalenthq/bsp-agent/internal/node.(*ethAgentNode).processStream(0xc00044b520, {{0xc000028510?, 0xc0003c0fd0?}, 0xc00012e090?}, 0xc00049b700)
bsp-agent-chorusone | 2024-04-14T12:47:19.623394519Z /bsp-agent/internal/node/eth_node.go:156 +0x95a
bsp-agent-chorusone | 2024-04-14T12:47:19.623397534Z created by github.com/covalenthq/bsp-agent/internal/node.(*ethAgentNode).consumeEvents
bsp-agent-chorusone | 2024-04-14T12:47:19.623404828Z /bsp-agent/internal/node/eth_node.go:69 +0x38a
bsp-agent-chorusone exited with code 0
The text was updated successfully, but these errors were encountered:
hritique
changed the title
Prevent bsp-agent to crashloop in case of transaction errors
Prevent bsp-agent to crash in case of transaction errors
Apr 18, 2024
BSP agent crashes (after some retries) when there is any issue in producing BS proofs
From the last conversation a while back, we didn't have any monitoring, hence crashing the bsp-agent helped devs to be aware of some issue and help them debugging
But now that we have the metrics for successful block specimens, it would be nice to properly catch the errors, and avoid crashing the application
Specially recently we have been facing alot of crashes due to chain congestion
Some error logs recently:
The text was updated successfully, but these errors were encountered: