Prevent bsp-agent to crash in case of transaction errors

[hritique]

BSP agent crashes (after some retries) when there is any issue in producing BS proofs
From the last conversation a while back, we didn't have any monitoring, hence crashing the bsp-agent helped devs to be aware of some issue and help them debugging

But now that we have the metrics for successful block specimens, it would be nice to properly catch the errors, and avoid crashing the application

Specially recently we have been facing alot of crashes due to chain congestion
Some error logs recently:

bsp-agent-chorusone  | 2024-04-14T12:47:17.858870813Z time="2024-04-14T12:47:17Z" level=info msg="Process pending streams at: 2024-04-14T12:47:17Z" function=consumePendingEvents line=120
bsp-agent-chorusone  | 2024-04-14T12:47:19.621061918Z time="2024-04-14T12:47:19Z" level=error msg="error sending tx to deployed contract: submit transaction to pool failed: Ok(ImmediatelyDropped)" function=executeWithRetry line=153
bsp-agent-chorusone  | 2024-04-14T12:47:19.621102596Z time="2024-04-14T12:47:19Z" level=error msg="failed to avro encode, prove and upload block-result segment with err: failed to prove & upload block-replica segment event: 1-19653865-replica" function=processStream line=155
bsp-agent-chorusone  | 2024-04-14T12:47:19.623321289Z panic: failed to prove & upload block-replica segment event: 1-19653865-replica
bsp-agent-chorusone  | 2024-04-14T12:47:19.623383788Z 
bsp-agent-chorusone  | 2024-04-14T12:47:19.623388126Z goroutine 7320 [running]:
bsp-agent-chorusone  | 2024-04-14T12:47:19.623391242Z github.com/covalenthq/bsp-agent/internal/node.(*ethAgentNode).processStream(0xc00044b520, {{0xc000028510?, 0xc0003c0fd0?}, 0xc00012e090?}, 0xc00049b700)
bsp-agent-chorusone  | 2024-04-14T12:47:19.623394519Z   /bsp-agent/internal/node/eth_node.go:156 +0x95a
bsp-agent-chorusone  | 2024-04-14T12:47:19.623397534Z created by github.com/covalenthq/bsp-agent/internal/node.(*ethAgentNode).consumeEvents
bsp-agent-chorusone  | 2024-04-14T12:47:19.623404828Z   /bsp-agent/internal/node/eth_node.go:69 +0x38a
bsp-agent-chorusone exited with code 0

bsp-agent-chorusone  | 2024-04-14T14:15:26.879654466Z time="2024-04-14T14:15:26Z" level=info msg="tip cap: 2000000000 fee cap: 252000000000" function=setTransactionFeeParams line=104
bsp-agent-chorusone  | 2024-04-14T14:15:37.065873908Z time="2024-04-14T14:15:37Z" level=error msg="error sending tx to deployed contract: replacement transaction underpriced" function=executeWithRetry line=153
bsp-agent-chorusone  | 2024-04-14T14:15:37.065928362Z time="2024-04-14T14:15:37Z" level=error msg="failed to avro encode, prove and upload block-result segment with err: failed to prove & upload block-replica segment event: 1-19654040-replica" function=processStream line=155
bsp-agent-chorusone  | 2024-04-14T14:15:37.068130283Z panic: failed to prove & upload block-replica segment event: 1-19654040-replica
bsp-agent-chorusone  | 2024-04-14T14:15:37.068172384Z 
bsp-agent-chorusone  | 2024-04-14T14:15:37.068179037Z goroutine 2775 [running]:
bsp-agent-chorusone  | 2024-04-14T14:15:37.068184176Z github.com/covalenthq/bsp-agent/internal/node.(*ethAgentNode).processStream(0xc00049e2e0, {{0xc0033f57a0?, 0x2e72fd?}, 0xc000c102a0?}, 0xc000127460)
bsp-agent-chorusone  | 2024-04-14T14:15:37.068190168Z   /bsp-agent/internal/node/eth_node.go:156 +0x95a
bsp-agent-chorusone  | 2024-04-14T14:15:37.068195037Z created by github.com/covalenthq/bsp-agent/internal/node.(*ethAgentNode).consumePendingEvents
bsp-agent-chorusone  | 2024-04-14T14:15:37.068199977Z   /bsp-agent/internal/node/eth_node.go:116 +0x625
bsp-agent-chorusone exited with code 0