Less error debugs for 0x api call failure

[josojo]

During oncall, it occured to me that the 0x liquidity fetching was generating noisy error messages like this:
https://cowservices.slack.com/archives/C0371SB243E/p1651405247972289

Tried to fix it with a different error handling.

Test Plan

[codecov-commenter]

Codecov Report

Merging #184 (c18f3e4) into main (3600251) will decrease coverage by 0.01%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main     #184      +/-   ##
==========================================
- Coverage   64.59%   64.57%   -0.02%     
==========================================
  Files         185      185              
  Lines       38596    38602       +6     
==========================================
- Hits        24931    24928       -3     
- Misses      13665    13674       +9     

[nlordell]

In general, its good to have metrics for measuring errors that we don't alert on.

That being said, I think we already count the number of 0x liquidity orders with another metric, so we can use that to see if we aren't getting any 0x liquidity through.

Also, are these errors intermittent? Is there some underlying issue that is causing them to happen (such as being rate limited? - if so maybe we can consider using the new RateLimiter that @MartinquaXD introduced?).

[MartinquaXD]

That being said, I think we already count the number of 0x liquidity orders with another metric, so we can use that to see if we aren't getting any 0x liquidity through.

There is currently only a metric for the number of LimitOrder liquidity items. At the moment we only get LimitOrder liquidity from 0x but this might change in the future. This metric also only shows the number of usable 0x orders we got. I believe we could theoretically get 0 usable 0x orders without errors while collecting liquidity.

if so maybe we can consider using the new RateLimiter

The underlying error is some generic reqwest::Error which happened during RequestBuilder::send(). The docs say the following about that:

    /// # Errors
    ///
    /// This method fails if there was an error while sending request,
    /// redirect loop was detected or redirect limit was exhausted.
    pub fn execute(
        &self,
        request: Request,
    ) -> impl Future<Output = Result<Response, crate::Error>> {

So I don't think there is anything we can do about that. To make use of the RateLimiter we need to actually get a response with status code 429.

[josojo]

Thanks, Martin, for the investigation!

[MartinquaXD]

I think it generally makes sense to overhaul the way we fetch liquidity.
At the moment an error while fetching uniswap-like, balancer or 0x liquidity will cause all the successfully collected liquidity to be discarded or not fetched in the first place.
I believe the PR offers a good enough temporary work around such that 0x errors no longer affect all the other liquidity but we should keep this issue in mind to have a proper solution for all liquidity types moving forward.