handle changed error responses from crates.io API

[syphar]

Since Dec 18 we were seeing errors when parsing search responses from crates.io, deployed on the evening of Dec 16th:

error decoding response body: missing field crates at line 1 column 47

Some digging lead me to this crates.io change which changed the status code for errors, which lead to error_for_status not doing the early return.

[syphar]

fyi @Turbo87

[GuillaumeGomez]

Some tests broke but other than that, looks good to me.

[Turbo87]

sorry about that. it was ultimately a bug in crates.io that it did not comply with the interface description that cargo expects. it should hopefully be more consistent now, and once we switch the feature flag off there will be an official announcement blog post before we do so.

[syphar]

sorry about that. it was ultimately a bug in crates.io that it did not comply with the interface description that cargo expects. it should hopefully be more consistent now, and once we switch the feature flag off there will be an official announcement blog post before we do so.

no worries, it's also on me that I didn't check how cargo itself handles the responses & errors :)

[syphar]

Some tests broke but other than that, looks good to me.

@GuillaumeGomez
the test failure is unrelated, re-running fixed it (I'll dig into this separately, looks like a race condition between the tests when run in parallel).