Use external prices for conversion of fees in autopilot

[sunce86]

Description

Related to discussion in the issue: #2862 (comment)

Use external price vector to convert "fee in surplus token" into "fee in sell token" (instead of uniform clearing prices).

This is important so that we make sure that it's always true:

total_fee = executed_surplus_fee + protocol fees

If we use uniform clearing prices for executed_surplus_fee and external prices for protocol fees I think above equation won't stand.

How to test

Existing unit test

[sunce86]

Here we see that for this specific example, difference is ~2% if we use external prices vector for conversion. Seems acceptable.

[sunce86]

@fhenneke FYI this change affects:
settlement_observation::fee
order_execution::executed_surplus_fee

I don't expect this to have any significant effect on solver scripts.

[fleupold]

We already trust external price vectors for the surplus computation, so I think it make sense to use them for fee computation as well.

[squadgazzz]

LGTM

[fhenneke]

This will probably impact slippage and/or solver rewards.

If we want to reconstruct network fees (in sell token and in ETH) given protocol fees (in surplus token) and the total fee (in ETH), would we need external prices for all buy and sell tokens?

[sunce86]

If we want to reconstruct network fees (in sell token and in ETH) given protocol fees (in surplus token) and the total fee (in ETH), would we need external prices for all buy and sell tokens?

Not sure what's the context of the question but given the protocol fees in surplus token, you need at least external price of buy token if you want to convert protocol fee to ETH for sell orders.
And also, if you want to convert protocol fee to sell token for sell orders you also need external price for both sell and buy token.
So I think the answer is yes.

[fleupold]

But this is not an issue, is it? The autopilot filters out orders for which we cannot determine the price of the buy or sell token (which is required for evaluating surplus in ETH)

[sunce86]

I think we don't give strong guarantees for COW AMM orders:

1. We try to get all prices but if some are missing we are not filtering cow amm order owners for those that are missing (this is a bug maybe?).
2. Reference driver would filter out solutions that miss external price but external driver who knows
3. Autopilot does not check if all orders in the /solve response have prices (at the competition time).

This can be resolved with #2720

[fhenneke]

The slippage query will have to be adapted (it uses surplus_fee for exactly computing network fees) and we might need to upload additional data to dune. At the moment, external prices are uploaded for protocol fee tokens only. With this PR we need it for sell and buy tokens.

The midterm plan is to compute slippage off-chain and then it does not make as much of a difference.

[fleupold]

(this is a bug maybe?).

yes, if we cannot compute native prices the AMM should not tradable because we won't be able to compute surplus. That's a bug.