Heuristically filter orders for batch cutting

[sistemd]

During batch cutting, only include orders which are a certain % away (configurable) from the estimated native price.

[MartinquaXD]

During batch cutting, only include orders which are a certain % away (configurable) from the estimated native price.

Eventually we also have to think about prioritizing for which limit order we want to update the surplus_fee in case we run into rate limits of the quoting mechanism.
I think the same heuristic can be applied there which would make filtering orders at the time of auction cutting redundant.

[nlordell]

Just to clarify, is this issue about solving (i.e. ordering of orders we chose to look at in our SingleOrderSolver implementation in the solver) or batch cutting (i.e. ordering of orders to look at when preparing the auction in the autopilot)?

[sistemd]

Just to clarify, is this issue about solving (i.e. ordering of orders we chose to look at in our SingleOrderSolver implementation in the solver) or batch cutting (i.e. ordering of orders to look at when preparing the auction in the autopilot)?

It's about batch cutting.

[harisang]

I am not sure I agree with this, although I definitely see where it's coming from.

To me, this will look like order censorship; having a valid order in the orderbook that no solver potentially ever sees.

[sistemd]

I am not sure I agree with this, although I definitely see where it's coming from.

To me, this will look like order censorship; having a valid order in the orderbook that no solver potentially ever sees.

Solvers won't see orders only if their prices are far outside the market range. We can't spam solvers with all limit orders, that's not going to work. But it's definitely not censorship.

Censorship would be if we omit limit orders even if they are within the market price, which we don't do.

[nlordell]

I am not sure I agree with this

While, I must admit I'm not super happy with filtering out limit orders, one of the major risks from this feature is the additional load on the services and not being able to keep up with auction creation. This changes is more about providing us with safety handles and to reduce that risk.

Another proposal by @MartinquaXD was to take N most favourably priced orders (#765). Maybe we can iterate on how we do this filtering and improve our algorithm and heuristics over time, but having an "easy way out" of having an orderbook that is just too large is an important backup IMO.

having a valid order in the orderbook that no solver potentially ever sees

I agree, but I think the alternative is worse (in having an order that a solver doesn't see because we can't compute fees fast enough).