[20][ety001] Order book: stack orders those prices are too close

[abitmore]

Check https://wallet.bitshares.org/#/market/BTS_CNY for example, unable to see other orders but only these:

[ahdigital]

Can you confirm what you mean by prices are too close?

[HarukaMa]

Could be difficult to determine which orders should be consolidated together as it could possibly affect some normal orders in market... But this bot is indeed annoying and was making order depth chart quite useless.

[abitmore]

On some exchanges, for example https://www.binance.com/trade.html?symbol=BTS_BTC , user can group orders, so the order book is clearer:

[wmbutler]

I'm not clear on how you'd group orders when the prices are all different.

[abitmore]

Group by decimals. Click the link you'll see.

[abitmore]

How about a new API that returns an order book that orders are grouped already? I think we can group the orders by percentage increase on price, we can say that a group of orders is a bucket, say 0.01% buckets, 0.1% buckets, 1% buckets and etc.

An order group (e.g. an ask order group) would have there attributes :

• max_price: the greatest asking price in the group
• min_price: the least asking price in the group
• total_for_sale: the total amount of asset that is being sold

And perhaps one or more of these attributes if you think are useful (to determine when to execute a new query, to subscribe to market order objects, to update user balance info and history, etc, although personally I think it's inefficient for UI to subscribe to order objects in busy markets in comparison to polling):

• min_expiration: the earliest time that an order in the group may expire
• id_array: a list of order object ID's
• seller_array: a list of sellers, maybe sorted by whatever attribute and/or have duplicates removed

In order to implement this, the simplest way for the backend would be calculating on the fly which will use no extra RAM but every query uses CPU, the way with highest query performance is an plugin that has data pre-calculated with pre-defines buckets. Personally I prefer the latter, in this case we should define a default bucket list for both the UI and the backend.

@wmbutler @svk31 @HarukaMa please advise.

//Update: we can add similar API's for force settlements and call orders later.

[tbone-bts]

Grouped orders is a nice feature. Like Binance, Poloniex also groups by decimal places. I think @abitmore's idea of grouping by buckets based on price % is very interesting, although it's uncertain how useful a limited # of pre-selected buckets (i.e. .01%, .1%, 1%) will be. Perhaps consider a spinbox that enables users to easily adjust the % grouping up/down more incrementally.

If this feature is added, Bitshares DEX and Poloniex will be the only 2 exchanges I can think of whose order books have both this grouping feature AND cumulative total of orders. This a very powerful combination of order book features that make placement of orders easier and much more convenient. We should not underestimate how much easier it is to place orders (esp. for noobs) when you can readily see things like a) how many XYZ coins can you get in exchange for the amount you wish to spend or b) how much it will cost you to get the number of XYZ coins you wish to buy. Grouped orders and especially cumulative order totals are incredibly helpful in this regard.

re: the plugin idea, it seems that it would make sense to include cumulative total of orders in such an effort. I'll add that to bitshares/bitshares-core#639.

[abitmore]

To whom it may concern, I've finished the plugin and API: bitshares/bitshares-core#662.

No need to replay after applied the patch, nor changed configured tracked-groups parameter, but do need to restart, the data will be reloaded at every restart. A side effect is need longer at startup. With default config (0.1% and 1% groups), it does need more RAM, but not much (I think it's around 1G 20M).

[wmbutler]

My gut says that the Foundation servers should implement this so we have some known nodes using it and others can follow at their leisure.

[abitmore]

Sample data with 0.1% groups pasted below, the first 10 groups contain 97 orders for BTS:CNY market:

$ curl -d '{"id":15,"method":"call","params":["orders","get_grouped_limit_orders",["1.3.113","1.3.0",10,null,10]]}' http://127.0.0.1:38690/rpc 2>/dev/null ;echo

{"id":15,"jsonrpc":"2.0","result":[{"group":10,"min_price":{"base":{"amount":44911,"asset_id":"1.3.113"},"quote":{"amount":300000,"asset_id":"1.3.0"}},"max_price":{"base":{"amount":755251,"asset_id":"1.3.113"},"quote":{"amount":5040051,"asset_id":"1.3.0"}},"total_for_sale":2965272},{"group":10,"min_price":{"base":{"amount":753991,"asset_id":"1.3.113"},"quote":{"amount":5040051,"asset_id":"1.3.0"}},"max_price":{"base":{"amount":59880000,"asset_id":"1.3.113"},"quote":{"amount":400000000,"asset_id":"1.3.0"}},"total_for_sale":68176675},{"group":10,"min_price":{"base":{"amount":753235,"asset_id":"1.3.113"},"quote":{"amount":5040051,"asset_id":"1.3.0"}},"max_price":{"base":{"amount":753941,"asset_id":"1.3.113"},"quote":{"amount":5040051,"asset_id":"1.3.0"}},"total_for_sale":333753347},{"group":10,"min_price":{"base":{"amount":752479,"asset_id":"1.3.113"},"quote":{"amount":5040051,"asset_id":"1.3.0"}},"max_price":{"base":{"amount":753185,"asset_id":"1.3.113"},"quote":{"amount":5040051,"asset_id":"1.3.0"}},"total_for_sale":12278520},{"group":10,"min_price":{"base":{"amount":751723,"asset_id":"1.3.113"},"quote":{"amount":5040051,"asset_id":"1.3.0"}},"max_price":{"base":{"amount":752429,"asset_id":"1.3.113"},"quote":{"amount":5040051,"asset_id":"1.3.0"}},"total_for_sale":20504114},{"group":10,"min_price":{"base":{"amount":100000000,"asset_id":"1.3.113"},"quote":{"amount":671140940,"asset_id":"1.3.0"}},"max_price":{"base":{"amount":751673,"asset_id":"1.3.113"},"quote":{"amount":5040051,"asset_id":"1.3.0"}},"total_for_sale":140476523},{"group":10,"min_price":{"base":{"amount":369988,"asset_id":"1.3.113"},"quote":{"amount":2484808,"asset_id":"1.3.0"}},"max_price":{"base":{"amount":369988,"asset_id":"1.3.113"},"quote":{"amount":2484808,"asset_id":"1.3.0"}},"total_for_sale":369988},{"group":10,"min_price":{"base":{"amount":1486000,"asset_id":"1.3.113"},"quote":{"amount":10000000,"asset_id":"1.3.0"}},"max_price":{"base":{"amount":882730,"asset_id":"1.3.113"},"quote":{"amount":5935206,"asset_id":"1.3.0"}},"total_for_sale":981420},{"group":10,"min_price":{"base":{"amount":487143,"asset_id":"1.3.113"},"quote":{"amount":3283379,"asset_id":"1.3.0"}},"max_price":{"base":{"amount":1306888,"asset_id":"1.3.113"},"quote":{"amount":8800000,"asset_id":"1.3.0"}},"total_for_sale":35545525},{"group":10,"min_price":{"base":{"amount":746683,"asset_id":"1.3.113"},"quote":{"amount":5040051,"asset_id":"1.3.0"}},"max_price":{"base":{"amount":747237,"asset_id":"1.3.113"},"quote":{"amount":5040051,"asset_id":"1.3.0"}},"total_for_sale":9941642}]}

[abitmore]

Seems nobody is working on this? Perhaps 2 hours is too few. Maybe 10 hours would be OK.

[wmbutler]

Moved this to April and increased hours to 5

[abitmore]

Update: I've got a guy to work on this issue. Will provide more info later.

[abitmore]

Update: sample data at the bottom of the page: http://open-explorer.io/#/markets/BTS/USD (although data on the page seems wrong).

[abitmore]

Update: we've made good progress, will submit PR asap.

@wmbutler according to the work we've done and foreseen, I think it's OK to update hours of this issue to 20.

[tbone-bts]

I really like this idea. I would just point out that it will only be partially helpful to display the total in terms of the base asset only. In other words, it should also show the total in terms of the quoted asset.

Also, it is not helpful to show the total of each individual group. Instead, each row should show the cumulative sum total of all rows/groups up to (and including) that one.

[abitmore]

@tbone-bts basic data is a thing, UX is another. What you're asking is more data showing on the page, which can be derived from the basic data.

By the way, since we don't have "buy limit order" feature, at this moment it's hard to place accurate bid orders when the market is changing frequently.

[svk31]

The GUI already does this when parsing the orderbook, I realize you already implemented this in the backend now but tbh I don't see why that's even needed. It would be easy to extend the current grouping in the GUI to do what you want.

[abitmore]

@svk31

1. to fetch less data from backend but get more useful info;
2. to show deeper market depth.

For example, https://wallet.bitshares.org/#/market/BTS_CNY, even clicked "show all bids", the lowest bid is at 1.61 CNY/BTS while the highest is at 1.69, showing in total 600K CNY on the book but there are much more orders:

In addition, to get this data with current code, need to fetch 300 orders from API node.

The depth chart is pretty ugly.

[abitmore]

See PR #1467.

[wmbutler]

@abitmore can you post the BTS address where you expect to receive payment?

[svk31]

It's here: #1467 (comment)

ety001.worker