add asset names and amount/price calculations to additional #1351

oxarbitrage · 2018-09-30T14:22:13Z

Pull add some more data like asset_name into additional_data. Also calculate prices with precision.

All new data is added into new fields to keep current clients working.

In open-explorer.io this is done client side, problem is you have to make additional queries to the remote database to get precision, with kibana this is currently not possible, this should help with some visualizations.

more info: #1295 (comment)

pmconrad · 2018-09-30T15:58:28Z

libraries/plugins/elasticsearch/elasticsearch_plugin.cpp

   vs.fee_data.amount = o_v.fee_amount;
+   vs.fee_data.amount_calculated = (float)(o_v.fee_amount.value)/pow(10, o_v.fee_asset(db).precision);


precision is not greater than 12, can use a lookup table instead of expensive pow operation.
IMO amount_calculated is a bad name, perhaps better amount_units?
Using float is not nice but ok in this context I guess. Perhaps better use double?

(Same for the other _calculated fields of course.)

pmconrad · 2018-09-30T15:59:21Z

libraries/plugins/elasticsearch/elasticsearch_plugin.cpp

   vs.fill_data.receives_amount = o_v.fill_receives_amount;
+   vs.fill_data.receives_amount_calculated = (float)(o_v.fill_receives_amount.value)/pow(10, o_v.fill_receives_asset_id(db).precision);
+
+   float fill_price = (float)(o_v.fill_receives_amount.value/o_v.fill_receives_asset_id(db).precision) /


This is wrong, must use pow(10,precision) (or lookup table like above) instead of just precision.

pmconrad · 2018-10-02T12:45:09Z

libraries/plugins/elasticsearch/elasticsearch_plugin.cpp

@@ -72,6 +73,7 @@ class elasticsearch_plugin_impl
      std::string bulk_line;
      std::string index_name;
      bool is_sync = false;
+      map <uint8_t , uint64_t > lookup_pow_table;


Why not use a simple static const array with static initialization instead of a map? Would be much simpler IMO.
Also, use double values instead of uint64_t - they're always converted to double anyway, and it even saves you some casting when using the values.

thanks, i agree and changed. cant make it static as a member of elasticsearch_plugin_impl. maybe somewhere else, however not sure where.

pmconrad · 2018-10-02T17:01:36Z

libraries/plugins/elasticsearch/elasticsearch_plugin.cpp

+              1000000000,
+              10000000000,
+              100000000000,
+              1000000000000


Hm, not sure if the last three require LL modifier?

FYI there is already a lookup table exists in asset.hpp. No need to reinvent the wheel.

asset::scaled_precision(n) returns 10^n.

bitshares-core/libraries/chain/include/graphene/chain/protocol/asset.hpp

Lines 94 to 98 in fe8ca2a

static share_type scaled_precision( uint8_t precision )

{

FC_ASSERT( precision < 19 );

return scaled_precision_lut[ precision ];

}

nice one @abitmore , using that now :)

pmconrad · 2018-10-02T17:01:51Z

libraries/plugins/elasticsearch/elasticsearch_plugin.cpp

   vs.fee_data.amount = o_v.fee_amount;
+   vs.fee_data.amount_units = (double)(o_v.fee_amount.value)/lookup_pow_table[o_v.fee_asset(db).precision];


Can remove all (double) casts here

had to keep (double) cast in the fill_price_units when using the lookup table integrated in the codebase or the program will segfault with SIGFPE, Arithmetic exception.

pmconrad · 2018-10-03T06:38:19Z

Good idea to use asset::scaled_precision, but it returns a share_type. That means you must cast to double before dividing, or you lose the decimals.

pmconrad · 2018-10-03T15:47:16Z

libraries/plugins/elasticsearch/elasticsearch_plugin.cpp

@@ -235,12 +235,12 @@ void elasticsearch_plugin_impl::doVisitor(const optional <operation_history_obje
   vs.fee_data.asset = o_v.fee_asset;
   vs.fee_data.asset_name = o_v.fee_asset(db).symbol;


You're doing two lookups on fee_asset - better to look up only once.

pmconrad · 2018-10-03T15:47:35Z

libraries/plugins/elasticsearch/elasticsearch_plugin.cpp

@@ -235,12 +235,12 @@ void elasticsearch_plugin_impl::doVisitor(const optional <operation_history_obje
   vs.fee_data.asset = o_v.fee_asset;
   vs.fee_data.asset_name = o_v.fee_asset(db).symbol;
   vs.fee_data.amount = o_v.fee_amount;
-   vs.fee_data.amount_units = (o_v.fee_amount.value)/asset::scaled_precision(o_v.fee_asset(db).precision).value;
+   vs.fee_data.amount_units = (o_v.fee_amount.value)/(double)asset::scaled_precision(o_v.fee_asset(db).precision).value;

   vs.transfer_data.asset = o_v.transfer_asset_id;


Same for transfer_asset.

pmconrad · 2018-10-03T15:48:10Z

libraries/plugins/elasticsearch/elasticsearch_plugin.cpp

@@ -249,14 +249,14 @@ void elasticsearch_plugin_impl::doVisitor(const optional <operation_history_obje
   vs.fill_data.pays_asset_id = o_v.fill_pays_asset_id;
   vs.fill_data.pays_asset_name = o_v.fill_pays_asset_id(db).symbol;


Three lookups for fill_pays_asset and fill_receives_asset each.

add asset names and amount/price calculations to additional

c9db5dd

oxarbitrage added this to the 201810 - Feature Release milestone Sep 30, 2018

oxarbitrage added 6 UX Impact flag identifying the User Interface (UX) 6 Plugin Impact flag identifying at least one plugin labels Sep 30, 2018

pmconrad suggested changes Sep 30, 2018

View reviewed changes

oxarbitrage added 3 commits October 1, 2018 08:40

change var names calculated to units

ae09a86

change all to double and fix fill price formula

0df639e

implement pow lookup table to compute precision

1b3b297

pmconrad reviewed Oct 2, 2018

View reviewed changes

change lookup map to simple array

c2e17b3

pmconrad reviewed Oct 2, 2018

View reviewed changes

use lookup table already available in codebase

e686567

cast to double share_type before divide

57aee32

pmconrad reviewed Oct 3, 2018

View reviewed changes

reduce number of database lookups

0db7c14

pmconrad approved these changes Oct 3, 2018

View reviewed changes

oxarbitrage merged commit 3cec8b4 into bitshares:develop Oct 3, 2018

oxarbitrage mentioned this pull request Oct 3, 2018

[ES plugin] Wrong value of additional_data.fill_data.fill_order #1295

Closed

16 tasks

TheTaconator mentioned this pull request Oct 15, 2018

Release Notes for BitShares Core 201810 #1381

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add asset names and amount/price calculations to additional #1351

add asset names and amount/price calculations to additional #1351

oxarbitrage commented Sep 30, 2018 •

edited

Loading

pmconrad Sep 30, 2018

pmconrad Sep 30, 2018

pmconrad Oct 2, 2018

oxarbitrage Oct 2, 2018

pmconrad Oct 2, 2018

abitmore Oct 2, 2018 •

edited

Loading

oxarbitrage Oct 2, 2018

pmconrad Oct 2, 2018

oxarbitrage Oct 2, 2018

pmconrad commented Oct 3, 2018

pmconrad Oct 3, 2018

pmconrad Oct 3, 2018

pmconrad Oct 3, 2018

		vs.fee_data.amount = o_v.fee_amount;
		vs.fee_data.amount_calculated = (float)(o_v.fee_amount.value)/pow(10, o_v.fee_asset(db).precision);

	static share_type scaled_precision( uint8_t precision )
	{
	FC_ASSERT( precision < 19 );
	return scaled_precision_lut[ precision ];
	}

		vs.fee_data.amount = o_v.fee_amount;
		vs.fee_data.amount_units = (double)(o_v.fee_amount.value)/lookup_pow_table[o_v.fee_asset(db).precision];

		@@ -235,12 +235,12 @@ void elasticsearch_plugin_impl::doVisitor(const optional <operation_history_obje
		vs.fee_data.asset = o_v.fee_asset;
		vs.fee_data.asset_name = o_v.fee_asset(db).symbol;

		@@ -249,14 +249,14 @@ void elasticsearch_plugin_impl::doVisitor(const optional <operation_history_obje
		vs.fill_data.pays_asset_id = o_v.fill_pays_asset_id;
		vs.fill_data.pays_asset_name = o_v.fill_pays_asset_id(db).symbol;

add asset names and amount/price calculations to additional #1351

add asset names and amount/price calculations to additional #1351

Conversation

oxarbitrage commented Sep 30, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abitmore Oct 2, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pmconrad commented Oct 3, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oxarbitrage commented Sep 30, 2018 •

edited

Loading

abitmore Oct 2, 2018 •

edited

Loading