Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-629: Batch request for redeeming multiple payables + the payable and pending_payable machinary reworked to request the DB do multiple operations in a single transactions #190

Merged
merged 88 commits into from
May 5, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
88 commits
Select commit Hold shift + click to select a range
e052252
GH-629: basic structure exists
Aug 29, 2022
18083da
GH-629: interim commit
Sep 23, 2022
14a2f8c
GH-629: test for the transfer of tokens via batch transport is roughl…
Sep 27, 2022
fa50905
GH-629: main contoures made and will think how to finish it to the wo…
Sep 28, 2022
b9b16ba
GH-629: in the middle of big rearrangement of connections between typ…
Sep 29, 2022
311eed3
GH-629: finaly compiling; after a lot of changes in data structures; …
Oct 3, 2022
8fb6d78
GH-629: problem with signed_transaction; now it seems like I cannot g…
Oct 12, 2022
3d46e18
GH-629: the base test is complete and passing... still a lot of work …
Oct 13, 2022
4ebad36
GH-629: another big test starts passing
Oct 15, 2022
bbd6125
GH-629: interim commit
Oct 17, 2022
943ef44
GH-629: improving blockchain bridge and testing and testing, and...
Oct 18, 2022
c6eb6d1
GH-629: fixed a couple of things in the joint with blockcahin bridge;…
Oct 19, 2022
5ff085b
GH-629: pretty printed error messages; very simple tests fixes, chang…
Oct 24, 2022
3d9f0e2
GH-629: advancing it repairing tests; but also continuous refactoring…
Oct 25, 2022
80431e4
GH-629: in the middle of the transition from one fingerprint processe…
Oct 31, 2022
54cc7f4
GH-629: continuation of implementing batch like processing for payabl…
Nov 1, 2022
5a73287
GH-629: mark_pending_payable() is fully implemented, I'm gonna switch…
Nov 2, 2022
5f6c70c
GH-629: fixing more paths in handle_sent_payable(); also finished del…
Nov 2, 2022
39f0b87
GH-629: in the middle of reconstructure of pending payable (receipts,…
Nov 3, 2022
fe98ae5
GH-629: confirming txs, deleting and updating of fingerprints reimple…
Nov 5, 2022
276c931
GH-629: all old tests fixed and some portion above; logging better se…
Nov 7, 2022
cefdaf0
GH-629: removed optionality at fields
Nov 7, 2022
9eb87f4
GH-629: last TODOs knocked off
Nov 7, 2022
71b0889
GH-629: little master merge and code hygien fixes + Clippy
Nov 8, 2022
c892f37
GH-629: first repair after my auto review
Nov 9, 2022
f885fbc
GH-629: a few more corrections from an auto review
Nov 10, 2022
af3ee56
GH-629: later fixes after diving more into the code when GH didn't le…
Nov 12, 2022
5f27695
GH-629: trying to find a way how to get the proper row changes count …
Nov 14, 2022
2c116b3
GH-629: fix for multi-record update transaction; and streightening th…
Nov 15, 2022
cdd29ce
GH-629: added a performance test for multi-record update sql with a c…
Nov 15, 2022
422f7a3
GH-629: extra helper for detecting panics connected to repeated payam…
Dec 5, 2022
33757bb
GH-629: more accure testing of performance of multirow update sql
Dec 6, 2022
1f2ba57
GH-629: my own remarks turned into modyfications as well as things fr…
Dec 6, 2022
e07b0e9
GH-629: modifying test slightly to pass in Actions
Dec 7, 2022
529126f
GH-629: tuning performance tests in payable dao
Dec 14, 2022
c3bb9de
GH-629: interim commit - a performance wise integration test half don…
Dec 18, 2022
30141e2
GH-629: trying performance integration test in Actions
Dec 20, 2022
ae9aaf8
GH-629: another adjustment in the effort for making the integ test re…
Dec 20, 2022
45c03a7
GH-629: formatting
Dec 20, 2022
8a11c03
GH-629: another adjustmen; also loosser requirements within my coefic…
Dec 21, 2022
8d55c1f
GH-629: another aproachment to the optimum; adding the fifth attempt …
Dec 21, 2022
67d6992
GH-629: merged master in (GH-497) and it's compiling but there are so…
Dec 22, 2022
4c81761
GH-629: cleaning up aftereffects of the master merge; not finished. T…
Dec 27, 2022
c989028
GH-629: finished merge of v0.7.0 and had to modify a couple of things…
Jan 3, 2023
2a99975
Merge branch 'master' into GH-629
Jan 3, 2023
f1cc277
GH-629: fixing conficts in multinode tests and formatting
Jan 3, 2023
bfd8660
GH-629: fixed platform specifically behaving test
Jan 3, 2023
850781f
GH-629: test fix and adjustment of another one
Jan 4, 2023
2504fb5
GH-629: I revamped the text explaining ArbitraryIdStamp and made the …
Jan 4, 2023
e20dd15
GH-629: mad merge; just a sketch of the future layout; will need much…
Jan 30, 2023
40be36e
GH-629: reduced some of the compilation errors considerably; nearing …
Feb 1, 2023
a1b507f
GH-629: another portion of restored tests and some production code sh…
Feb 6, 2023
d05d359
GH-629: repaired scanners to the probable compilation state; progress…
Feb 7, 2023
6adf529
GH-629: failing tests almost resolved; a few last to go
Feb 8, 2023
7233492
GH-629: finally all test passing; but will have to go through todos I…
Feb 9, 2023
c4f4319
GH-629: fixed; now knocking off todos; before trying PendingPayableId…
Feb 9, 2023
6da477f
GH-629: PendingPayableId constructor added
Feb 9, 2023
3b52cae
GH-629: new portion of killed todos and some refactoring...; going on…
Feb 10, 2023
8276cc4
GH-629: merging beleived done; but one integration test is failing
Feb 10, 2023
9f77827
GH-629: one older integration test repaired
Feb 13, 2023
a86a46c
GH-629: removed tests to be worked out in a diff card; plus some cosm…
Feb 15, 2023
f623a40
GH-629: the first few things fixed
Mar 6, 2023
6e37e4e
GH-629-Dan-review-two: interim commit
Mar 10, 2023
59f9ef4
GH-629-Dan-review-two: a lot more fixed; a savepoint before trying to…
Mar 10, 2023
0217cd9
GH-629-Dan-review-two: Sorry, new MERGE OF MASTER is coming; otherwis…
Mar 10, 2023
042b4ba
GH-629: mater merge completed with all tests passing
Mar 13, 2023
222e38d
GH-629-Dan-review-teo: another big chunk of fixed things for the review
Mar 15, 2023
c4d2eef
GH-629-Dan-review-two: some more fixed items
Mar 21, 2023
d1e3a91
GH-629-Dan-review-two: made error distinguishing eaiser and nicer for…
Mar 23, 2023
bf00559
GH-629-Dan-review-two: a big bunch of other newly fixed things; a few…
Mar 25, 2023
debbba3
GH-629-Dan-review-two: some significant refactoring of a DAO fn
Mar 27, 2023
4ee0053
GH-629-Dan-review-two: interim commit
Mar 28, 2023
832a910
GH-629-Dan-review-two: a save-point, still dealing with the finished …
Mar 30, 2023
0e224d8
GH-629-Dan-review-two: all I know of from the review is fixed by now
Mar 30, 2023
f1ae437
GH-629-Dan-review-two: merged master in
Mar 30, 2023
0d087ce
GH-629: accountant/dao_utils is fixed
Apr 23, 2023
b23d964
GH-629: pending_payable_dao fixed
Apr 24, 2023
093f4eb
GH-629: mostly fixes of stuff tied up with scanners, daos and a littl…
Apr 26, 2023
4e46301
GH-629: error messaging for failed mark_pending_payables_rowids fixed
Apr 27, 2023
a38fd67
GH-629: added a few more fixes
Apr 27, 2023
7685f16
GH-629: save point before trying to come up with better explanation o…
Apr 27, 2023
764f2ba
GH-629: last fixes, also better examples for how to implement Arbitra…
Apr 28, 2023
8b803b1
GH-629: review 3 answered
May 3, 2023
c6f63cb
GH-629: little fix in a test
May 4, 2023
b1f6c46
GH-629: getting cargo files up to date in their version
May 4, 2023
e541a89
GH-629: master merged in
May 4, 2023
67584f5
GH-629: fixes for old bad comments
May 4, 2023
9d88064
GH-629: found and fix issue in utils for special integration test wit…
May 5, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion automap/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion dns_utility/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

103 changes: 67 additions & 36 deletions masq/src/commands/financials_command/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -328,7 +328,7 @@ mod tests {
UiPayableAccount, UiReceivableAccount,
};
use masq_lib::ui_gateway::MessageBody;
use masq_lib::utils::array_of_borrows_to_vec;
use masq_lib::utils::slice_of_strs_to_vec_of_strings;
use regex::Regex;
use std::sync::{Arc, Mutex};

Expand Down Expand Up @@ -375,7 +375,7 @@ mod tests {
.transact_result(Ok(meaningless_financials_response()))
.transact_params(&transact_params_arc);
let subject = factory
.make(&array_of_borrows_to_vec(&[
.make(&slice_of_strs_to_vec_of_strings(&[
"financials",
"--top",
"20",
Expand Down Expand Up @@ -412,7 +412,7 @@ mod tests {
.transact_result(Ok(meaningless_financials_response()))
.transact_params(&transact_params_arc);
let subject = factory
.make(&array_of_borrows_to_vec(&[
.make(&slice_of_strs_to_vec_of_strings(&[
"financials",
"--top",
"10",
Expand Down Expand Up @@ -449,7 +449,7 @@ mod tests {
.transact_result(Ok(meaningless_financials_response()))
.transact_params(&transact_params_arc);
let subject = factory
.make(&array_of_borrows_to_vec(&[
.make(&slice_of_strs_to_vec_of_strings(&[
"financials",
"--payable",
"200-450|480000-158000008",
Expand Down Expand Up @@ -494,7 +494,7 @@ mod tests {
fn supplied_big_masq_values_are_not_fatal_for_non_decimal_values() {
let factory = CommandFactoryReal::new();
let result = factory
.make(&array_of_borrows_to_vec(&[
.make(&slice_of_strs_to_vec_of_strings(&[
"financials",
"--payable",
"200-450|480000-15800000800045",
Expand All @@ -512,7 +512,7 @@ mod tests {
fn supplied_big_masq_values_are_not_fatal_for_decimal_values() {
let factory = CommandFactoryReal::new();
let result = factory
.make(&array_of_borrows_to_vec(&[
.make(&slice_of_strs_to_vec_of_strings(&[
"financials",
"--payable",
"200-450|480045454455.00-158000008000455",
Expand All @@ -534,7 +534,10 @@ mod tests {
fn command_factory_no_stats_arg_is_forbidden_if_no_other_arg_present() {
let factory = CommandFactoryReal::new();

let result = factory.make(&array_of_borrows_to_vec(&["financials", "--no-stats"]));
let result = factory.make(&slice_of_strs_to_vec_of_strings(&[
"financials",
"--no-stats",
]));

let err = match result {
Err(CommandFactoryError::CommandSyntax(err_msg)) => err_msg,
Expand All @@ -556,7 +559,7 @@ mod tests {
) {
let factory = CommandFactoryReal::new();

let result = factory.make(&array_of_borrows_to_vec(args));
let result = factory.make(&slice_of_strs_to_vec_of_strings(args));

let err = match result {
Ok(_) => panic!("we expected error but got ok"),
Expand Down Expand Up @@ -634,7 +637,7 @@ mod tests {
fn ordered_can_be_combined_with_top_records_only() {
let factory = CommandFactoryReal::new();

let result = factory.make(&array_of_borrows_to_vec(&[
let result = factory.make(&slice_of_strs_to_vec_of_strings(&[
"financials",
"--receivable",
"5-100|600-7000",
Expand All @@ -661,8 +664,13 @@ mod tests {

#[test]
fn ordered_have_just_two_possible_values() {
let args =
array_of_borrows_to_vec(&["financials", "--top", "11", "--ordered", "upside-down"]);
let args = slice_of_strs_to_vec_of_strings(&[
"financials",
"--top",
"11",
"--ordered",
"upside-down",
]);

let result = financials_subcommand()
.get_matches_from_safe(args)
Expand All @@ -688,8 +696,15 @@ mod tests {
#[test]
fn financials_command_allows_shorthands_including_top_records() {
let transact_params_arc = Arc::new(Mutex::new(vec![]));
let args =
array_of_borrows_to_vec(&["financials", "-g", "-t", "123", "-o", "balance", "-n"]);
let args = slice_of_strs_to_vec_of_strings(&[
"financials",
"-g",
"-t",
"123",
"-o",
"balance",
"-n",
]);
let mut context = CommandContextMock::new()
.transact_params(&transact_params_arc)
.transact_result(Ok(meaningless_financials_response()));
Expand Down Expand Up @@ -719,7 +734,7 @@ mod tests {
#[test]
fn financials_command_allows_shorthands_including_custom_query() {
let transact_params_arc = Arc::new(Mutex::new(vec![]));
let args = array_of_borrows_to_vec(&[
let args = slice_of_strs_to_vec_of_strings(&[
"financials",
"-g",
"-p",
Expand Down Expand Up @@ -767,8 +782,14 @@ mod tests {
#[test]
fn financials_command_top_records_ordered_by_age_instead_of_balance() {
let transact_params_arc = Arc::new(Mutex::new(vec![]));
let args =
array_of_borrows_to_vec(&["financials", "--no-stats", "--top", "7", "-o", "age"]);
let args = slice_of_strs_to_vec_of_strings(&[
"financials",
"--no-stats",
"--top",
"7",
"-o",
"age",
]);
let mut context = CommandContextMock::new()
.transact_params(&transact_params_arc)
.transact_result(Ok(meaningless_financials_response()));
Expand Down Expand Up @@ -797,7 +818,7 @@ mod tests {

#[test]
fn parse_top_records_arg_with_ordered_defaulted_to_balance() {
let args = array_of_borrows_to_vec(&["financials", "--top", "11"]);
let args = slice_of_strs_to_vec_of_strings(&["financials", "--top", "11"]);
let matches = financials_subcommand().get_matches_from_safe(args).unwrap();

let result = FinancialsCommand::parse_top_records_args(&matches);
Expand All @@ -813,8 +834,11 @@ mod tests {

#[test]
fn financials_command_allows_obscure_leading_zeros_in_positive_numbers() {
let args =
array_of_borrows_to_vec(&["financials", "--receivable", "05000-0010000|040-050"]);
let args = slice_of_strs_to_vec_of_strings(&[
"financials",
"--receivable",
"05000-0010000|040-050",
]);

let result = FinancialsCommand::new(&args).unwrap();

Expand Down Expand Up @@ -845,7 +869,11 @@ mod tests {

#[test]
fn financials_command_allows_obscure_leading_zeros_in_negative_numbers() {
let args = array_of_borrows_to_vec(&["financials", "--receivable", "5000-10000|-050--040"]);
let args = slice_of_strs_to_vec_of_strings(&[
"financials",
"--receivable",
"5000-10000|-050--040",
]);

let result = FinancialsCommand::new(&args).unwrap();

Expand Down Expand Up @@ -927,9 +955,12 @@ mod tests {
#[test]
fn are_both_sets_to_be_displayed_works_for_top_records() {
//top records always print as a pair so it always consists of both sets
let subject =
FinancialsCommand::new(&array_of_borrows_to_vec(&["financials", "--top", "20"]))
.unwrap();
let subject = FinancialsCommand::new(&slice_of_strs_to_vec_of_strings(&[
"financials",
"--top",
"20",
]))
.unwrap();

let result = subject.are_both_sets_to_be_displayed();

Expand All @@ -938,7 +969,7 @@ mod tests {

#[test]
fn are_both_sets_to_be_displayed_works_for_custom_query_with_only_payable() {
let subject = FinancialsCommand::new(&array_of_borrows_to_vec(&[
let subject = FinancialsCommand::new(&slice_of_strs_to_vec_of_strings(&[
"financials",
"--payable",
"20-40|60-120",
Expand All @@ -952,7 +983,7 @@ mod tests {

#[test]
fn are_both_sets_to_be_displayed_works_for_custom_query_with_only_receivable() {
let subject = FinancialsCommand::new(&array_of_borrows_to_vec(&[
let subject = FinancialsCommand::new(&slice_of_strs_to_vec_of_strings(&[
"financials",
"--receivable",
"20-40|-50-120",
Expand All @@ -966,7 +997,7 @@ mod tests {

#[test]
fn are_both_sets_to_be_displayed_works_for_custom_query_with_both_parts() {
let subject = FinancialsCommand::new(&array_of_borrows_to_vec(&[
let subject = FinancialsCommand::new(&slice_of_strs_to_vec_of_strings(&[
"financials",
"--receivable",
"20-40|-50-120",
Expand Down Expand Up @@ -1043,7 +1074,7 @@ mod tests {
fn financials_command_stats_and_top_records_default_units_as_masq() {
let transact_params_arc = Arc::new(Mutex::new(vec![]));
let expected_response = response_with_stats_and_either_top_records_or_top_queries(true);
let args = array_of_borrows_to_vec(&["financials", "--top", "123"]);
let args = slice_of_strs_to_vec_of_strings(&["financials", "--top", "123"]);
let mut context = CommandContextMock::new()
.transact_params(&transact_params_arc)
.transact_result(Ok(expected_response.tmb(31)));
Expand Down Expand Up @@ -1104,7 +1135,7 @@ mod tests {
fn financials_command_stats_and_custom_query_demanded_default_units_as_masq() {
let transact_params_arc = Arc::new(Mutex::new(vec![]));
let expected_response = response_with_stats_and_either_top_records_or_top_queries(false);
let args = array_of_borrows_to_vec(&[
let args = slice_of_strs_to_vec_of_strings(&[
"financials",
"--payable",
"0-350000|0.005-9",
Expand Down Expand Up @@ -1179,7 +1210,7 @@ mod tests {
fn financials_command_statistics_and_top_records_with_gwei_precision() {
let transact_params_arc = Arc::new(Mutex::new(vec![]));
let expected_response = response_with_stats_and_either_top_records_or_top_queries(true);
let args = array_of_borrows_to_vec(&["financials", "--top", "123", "--gwei"]);
let args = slice_of_strs_to_vec_of_strings(&["financials", "--top", "123", "--gwei"]);
let mut context = CommandContextMock::new()
.transact_params(&transact_params_arc)
.transact_result(Ok(expected_response.tmb(31)));
Expand Down Expand Up @@ -1241,7 +1272,7 @@ mod tests {
fn financials_command_statistics_and_custom_query_with_gwei_precision() {
let transact_params_arc = Arc::new(Mutex::new(vec![]));
let expected_response = response_with_stats_and_either_top_records_or_top_queries(false);
let args = array_of_borrows_to_vec(&[
let args = slice_of_strs_to_vec_of_strings(&[
"financials",
"--payable",
"0-350000|0.005-9",
Expand Down Expand Up @@ -1333,7 +1364,7 @@ mod tests {
}]),
}),
};
let args = array_of_borrows_to_vec(&[
let args = slice_of_strs_to_vec_of_strings(&[
"financials",
"--payable",
"0-350000|5",
Expand Down Expand Up @@ -1408,7 +1439,7 @@ mod tests {
receivable_opt: Some(vec![]),
}),
};
let args = array_of_borrows_to_vec(&["financials", "--top", "10"]);
let args = slice_of_strs_to_vec_of_strings(&["financials", "--top", "10"]);
let mut context = CommandContextMock::new()
.transact_params(&transact_params_arc)
.transact_result(Ok(expected_response.tmb(31)));
Expand Down Expand Up @@ -1485,7 +1516,7 @@ mod tests {
receivable_opt: None,
}),
};
let args = array_of_borrows_to_vec(&[
let args = slice_of_strs_to_vec_of_strings(&[
"financials",
"--payable",
"0-400000|355-6000",
Expand Down Expand Up @@ -1598,7 +1629,7 @@ mod tests {
]),
}),
};
let args = array_of_borrows_to_vec(&["financials", "--no-stats", "--top", "7"]);
let args = slice_of_strs_to_vec_of_strings(&["financials", "--no-stats", "--top", "7"]);
let mut context = CommandContextMock::new()
.transact_params(&transact_params_arc)
.transact_result(Ok(expected_response.tmb(31)));
Expand Down Expand Up @@ -1679,7 +1710,7 @@ mod tests {
receivable_opt: None,
}),
};
let args = array_of_borrows_to_vec(&[
let args = slice_of_strs_to_vec_of_strings(&[
"financials",
"--payable",
"3000-40000|88-1000",
Expand Down Expand Up @@ -1755,7 +1786,7 @@ mod tests {
]),
}),
};
let args = array_of_borrows_to_vec(&[
let args = slice_of_strs_to_vec_of_strings(&[
"financials",
"--no-stats",
"--receivable",
Expand Down
1 change: 0 additions & 1 deletion masq/src/terminal/terminal_interface.rs
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,6 @@ impl TerminalWrapper {
Self::configure_interface_generic(Box::new(Self::result_wrapper_for_in_memory_terminal))
}

#[allow(clippy::unnecessary_wraps)]
pub fn result_wrapper_for_in_memory_terminal() -> std::io::Result<test_cfg::MemoryTerminal> {
Ok(test_cfg::MemoryTerminal::new())
}
Expand Down
2 changes: 1 addition & 1 deletion masq_lib/src/utils.rs
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ pub fn exit_process_with_sigterm(message: &str) {
}
}

pub fn array_of_borrows_to_vec(slice: &[&str]) -> Vec<String> {
pub fn slice_of_strs_to_vec_of_strings(slice: &[&str]) -> Vec<String> {
slice
.iter()
.map(|item| item.to_string())
Expand Down
9 changes: 4 additions & 5 deletions multinode_integration_tests/tests/verify_bill_payment.rs
Original file line number Diff line number Diff line change
Expand Up @@ -409,19 +409,18 @@ fn build_config(
payment_thresholds: PaymentThresholds,
wallet_derivation_path: String,
) -> (NodeStartupConfig, Wallet) {
let (serving_node_wallet, serving_node_secret) =
make_node_wallet(seed, wallet_derivation_path.as_str());
let (node_wallet, node_secret) = make_node_wallet(seed, wallet_derivation_path.as_str());
let config = NodeStartupConfigBuilder::standard()
.blockchain_service_url(server_url_holder.url())
.chain(Chain::Dev)
.payment_thresholds(payment_thresholds)
.consuming_wallet_info(ConsumingWalletInfo::PrivateKey(serving_node_secret))
.consuming_wallet_info(ConsumingWalletInfo::PrivateKey(node_secret))
.earning_wallet_info(EarningWalletInfo::Address(format!(
"{}",
serving_node_wallet.clone()
node_wallet.clone()
)))
.build();
(config, serving_node_wallet)
(config, node_wallet)
}

fn expire_payables(path: PathBuf) {
Expand Down
Loading