Skip to content

GH-580: test for data_directory_from_context with opt dir and implementation … #248

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 79 commits into from
Aug 21, 2023
Merged
Show file tree
Hide file tree
Changes from 62 commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
3307e4b
test for data_directory_from_context with opt dir and implementation …
Mar 22, 2023
9894edf
fixed naming and optimizing call for add_chain_specific_directories()
Mar 23, 2023
59fb959
add_chain_specific_directories to tests in bootstrapper.rs, mod.rs, …
Mar 23, 2023
db1b485
calculate_configured_setup returns None -> panicked at 'called on a …
Mar 24, 2023
4c4f9af
formatting
Mar 24, 2023
ecfd9ec
git repair commit
Mar 24, 2023
ff95c7a
cleaning misstake
Mar 24, 2023
bc1fbb3
commit changes - move to laptop
Mar 31, 2023
9270242
fix tests failing on the chain specific data direcotry
Apr 4, 2023
ed5b782
changes based on Berts review comments
czarte Apr 5, 2023
1134be8
formatting
czarte Apr 5, 2023
27a29d7
remove todo causing crash
czarte Apr 5, 2023
2b261ab
half way to fix cleanup_after_deceased_clients_integration
Apr 10, 2023
dac9533
failing on creating log file in node_logs_panic_integration
Apr 10, 2023
db221f5
change joins for add_chain_specific_directories fn
Apr 13, 2023
c0f205a
remove unused import
Apr 13, 2023
9def8c3
add_chain_specific_directories moved to masq_lib::utils, introduced n…
czarte May 2, 2023
0ab8322
remove comments
czarte May 3, 2023
226567d
remove comments, replace joins wtih add_chain_specific_directories fn…
czarte May 5, 2023
34591ca
Revert "remove unused import"
May 5, 2023
f9133db
pull and resolve conflicts
May 11, 2023
426fc7f
vec! with foreach for test server_initializer_collected_params_senses…
May 14, 2023
0214c09
add more scenarios for test with custom data_directory_opt in test se…
May 15, 2023
13c586e
continue of solving Bert's comments, start of modifying UI to show fu…
May 15, 2023
54d4e40
try to retrieve chain-name from inner.values
May 15, 2023
48ab4bc
chain_name collecting with map().collect, value_value collecting with…
May 16, 2023
aa79c33
move match for value_value to separate fn to clar dump_setup
May 16, 2023
9a0938c
chainge test for right fn calls, added message about changed data-dir…
May 17, 2023
1d697a9
change of chain_specific_data_directory to add alway chain_specific_d…
May 17, 2023
c2b7023
new integration test for specific data_directory ensure_data_director…
May 19, 2023
05d3103
fix Default arm for directory path enrichment with chain specific dir…
czarte May 23, 2023
264d137
integration test to ensure setup command do the right job to add chai…
czarte May 23, 2023
bdd7794
change data-directory, comparsion and panic messages in chain_specifi…
May 24, 2023
33f8c27
fix formatting for expected in chain_specific_directory test, remove …
czarte May 24, 2023
ec126a6
finalize test for setup_reporter calculate_fundamentals_with_chain_sp…
May 24, 2023
6c312e2
commented code to test in linux
May 27, 2023
c15c53b
compilable version to test in linux
May 27, 2023
79cfa3f
move changes to desktop
czarte Jun 12, 2023
6ab99f7
for bert from my linux machine
czarte Jun 12, 2023
a6903ff
GH-580: showing the right direction
Jun 12, 2023
29b3d1f
fixing more tests
czarte Jun 13, 2023
1175952
GH-580: ongoing repairment in setup_reporter
Jun 13, 2023
32a02cf
GH-580: fixed data directory niches that were causing a lot of troubles
Jun 13, 2023
328c91c
remove unvanted println
czarte Jun 19, 2023
4ae75f3
formatting
czarte Jun 19, 2023
a2701b5
fixed test calculate_setup_with_chain_specific_dir to pass on mac, re…
czarte Jun 20, 2023
ed12786
remove unused imports and chain from mock node
czarte Jun 23, 2023
ee812c6
remove unused code from setup_command
czarte Jun 26, 2023
eb13061
resolving review comments - add closure for retriving data-dir and chain
Jul 6, 2023
73d7856
resolving comments: cleaning setup_command tests, cleaning setup_repo…
Jul 6, 2023
141a43b
try to send less data to determine_setup_value_of_set_data_directory fn
Jul 11, 2023
8108ef5
remove unused data from SetupReporterReal::determine_setup_value_of_s…
czarte Jul 11, 2023
381210c
formatting
Jul 12, 2023
1d1e3cc
fixed test for get_modified_setup::get_modified_setup_user_specified_…
czarte Jul 12, 2023
2ae1922
resolved review comments on ensure_data_directory_has_specific_chain_…
czarte Jul 13, 2023
4385c89
split of test calculate_setup_with_chain_specific_dir into two separa…
czarte Jul 13, 2023
53065b5
resolving review comments, renaming, optimization, fixing tests
czarte Jul 13, 2023
e4cdffa
resolving review comments - mainly renaming
Jul 15, 2023
0daab9a
Half way to get rid of DirsWrapper from go fn in DumpConfigRunnerReal
Jul 15, 2023
575d76f
remove DirsWrapper from arguments to DumpConfigRunner for DumpConfigR…
Jul 16, 2023
648b94e
fixed windows tests for determine_fundamentals
czarte Jul 17, 2023
23caf03
renaming functions, removing TODOs and removing unnecesary attributes…
czarte Jul 17, 2023
979ffbe
resolving conflicts after merged master into GH-580
czarte Jul 21, 2023
8dd6f5d
fixed node_chain_specific_data_directory and impl MASQRealNode plus t…
czarte Jul 24, 2023
0f5ea83
fixed node_creates_log_file_with_heading_integration
czarte Jul 25, 2023
d5a463f
remove unused import and debug print, tests failing in ci/all.sh but …
czarte Jul 25, 2023
b23dfef
fix paths in tests for windows: setup_reporter
czarte Jul 30, 2023
3ccbbdb
fix windows paths for test server_initializer_collected_params_senses…
czarte Jul 31, 2023
ce2036f
fix setup_command and broadcast_handler tests: added missing data-dir…
czarte Aug 1, 2023
3e13944
setup_command removign ropsten and correcting UiResponseValue, broadc…
czarte Aug 3, 2023
d575707
correction of DATA_DIRECTORY_HELP
czarte Aug 3, 2023
62986f8
lazy_static for STANDARD_ARG_LIST in shared_schema
czarte Aug 4, 2023
432fed1
separate help for --data-directory in Node, Config Dump and UI
czarte Aug 8, 2023
3cf0972
remove duplication of data-directory clap app argument, add dirs to m…
czarte Aug 10, 2023
e3e12f2
move data-directory back to shared schema::shared_app::head
czarte Aug 10, 2023
63deecd
remove typo misstake
czarte Aug 11, 2023
2c4fc68
apply Dans comments on review 6, put code in the order, formatting an…
czarte Aug 15, 2023
7e89ede
remove unused import
czarte Aug 15, 2023
14f9e5a
resolve DATA_DIRECTORY_DAEMON_HELP for masq UI
czarte Aug 15, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -136,3 +136,6 @@ node.log
lnk*.tmp
/*/target/
node_ui/static/binaries/

# VSCode
.vscode
138 changes: 135 additions & 3 deletions masq/src/commands/setup_command.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,15 @@ use crate::terminal::terminal_interface::TerminalWrapper;
use clap::{value_t, App, SubCommand};
use masq_lib::constants::SETUP_ERROR;
use masq_lib::implement_as_any;
use masq_lib::messages::{
UiSetupBroadcast, UiSetupInner, UiSetupRequest, UiSetupRequestValue, UiSetupResponse,
};
use masq_lib::messages::{UiSetupBroadcast, UiSetupInner, UiSetupRequest, UiSetupRequestValue, UiSetupResponse, UiSetupResponseValue, UiSetupResponseValueStatus};
use masq_lib::shared_schema::shared_app;
use masq_lib::short_writeln;
use masq_lib::utils::index_of_from;
#[cfg(test)]
use std::any::Any;
use std::fmt::Debug;
use std::io::Write;
use std::iter::Iterator;

pub const SETUP_COMMAND_TIMEOUT_MILLIS: u64 = 30000;

Expand Down Expand Up @@ -72,6 +71,7 @@ impl SetupCommand {
}
})
.collect::<Vec<UiSetupRequestValue>>();

values.sort_by(|a, b| {
a.name
.partial_cmp(&b.name)
Expand Down Expand Up @@ -107,6 +107,10 @@ impl SetupCommand {
.expect("String comparison failed")
});
short_writeln!(stdout, "{:29} {:64} {}", "NAME", "VALUE", "STATUS");
let chain_and_data_dir = |p: &UiSetupResponseValue| { (p.name.to_owned(), (p.value.clone(), p.status)) };
let chain = inner.values.iter().find(|&p| p.name.as_str() == "chain").map(chain_and_data_dir).expect("Chain name is missing in setup cluster!");
let data_directory = inner.values.iter().find(|&p| p.name.as_str() == "data-directory").map(chain_and_data_dir).expect("data-directory is missing in setup cluster!");

inner.values.into_iter().for_each(|value| {
short_writeln!(
stdout,
Expand All @@ -130,6 +134,14 @@ impl SetupCommand {
"NOTE: no changes were made to the setup because the Node is currently running.\n"
);
}
if chain.1.1 != UiSetupResponseValueStatus::Default
|| data_directory.1.1 != UiSetupResponseValueStatus::Default
{
short_writeln!(
stdout,
"NOTE: your data directory was modified to match the chain parameter.\n"
);
}
}
}

Expand Down Expand Up @@ -338,4 +350,124 @@ ERRORS:
ip No sir, I don't like it.\n\
\n");
}

#[derive(Debug, PartialEq, Eq)]
pub struct SetupCommandData {
chain_str: Option<String>,
data_directory: Option<String>,
chain_name_expected: Option<&'static str>,
data_directory_expected: Option<&'static str>,
note_expected: bool,
status_chain: UiSetupResponseValueStatus,
status_data_dir: UiSetupResponseValueStatus
}

#[test]
fn setup_command_with_data_directory_shows_right_path() {
#[rustfmt::skip]
vec![
SetupCommandData {
chain_str: None,
data_directory: None,
chain_name_expected: Some("polygon-mainnet"),
data_directory_expected: Some("/home/cooga/.local/MASQ/polygon-mainnet"),
note_expected: false,
status_chain: UiSetupResponseValueStatus::Default,
status_data_dir: UiSetupResponseValueStatus::Default,
},
SetupCommandData {
chain_str: Some("polygon-mumbai".to_owned()),
data_directory: None,
chain_name_expected: Some("polygon-mumbai"),
data_directory_expected: Some("/home/cooga/.local/MASQ/polygon-mumbai"),
note_expected: true,
status_chain: UiSetupResponseValueStatus::Set,
status_data_dir: UiSetupResponseValueStatus::Default,
},
SetupCommandData {
chain_str: None,
data_directory: Some("booga".to_owned()),
chain_name_expected: Some("polygon-mainnet"),
data_directory_expected: Some("booga/polygon-mainnet"),
note_expected: true,
status_chain: UiSetupResponseValueStatus::Default,
status_data_dir: UiSetupResponseValueStatus::Set,
},
SetupCommandData {
chain_str: Some("polygon-mumbai".to_owned()),
data_directory: Some("booga/polygon-mumbai".to_owned()),
chain_name_expected: Some("polygon-mumbai"),
data_directory_expected: Some("booga/polygon-mumbai/polygon-mumbai"),
note_expected: true,
status_chain: UiSetupResponseValueStatus::Set,
status_data_dir: UiSetupResponseValueStatus::Set,
},
].iter().for_each(|
data| {
let note_expected_real = match data.note_expected {
true => "\nNOTE: your data directory was modified to match the chain parameter.\n",
_ => ""
};
let status_data_dir_str = match data.status_data_dir {
Default => "Default",
Set => "Set",
Configured => "Configured",
UiSetupResponseValueStatus::Blank => "Blank",
UiSetupResponseValueStatus::Required => "Required"
};
let status_chain_str = match data.status_chain {
Default => "Default",
Set => "Set",
Configured => "Configured",
UiSetupResponseValueStatus::Blank => "Blank",
UiSetupResponseValueStatus::Required => "Required"
};
let expected = format!("\
NAME VALUE STATUS\n\
{:29} {:64} {}\n{:29} {:64} {}\n{}\n",
"chain",
data.chain_name_expected.unwrap(),
status_chain_str,
"data-directory",
data.data_directory_expected.unwrap(),
status_data_dir_str,
note_expected_real
);
let chain_real = match &data.chain_str { Some(chain) => chain, _ => "polygon-mainnet" };
let data_directory_real = match &data.data_directory {
Some(dir) => format!("{}/{}", dir, chain_real),
_ => format!("/home/cooga/.local/MASQ/{}", chain_real)
};
process_setup_command_for_given_attributes(
chain_real,
&data_directory_real,
&expected,
data.status_chain,
data.status_data_dir
);
});
}

fn process_setup_command_for_given_attributes(
chain: &str,
data_directory: &str,
expected: &str,
status_chain: UiSetupResponseValueStatus,
status_data_dir: UiSetupResponseValueStatus,
) {
let message = UiSetupResponse {
running: false,
values: vec![
UiSetupResponseValue::new("chain", chain, status_chain),
UiSetupResponseValue::new("data-directory", data_directory, status_data_dir),
],
errors: vec![],
};
let (stream_factory, handle) = TestStreamFactory::new();
let (mut stdout, _) = stream_factory.make();

SetupCommand::dump_setup(UiSetupInner::from(message), &mut stdout);

assert_eq!(handle.stdout_so_far(), expected);
}
}
39 changes: 39 additions & 0 deletions masq/tests/chain_specific_directory_test_integration.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
// Copyright (c) 2019, MASQ (https://masq.ai) and/or its affiliates. All rights reserved.

use crate::utils::{DaemonProcess, MasqProcess};
use masq_lib::utils::find_free_port;
use std::thread;
use std::time::Duration;

mod utils;

#[test]
fn ensure_data_directory_has_specific_chain_directory_within_integration() {
let port = find_free_port();
let daemon_handle = DaemonProcess::new().start(port.clone());
let masq_handle = MasqProcess::new().start_noninteractive(vec![
"--ui-port",
*&port.to_string().to_owned().as_str(),
"setup",
]);
let (stdout, _stderr, _exit_code) = masq_handle.stop();
let mut masq_handle2 = MasqProcess::new().start_interactive(port, true);
let mut stdin_handle = masq_handle2.create_stdin_handle();

stdin_handle.type_command("setup --data-directory /home/booga/masqhome/polygon-mainnet");

thread::sleep(Duration::from_millis(1000));

stdin_handle.type_command("exit");

let (stdout2, _stderr2, _exit_code2) = masq_handle2.stop();
let expected = format!(
"{:29} {:64} {}",
"data-directory", "/home/booga/masqhome/polygon-mainnet", "Set"
);

assert!(!stdout.contains("MASQ/polygon-mainnet/MASQ/polygon-mainnet Default"), "Wrong directory: duplication of /MASQ/polygon-mainnet when Default");
assert!(stdout2.contains(&expected), "Wrong directory: missing chain specific directory when Set:\nstdout: {}\n", stdout2);

daemon_handle.kill();
}
8 changes: 5 additions & 3 deletions masq/tests/utils.rs
Original file line number Diff line number Diff line change
Expand Up @@ -107,10 +107,12 @@ impl StdinHandle {
}
pub fn type_command(&mut self, command: &str) {
match self.stdin.write_fmt(format_args!("{}\n", command)) {
Ok(_) => (),
Ok(_) => match self.stdin.flush() {
Ok(_) => (),
Err(e) => panic!("flush failed in type_command: {}", e),
},
Err(e) => {
eprintln!("{}", e);
panic!("type_command failed: {}", e)
panic!("type_command write failed: {}", e)
}
}
}
Expand Down
2 changes: 2 additions & 0 deletions masq_lib/src/multi_config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ macro_rules! values_m {
}};
}

#[derive(Debug)]
pub struct MultiConfig<'a> {
arg_matches: ArgMatches<'a>,
}
Expand Down Expand Up @@ -1101,6 +1102,7 @@ pub mod tests {
}

let result = ConfigFileVcl::new(&file_path, true).err().unwrap();

assert_contains(
&result.to_string(),
"doesn't make sense: parameter 'array' must have a scalar value, not an array value.",
Expand Down
14 changes: 14 additions & 0 deletions masq_lib/src/utils.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
// Copyright (c) 2019, MASQ (https://masq.ai) and/or its affiliates. All rights reserved.

use crate::blockchains::chains::Chain;
use lazy_static::lazy_static;
use std::fmt;
use std::fmt::{Debug, Display, Formatter};
use std::io;
use std::io::ErrorKind;
use std::net::{IpAddr, Ipv4Addr, SocketAddr, TcpListener, UdpSocket};
use std::path::{Path, PathBuf};
use std::str::FromStr;
use std::sync::{Arc, Mutex};

Expand Down Expand Up @@ -129,6 +131,18 @@ fn port_is_free_for_ip_addr(ip_addr: IpAddr, port: u16) -> bool {
true
}

pub fn add_masq_and_chain_directories(chain: Chain, local_data_dir: &Path) -> PathBuf {
let masq_dir = PathBuf::from(local_data_dir).join("MASQ");
add_chain_specific_directory(chain, masq_dir.as_path())
}

pub fn add_chain_specific_directory(chain: Chain, local_data_dir: &Path) -> PathBuf {
match local_data_dir.ends_with(chain.rec().literal_identifier) {
true => PathBuf::from(local_data_dir),
false => PathBuf::from(local_data_dir).join(chain.rec().literal_identifier)
}
}

pub fn localhost() -> IpAddr {
IpAddr::V4(Ipv4Addr::LOCALHOST)
}
Expand Down
2 changes: 1 addition & 1 deletion multinode_integration_tests/src/masq_node.rs
Original file line number Diff line number Diff line change
Expand Up @@ -274,7 +274,7 @@ impl MASQNodeUtils {
"-t",
name,
"cat",
&format!("/node_root/home/{}", CURRENT_LOGFILE_NAME),
&format!("/node_root/home/MASQ/dev/{}", CURRENT_LOGFILE_NAME),
]),
);
command.stdout_and_stderr()
Expand Down
16 changes: 13 additions & 3 deletions multinode_integration_tests/src/masq_real_node.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ use log::Level;
use masq_lib::blockchains::chains::Chain;
use masq_lib::constants::{CURRENT_LOGFILE_NAME, DEFAULT_UI_PORT};
use masq_lib::test_utils::utils::TEST_DEFAULT_MULTINODE_CHAIN;
use masq_lib::utils::localhost;
use masq_lib::utils::{add_masq_and_chain_directories, localhost};
use masq_lib::utils::{DEFAULT_CONSUMING_DERIVATION_PATH, DEFAULT_EARNING_DERIVATION_PATH};
use node_lib::blockchain::bip32::Bip32ECKeyProvider;
use node_lib::sub_lib::accountant::{
Expand All @@ -29,6 +29,7 @@ use std::fmt::Display;
use std::net::IpAddr;
use std::net::Ipv4Addr;
use std::net::SocketAddr;
use std::path::Path;
use std::rc::Rc;
use std::str::FromStr;
use std::string::ToString;
Expand Down Expand Up @@ -1151,6 +1152,8 @@ impl MASQRealNode {

fn extract_node_reference(name: &str) -> Result<NodeReference, String> {
let descriptor_regex = Self::descriptor_regex();
let chain_specific_directory =
add_masq_and_chain_directories(TEST_DEFAULT_MULTINODE_CHAIN, Path::new(DATA_DIRECTORY));
let mut retries_left = 25;
loop {
if retries_left <= 0 {
Expand All @@ -1163,7 +1166,11 @@ impl MASQRealNode {
name,
vec![
"cat",
&format!("{}/{}", DATA_DIRECTORY, CURRENT_LOGFILE_NAME),
&format!(
"{}/{}",
&chain_specific_directory.to_string_lossy(),
CURRENT_LOGFILE_NAME
),
],
) {
Ok(output) => {
Expand All @@ -1182,7 +1189,10 @@ impl MASQRealNode {
Err(e) => {
println!(
"Failed to cat logfile for {} at {}/{}: {}",
name, DATA_DIRECTORY, CURRENT_LOGFILE_NAME, e
name,
&chain_specific_directory.to_string_lossy(),
CURRENT_LOGFILE_NAME,
e
);
}
};
Expand Down
12 changes: 8 additions & 4 deletions multinode_integration_tests/src/utils.rs
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,7 @@ pub fn wait_for_chunk(stream: &mut TcpStream, timeout: &Duration) -> Result<Vec<

pub fn database_conn(node_name: &str) -> Box<dyn ConnectionWrapper> {
let db_initializer = DbInitializerReal::default();
let path = std::path::PathBuf::from(MASQRealNode::node_home_dir(
&MASQNodeUtils::find_project_root(),
node_name,
));
let path = std::path::PathBuf::from(node_chain_specific_data_directory(node_name));
db_initializer
.initialize(
&path,
Expand All @@ -86,6 +83,13 @@ pub fn database_conn(node_name: &str) -> Box<dyn ConnectionWrapper> {
.unwrap()
}

pub fn node_chain_specific_data_directory(node_name: &str) -> String {
format!(
"{}/MASQ/dev",
MASQRealNode::node_home_dir(&MASQNodeUtils::find_project_root(), node_name,)
)
}

pub fn config_dao(node_name: &str) -> Box<dyn ConfigDao> {
Box::new(ConfigDaoReal::new(database_conn(node_name)))
}
Expand Down
Loading