Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 4 additions & 52 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion src/dfx/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ actix-files = "0.1.4"
atty = "0.2.13"
base64 = "0.11.0"
clap = "2.33.0"
console = "0.7.7"
console = "0.9.2"
crossbeam = "0.7.3"
flate2 = "1.0.11"
futures = "0.1.28"
Expand Down
94 changes: 71 additions & 23 deletions src/dfx/src/commands/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ use console::Style;
use ic_http_agent::CanisterId;
use indicatif::{ProgressBar, ProgressDrawTarget, ProgressStyle};
use rand::{thread_rng, Rng};
use std::cell::RefCell;
use std::collections::{HashMap, HashSet};
use std::ffi::OsStr;
use std::io::Read;
Expand All @@ -31,8 +32,22 @@ pub fn construct() -> App<'static, 'static> {
.takes_value(false)
.help(UserMessage::SkipFrontend.to_str()),
)
.arg(
Arg::with_name("extra-args")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should separate those between where they're used. extra-args has no semantic meaning (moc-args? compiler-args?). We're also building more and more of a Motoko-specific build system here, and I don't agree with that direction. We can chat offline.

Copy link
Contributor Author

@chenyan-dfinity chenyan-dfinity Jan 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed on slack, the proper way is to add flag into a config file like dfx.json. Before we have the design, it would be nice to have an CLI flag to pass in extra flags. I'm fine rename it to moc-args, but the intension is to provide a way to pass in extra arguments to any compilers. We can also add a comment in the user message warning that this flag is experimental and will go away.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any update on this? Let's place discussions/more details online.

I agree with the compiler arguments flag. I don't like the verbosity change -- or maybe I am missing something. I think we can use slog just in build instead, and then fully integrate it piece by piece.

.long("extra-args")
.takes_value(true)
.help(UserMessage::BuildArgs.to_str()),
)
.arg(
Arg::with_name("verbose")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verbosity should be a global flag. Also, this code is most likely to be wiped out when we start using slog, so why not do a PR now to do that instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verbosity should be a global flag.

I agree. Do you prefer to put the flag to the global scope with only build supporting it, or promote it to global after we have verbose mode for most commands?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, this code is most likely to be wiped out when we start using slog, so why not do a PR now to do that instead?

This again seems to be a global change, which should be done in a dedicated PR. I don't see an urgency to switch to slog, but I do see the urgency to enable verbose mode on build.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can start using slog here honestly. It doesn't have to be global. @chenyan-dfinity let's sync on this.

.long("verbose")
.takes_value(false)
.help(UserMessage::BuildVerbose.to_str()),
)
}

thread_local!(static VERBOSE: RefCell<bool> = RefCell::new(false));

fn get_asset_fn(assets: &AssetMap) -> String {
// Create the if/else series.
let mut cases = String::new();
Expand Down Expand Up @@ -79,9 +94,9 @@ struct MotokoParams<'a> {
// The following fields will not be used by self.to_args()
// TODO move input into self.to_args once inject_code is deprecated.
input: &'a Path,
verbose: bool,
surpress_warning: bool,
inject_code: bool,
extra_args: &'a str,
}

impl MotokoParams<'_> {
Expand All @@ -98,6 +113,10 @@ impl MotokoParams<'_> {
cmd.args(&["--actor-alias", name, canister_id]);
}
};
if !self.extra_args.is_empty() {
let args: Vec<_> = self.extra_args.split(' ').collect();
cmd.args(args);
};
}
}

Expand Down Expand Up @@ -137,7 +156,7 @@ fn motoko_compile(cache: &dyn Cache, params: &MotokoParams<'_>, assets: &AssetMa
.arg("--package")
.arg("stdlib")
.arg(&stdlib_path.as_path());
run_command(cmd, params.verbose, params.surpress_warning)?;
run_command(cmd, params.surpress_warning)?;

if params.inject_code {
std::fs::remove_file(input_path)?;
Expand Down Expand Up @@ -201,7 +220,7 @@ fn parse_moc_deps(line: &str) -> DfxResult<MotokoImport> {
}
None => match fullpath {
Some(fullpath) => {
let path = PathBuf::from(fullpath);
let path = PathBuf::from(fullpath).canonicalize()?;
if !path.is_file() {
return Err(DfxError::BuildError(BuildErrorKind::DependencyError(
format!("Cannot find import file {}", path.display()),
Expand All @@ -228,7 +247,7 @@ fn find_deps(cache: &dyn Cache, input_path: &Path, deps: &mut MotokoImports) ->

let mut cmd = cache.get_binary_command("moc")?;
let cmd = cmd.arg("--print-deps").arg(&input_path);
let output = run_command(cmd, false, false)?;
let output = run_command(cmd, false)?;

let output = String::from_utf8_lossy(&output.stdout);
for dep in output.lines() {
Expand All @@ -248,18 +267,16 @@ fn find_deps(cache: &dyn Cache, input_path: &Path, deps: &mut MotokoImports) ->
fn build_did_js(cache: &dyn Cache, input_path: &Path, output_path: &Path) -> DfxResult {
let mut cmd = cache.get_binary_command("didc")?;
let cmd = cmd.arg("--js").arg(&input_path).arg("-o").arg(&output_path);
run_command(cmd, false, false)?;
run_command(cmd, false)?;
Ok(())
}

fn run_command(
cmd: &mut std::process::Command,
verbose: bool,
surpress_warning: bool,
) -> DfxResult<Output> {
if verbose {
println!("{:?}", cmd);
}
fn run_command(cmd: &mut std::process::Command, surpress_warning: bool) -> DfxResult<Output> {
VERBOSE.with(|v| {
if *v.borrow() {
println!("{:?}", cmd);
}
});
let output = cmd.output()?;
if !output.status.success() {
Err(DfxError::BuildError(BuildErrorKind::CompilerError(
Expand Down Expand Up @@ -306,6 +323,7 @@ fn build_file(
name: &str,
id_map: &CanisterIdMap,
assets: &AssetMap,
extra_args: &str,
) -> DfxResult {
let canister_info = CanisterInfo::load(config, name).map_err(|_| {
BuildError(BuildErrorKind::CanisterNameIsNotInConfigError(
Expand Down Expand Up @@ -345,11 +363,11 @@ fn build_file(
},
surpress_warning: false,
inject_code: true,
verbose: false,
input: &input_path,
output: &output_wasm_path,
idl_path: &idl_dir_path,
idl_map: &id_map,
extra_args,
};
motoko_compile(cache.as_ref(), &params, assets)?;
// Generate IDL
Expand All @@ -362,11 +380,11 @@ fn build_file(
// Surpress the warnings the second time we call moc
surpress_warning: true,
inject_code: false,
verbose: false,
input: &input_path,
output: &output_idl_path,
idl_path: &idl_dir_path,
idl_map: &id_map,
extra_args,
};
motoko_compile(cache.as_ref(), &params, &HashMap::new())?;
std::fs::copy(&output_idl_path, &idl_file_path)?;
Expand Down Expand Up @@ -462,7 +480,6 @@ pub fn exec(env: &dyn Environment, args: &ArgMatches<'_>) -> DfxResult {
env.get_cache().install()?;

let green = Style::new().green().bold();

let status_bar = ProgressBar::new_spinner();
status_bar.set_draw_target(ProgressDrawTarget::stderr());
status_bar.set_message("Building canisters...");
Expand All @@ -475,6 +492,13 @@ pub fn exec(env: &dyn Environment, args: &ArgMatches<'_>) -> DfxResult {
}
let canisters = maybe_canisters.as_ref().unwrap();

if args.is_present("verbose") {
VERBOSE.with(|v| {
*v.borrow_mut() = true;
});
status_bar.set_draw_target(ProgressDrawTarget::hidden());
}

// Build canister id map and dependency graph
let mut id_map = HashMap::new();
let mut deps = HashMap::new();
Expand Down Expand Up @@ -518,10 +542,27 @@ pub fn exec(env: &dyn Environment, args: &ArgMatches<'_>) -> DfxResult {
);
build_stage_bar.enable_steady_tick(80);

if args.is_present("verbose") {
build_stage_bar.set_draw_target(ProgressDrawTarget::hidden());
}

let extra_args: &str = args.value_of("extra-args").unwrap_or("").trim();

// Build canister
for name in &seq.canisters {
build_stage_bar.println(&format!("{} canister {}", green.apply_to("Building"), name));
match build_file(env, &config, name, &seq.get_ids(name), &HashMap::new()) {
build_stage_bar.println(&format!(
"{} canister {}",
green.apply_to("Compiling"),
name
));
match build_file(
env,
&config,
name,
&seq.get_ids(name),
&HashMap::new(),
extra_args,
) {
Ok(()) => {}
Err(e) => {
build_stage_bar.abandon();
Expand All @@ -547,7 +588,7 @@ pub fn exec(env: &dyn Environment, args: &ArgMatches<'_>) -> DfxResult {
.current_dir(config.get_project_root())
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped());
run_command(&mut cmd, false, false)?;
run_command(&mut cmd, false)?;

build_stage_bar.inc(1);
build_stage_bar.println(&format!(
Expand Down Expand Up @@ -587,7 +628,14 @@ pub fn exec(env: &dyn Environment, args: &ArgMatches<'_>) -> DfxResult {
}
}

match build_file(env, &config, &name, &seq.get_ids(&name), &assets) {
match build_file(
env,
&config,
&name,
&seq.get_ids(&name),
&assets,
extra_args,
) {
Ok(()) => {}
Err(e) => {
build_stage_bar.abandon();
Expand Down Expand Up @@ -658,23 +706,23 @@ mod tests {
build_target: BuildTarget::Debug,
surpress_warning: false,
inject_code: true,
verbose: false,
input: &input_path,
output: Path::new("/out/file.wasm"),
idl_path: Path::new("."),
idl_map: &HashMap::new(),
extra_args: "",
};
motoko_compile(&cache, &params, &HashMap::new()).expect("Function failed.");

let params = MotokoParams {
build_target: BuildTarget::IDL,
surpress_warning: false,
inject_code: false,
verbose: false,
input: Path::new("/in/file.mo"),
output: Path::new("/out/file.did"),
idl_path: Path::new("."),
idl_map: &HashMap::new(),
extra_args: "",
};
motoko_compile(&cache, &params, &HashMap::new()).expect("Function failed (didl_compile)");
build_did_js(
Expand Down Expand Up @@ -738,7 +786,7 @@ mod tests {
)
.unwrap();

build_file(&env, &config, "name", &HashMap::new(), &HashMap::new())
build_file(&env, &config, "name", &HashMap::new(), &HashMap::new(), "")
.expect("Function failed - build_file");
assert!(output_path.exists());
}
Expand Down
2 changes: 2 additions & 0 deletions src/dfx/src/lib/message.rs
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ user_message!(
// dfx build
BuildCanister => "Builds all or specific canisters from the code in your project. By default, all canisters are built.",
SkipFrontend => "Skip building the frontend, only build the canisters.",
BuildVerbose => "Outputs all commands run by the build process.",
BuildArgs => "Specifies extra arguments passed to the compiler.",

// dfx config
ConfigureOptions => "Configures project options for your currently-selected project.",
Expand Down