Skip to content
Closed
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion rust/benchmarks/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,16 @@ repository = "https://github.com/apache/arrow"
license = "Apache-2.0"
publish = false

[features]
simd = ["datafusion/simd"]
snmalloc = ["snmalloc-rs"]

[dependencies]
arrow = { path = "../arrow" }
parquet = { path = "../parquet" }
datafusion = { path = "../datafusion" }
structopt = { version = "0.3", default-features = false }
tokio = { version = "0.2", features = ["macros", "rt-core", "rt-threaded"] }
futures = "0.3"
env_logger = "^0.8"
env_logger = "^0.8"
snmalloc-rs = {version = "0.2", optional = true }
Comment thread
jorgecarleitao marked this conversation as resolved.
Outdated
6 changes: 6 additions & 0 deletions rust/benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,12 @@ The benchmark can then be run (assuming the data created from `dbgen` is in `/mn
cargo run --release --bin tpch -- benchmark --iterations 3 --path /mnt/tpch-dbgen --format tbl --query 1 --batch-size 4096
```

You can enable the features `simd` (to use SIMD instructions) and/or `snmalloc` (to use the snmalloc allocator) as features by passing them in as `--features`:

```
cargo run --release --features "simd snmalloc" --bin tpch -- benchmark --iterations 3 --path /mnt/tpch-dbgen --format tbl --query 1 --batch-size 4096
```

The benchmark program also supports CSV and Parquet input file formats and a utility is provided to convert from `tbl`
(generated by the `dbgen` utility) to CSV and Parquet.

Expand Down
4 changes: 4 additions & 0 deletions rust/benchmarks/src/bin/nyctaxi.rs
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,10 @@ use datafusion::physical_plan::collect;
use datafusion::physical_plan::csv::CsvReadOptions;
use structopt::StructOpt;

#[cfg(feature = "snmalloc")]
#[global_allocator]
static ALLOC: snmalloc_rs::SnMalloc = snmalloc_rs::SnMalloc;

#[derive(Debug, StructOpt)]
#[structopt(name = "Benchmarks", about = "Apache Arrow Rust Benchmarks.")]
struct Opt {
Expand Down
4 changes: 4 additions & 0 deletions rust/benchmarks/src/bin/tpch.rs
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,10 @@ use parquet::basic::Compression;
use parquet::file::properties::WriterProperties;
use structopt::StructOpt;

#[cfg(feature = "snmalloc")]
#[global_allocator]
static ALLOC: snmalloc_rs::SnMalloc = snmalloc_rs::SnMalloc;

#[derive(Debug, StructOpt)]
struct BenchmarkOpt {
/// Query number
Expand Down