Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion dev/tasks/homebrew-formulae/autobrew/apache-arrow.rb
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ def install
return 0;
}
EOS
system ENV.cxx, "test.cpp", "-std=c++11", "-I#{include}", "-L#{lib}", "-larrow", "-lparquet", "-lthrift", "-llz4", "-lboost_system", "-lboost_filesystem", "-lboost_regex", "-ldouble-conversion", "-lsnappy", "-o", "test"
system ENV.cxx, "test.cpp", "-std=c++11", "-I#{include}", "-L#{lib}", "-larrow", "-lparquet", "-lthrift", "-llz4", "-lboost_system", "-lboost_regex", "-ldouble-conversion", "-lsnappy", "-o", "test"
system "./test"
end
end
1 change: 0 additions & 1 deletion r/DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@ Roxygen: list(markdown = TRUE, r6 = FALSE, load = "source")
RoxygenNote: 6.1.99.9001
VignetteBuilder: knitr
Suggests:
covr,
dplyr,
hms,
knitr,
Expand Down
3 changes: 2 additions & 1 deletion r/NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@

## Data exploration

* Tables and RecordBatches now have `dplyr` methods, conditionally loaded if you have `dplyr` available. You can `select()`, `filter()`, etc. and work will be done where possible in Arrow memory. When necessary, data is pulled into R for further computation.
* Explore a multi-file dataset with `open_dataset()` and then use `dplyr` methods to `select()`, `filter()`, etc., and work will be done where possible in Arrow memory. When necessary, data is pulled into R for further computation. `dplyr` methods are conditionally loaded if you have `dplyr` available; it is not a hard dependency.
* Tables and RecordBatches also have `dplyr` methods.
* For exploration without `dplyr`, `[` methods for Tables, RecordBatches, Arrays, and ChunkedArrays now support natural row extraction operations. These use the C++ `Filter`, `Slice`, and `Take` methods for efficient access, depending on the type of selection vector.
* An experimental, lazily evaluated `array_expression` class has also been added, enabling among other things the ability to filter a Table with some function of Arrays, such as `arrow_table[arrow_table$var1 > 5, ]` without having to pull everything into R first.

Expand Down
18 changes: 3 additions & 15 deletions r/README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -74,20 +74,7 @@ install.packages("arrow", repos="https://dl.bintray.com/ursalabs/arrow-r")

These daily package builds are not official Apache releases and are not recommended for production use. They may be useful for testing bug fixes and new features under active development.

Linux users will need to build the Arrow C++ library from source. See "Development" below. Once you have the C++ library, you can install the R package from GitHub using the [`remotes`](https://remotes.r-lib.org/) package. From within an R session,

```r
# install.packages("remotes") # Or install "devtools", which includes remotes
remotes::install_github("apache/arrow/r")
```

or if you prefer to stay at the command line,

```shell
R -e 'remotes::install_github("apache/arrow/r")'
```

You can specify a particular commit, branch, or [release](https://github.com/apache/arrow/releases) to install by including a `ref` argument to `install_github()`. This is particularly useful to match the R package version to the C++ library version you've installed.
Linux users will need to build the Arrow C++ library from source, then install the R package from source, as described in the next section.

## Developing

Expand Down Expand Up @@ -118,7 +105,7 @@ dependencies, along with additional dev dependencies, from the git checkout:

```shell
cd ../../r
R -e 'install.packages(c("devtools", "roxygen2", "pkgdown")); devtools::install_dev_deps()'
R -e 'install.packages(c("devtools", "roxygen2", "pkgdown", "covr")); devtools::install_dev_deps()'
R CMD INSTALL .
```

Expand Down Expand Up @@ -190,6 +177,7 @@ devtools::document() # Update roxygen documentation
rmarkdown::render("README.Rmd") # To rebuild README.md
pkgdown::build_site() # To preview the documentation website
devtools::check() # All package checks; see also below
covr::package_coverage() # See test coverage statistics
```

Any of those can be run from the command line by wrapping them in `R -e
Expand Down
27 changes: 4 additions & 23 deletions r/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,28 +129,8 @@ These daily package builds are not official Apache releases and are not
recommended for production use. They may be useful for testing bug fixes
and new features under active development.

Linux users will need to build the Arrow C++ library from source. See
“Development” below. Once you have the C++ library, you can install
the R package from GitHub using the
[`remotes`](https://remotes.r-lib.org/) package. From within an R
session,

``` r
# install.packages("remotes") # Or install "devtools", which includes remotes
remotes::install_github("apache/arrow/r")
```

or if you prefer to stay at the command line,

``` shell
R -e 'remotes::install_github("apache/arrow/r")'
```

You can specify a particular commit, branch, or
[release](https://github.com/apache/arrow/releases) to install by
including a `ref` argument to `install_github()`. This is particularly
useful to match the R package version to the C++ library version you’ve
installed.
Linux users will need to build the Arrow C++ library from source, then
install the R package from source, as described in the next section.

## Developing

Expand Down Expand Up @@ -197,7 +177,7 @@ checkout:

``` shell
cd ../../r
R -e 'install.packages(c("devtools", "roxygen2", "pkgdown")); devtools::install_dev_deps()'
R -e 'install.packages(c("devtools", "roxygen2", "pkgdown", "covr")); devtools::install_dev_deps()'
R CMD INSTALL .
```

Expand Down Expand Up @@ -265,6 +245,7 @@ devtools::document() # Update roxygen documentation
rmarkdown::render("README.Rmd") # To rebuild README.md
pkgdown::build_site() # To preview the documentation website
devtools::check() # All package checks; see also below
covr::package_coverage() # See test coverage statistics
```

Any of those can be run from the command line by wrapping them in `R -e
Expand Down
10 changes: 5 additions & 5 deletions r/configure
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,11 @@ if [ "$ARROW_USE_OLD_CXXABI" ]; then
PKG_CFLAGS="$PKG_CFLAGS -D_GLIBCXX_USE_CXX11_ABI=0"
fi

# Set any user-defined CXXFLAGS
if [ "$ARROW_R_CXXFLAGS" ]; then
PKG_CFLAGS="$PKG_CFLAGS $ARROW_R_CXXFLAGS"
fi

# Test configuration
TEST_CMD="${CXXCPP} ${CPPFLAGS} ${PKG_CFLAGS} ${CXX11FLAGS} ${CXX11STD} -xc++ -"
echo "#include $PKG_TEST_HEADER" | ${TEST_CMD} >/dev/null 2>&1
Expand All @@ -129,11 +134,6 @@ else
PKG_CFLAGS=""
fi

# Set any user-defined CXXFLAGS
if [ "$ARROW_R_CXXFLAGS" ]; then
PKG_CFLAGS="$PKG_CFLAGS $ARROW_R_CXXFLAGS"
fi

# Write to Makevars
sed -e "s|@cflags@|$PKG_CFLAGS|" -e "s|@libs@|$PKG_LIBS|" src/Makevars.in > src/Makevars

Expand Down
2 changes: 1 addition & 1 deletion r/configure.win
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ RWINLIB="../windows/$(ls windows/ | grep ^arrow-)"
OPENSSL_LIBS="-lcrypto -lcrypt32"

PKG_CFLAGS="-I${RWINLIB}/include -DARROW_STATIC -DPARQUET_STATIC -DARROW_DS_STATIC -DARROW_R_WITH_ARROW"
PKG_LIBS="-L${RWINLIB}/lib"'$(subst gcc,,$(COMPILED_BY))$(R_ARCH) '"-L${RWINLIB}/lib"'$(R_ARCH) '"-lparquet -larrow_dataset -larrow -lthrift -lsnappy -lboost_regex-mt-s -lboost_filesystem-mt-s -lboost_system-mt-s -ldouble-conversion -lz ${OPENSSL_LIBS} -lws2_32"
PKG_LIBS="-L${RWINLIB}/lib"'$(subst gcc,,$(COMPILED_BY))$(R_ARCH) '"-L${RWINLIB}/lib"'$(R_ARCH) '"-lparquet -larrow_dataset -larrow -lthrift -lsnappy -lboost_regex-mt-s -lboost_system-mt-s -ldouble-conversion -lz ${OPENSSL_LIBS} -lws2_32"

echo "*** Writing Makevars.win"
sed -e "s|@cflags@|$PKG_CFLAGS|" -e "s|@libs@|$PKG_LIBS|" src/Makevars.in > src/Makevars.win
Expand Down
2 changes: 1 addition & 1 deletion r/tools/autobrew
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ fi

# Hardcode this for my custom autobrew build
rm -f $BREWDIR/lib/*.dylib
PKG_LIBS="-L$BREWDIR/lib -lparquet -larrow_dataset -larrow -lthrift -llz4 -lboost_system -lboost_filesystem -lboost_regex -ldouble-conversion -lsnappy"
PKG_LIBS="-L$BREWDIR/lib -lparquet -larrow_dataset -larrow -lthrift -llz4 -lboost_system -lboost_regex -ldouble-conversion -lsnappy"

# Prevent CRAN builder from linking against old libs in /usr/local/lib
for FILE in $BREWDIR/Cellar/*/*/lib/*.a; do
Expand Down