Submission: MODIStsp

[lbusett]

Summary

• What does this package do? (explain in 50 words or less):

Facilitates the creation of time series of raster images derived from MODIS Satellite Land Products by automating and standardizing several preprocessing steps (e.g., download, mosaicing, reprojection, resize, indexes computation, data extraction). Processing parameters are set using a powerful GUI. Non-interactive execution using saved Options Files allows updating time series when new images are available using cron jobs.

• Paste the full DESCRIPTION file inside a code block below:

Package: MODIStsp
Title: A Tool for Automating Download and Preprocessing of MODIS Land Products
    Data
Type: Package
Version: 1.3.3.9000
Authors@R: c(person("Lorenzo", "Busetto", email = "[email protected]", role = c("aut", "cre"),
    comment = c(ORCID = '0000-0001-9634-6038')), 
    person("Luigi", "Ranghetti", email = "[email protected]", role = c("aut")))
Description: Allows automating the creation of time series of rasters derived
    from MODIS Satellite Land Products data. It performs several typical
    preprocessing steps such as download, mosaicking, reprojection and resize
    of data acquired on a specified time period. All processing parameters
    can be set using a user-friendly GUI. Users can select which layers of
    the original MODIS HDF files they want to process, which additional
    Quality Indicators should be extracted from aggregated MODIS Quality
    Assurance layers and, in the case of Surface Reflectance products
    , which Spectral Indexes should be computed from the original reflectance
    bands. For each output layer, outputs are saved as single-band raster
    files corresponding to each available acquisition date. Virtual files
    allowing access to the entire time series as a single file are also created.
    Command-line execution exploiting a previously saved processing options
    file is also possible, allowing to automatically update time series
    related to a MODIS product whenever a new image is available.
License: GPL-3
Depends:
    R (>= 3.1.3)
Imports:
    bitops (>= 1.0-6),
    data.table (>= 1.9.6),
    gdalUtils (>= 2.0.1.7),
    gWidgets (>= 0.0-54),
    gWidgetsRGtk2,
    methods,
    httr (>= 1.1.0),
    jsonlite,
    pacman,
    parallel,
    raster (>= 2.5-2),
    RCurl (>= 1.95-4.8),
    rgdal (>= 1.0-3),
    rgeos (>= 0.3-8),
    sp (>= 1.2-2),
    stringr (>= 1.0.0),
    xts (>= 0.9-7),
    xml2
Suggests:
    spelling,
    knitr,
    rmarkdown,
    png,
    grid,
    httptest,
    covr
SystemRequirements: Cairo >= 1.0.0, ATK (>= 1.10.0), Pango (>= 1.10.0), GTK+ (>=
    2.8.0), GLib (>= 2.8.0), Curl, GDAL (>= 1.6.3), PROJ.4 (>= 4.4.9)
URL: https://github.com/lbusett/MODIStsp
BugReports: https://github.com/lbusett/MODIStsp/issues
LazyData: true
VignetteBuilder: knitr
RoxygenNote: 6.0.1
Roxygen: list(markdown = TRUE)

• URL for the package (the development repository, not a stylized html page):

  https://github.com/lbusett/MODIStsp

• Please indicate which category or categories from our package fit policies this package falls under *and why(? (e.g., data retrieval, reproducibility. If you are unsure, we suggest you make a pre-submission inquiry.):

  • data retrieval, because the package facilitates access to MODIS satellite data;
  • geospatial data, because MODIS time series are widely used by the scientific community for several geospatial applications

•   Who is the target audience and what are scientific applications of this package?  

  The target audience includes anyone involved/interested in monitoring the properties of the Earth surface through the analysis of time series of coarse resolution satellite imagery.
  MODIS satellite data are in fact used in many different scientific applications, since they allow global daily monitoring of the earth surface. Their applications are therefore too many to analyze in this context. Among the main ones, I can cite (borrowing a bit from wikipedia...) the monitoring of vegetation health by means of time-series analyses with vegetation indices, the analysis of long term land cover changes (e.g. to monitor deforestation rates), real time monitoring of snow/ice cover, analysis of water quality and of impacts of floodings, change of water levels of major lakes and the detection and mapping of wildland fires. MODIS data are also extensively used as ancillary input in modelling applications (e.g., biogeochemical models for Primary Production modelling; crop yield models). Additionally, MODIS data are often used by biologists/zoologists as ancillary data to monitor/model migratory patterns.

  (Introductory material concerning MODIS and its main applications can be found for example here and here)

• Are there other R packages that accomplish the same thing? If so, how does
  yours differ or meet our criteria for best-in-category?

  The main “R” packages overlapping MODIStsp functionality are MODIS and MODISTools
  MODIStsp and MODIS have substantial overlaps, since they are both meant to provide access and preprocessing to MODIS products. Two aspects that in my opinion differentiate MODIStsp are its user-friendliness (I think that the MODIStsp Graphical User Interface is very helpful, in particular for less skilled "R" users, though I am clearly a bit biased on the topic), and its ability to allow on-the-fly creation of time series of standard/user provided Spectral Indexes (often used for time series analysis) and of MODIS quality indicators. On the other hand, MODIS provides some additional post-processing functionality (e.g., filtering/smoothing), which is something I'm planning to add in the near future.

  MODIStools is a very nice package, but it provides somewhat more limited functionality. It is based on the ORNL DAAC webservice (https://modis.ornl.gov/data/modis_webservice.html), and therefore provides access only to some MODIS products. Additionally, it is meant to allow access to data concerning a given set of points and their surroundings, while both MODIStsp and MODIS are meant to provide downloading capabilitites to the "full" MODIS raster data (or subsets of those, although the full HDFs need always to be downloaded).

  The ModisDownload function of package rts is also overlapping, but its functionalities are more limited than either MODIStsp and MODIS. (Actually, an early version of ModisDownload is what got us started in developing the package - proper acknwoledgement is given in MODIStsp documentation).

  Other packages providing access to MODIS data include MODISSnow (limited to snow cover products distributed by NSIDC) and modiscloud (which provides tools for processing Level 2 Cloud Mask SWATH products – not supported by MODIStsp)
  For further reference, our paper concerning MODIStsp published on CAGEO provides in Section 1 a short comparison of the different tools currently available for accessing MODIS data (both "R" based or not).

•   If you made a pre-submission enquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted.

  pre-submission inquiry for MODIStsp #143

Requirements

Confirm each of the following by checking the box. This package:

• does not violate the Terms of Service of any service it interacts with.

• has a CRAN and OSI accepted license.

• contains a README with instructions for installing the development version.

  NOTE ON INSTALLATION Installation (both from github or from CRAN) may be a bit problematic due the fact that MODIStsp requires the gWidgetsRGtk2 library to build its GUI, which often have installation problems on Windows and Mac due to requiring the GTK+ library, and needing several packages to be installed beforehand on Linux. In case of problems, it is better to install gWidgetsRGtk2 beforehand. Specific instructions are provided at (http://lbusett.github.io/MODIStsp/articles/installation.html)

• includes documentation with examples for all functions.

• contains a vignette with examples of its essential functions and uses.

• has a test suite.

• has continuous integration, including reporting of test coverage, using services such as Travis CI, Coveralls and/or CodeCov.

• I agree to abide by ROpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.

Publication options

• Do you intend for this package to go on CRAN?
  The package is already on CRAN (https://cran.r-project.org/package=MODIStsp)

• Do you wish to automatically submit to the Journal of Open Source Software? If so:

  • The package has an obvious research application according to JOSS's definition.
  • The package contains a paper.md matching JOSS's requirements with a high-level description in the package root or in inst/.
  • The package is deposited in a long-term repository with the DOI:
  • (Do not submit your package separately to JOSS)

• Do you wish to submit an Applications Article about your package to Methods in Ecology and Evolution? If so:

  • The package is novel and will be of interest to the broad readership of the journal.
  • The manuscript describing the package is no longer than 3000 words.
  • You intend to archive the code for the package in a long-term repository which meets the requirements of the journal.
  • (Please do not submit your package separately to Methods in Ecology and Evolution)

Detail

• Does R CMD check (or devtools::check()) succeed? Paste and describe any errors or warnings:

• Does the package conform to rOpenSci packaging guidelines? Please describe any exceptions:

• If this is a resubmission following rejection, please explain the change in circumstances:

• If possible, please provide recommendations of reviewers - those with experience with similar packages and/or likely users of your package - and their GitHub user names:

Michael Sumner - @mdsumner
Jeff Hanson - @jeffreyhanson
Leah Wasser - @lwasser
Robin Lovelace - @robin

[lwasser]

hey @lbusett one of my students used this package in his final project in the fall! he showed it to me and said it was really nice.

[karthik]

👋 @lbusett I will be handling your submission. Thanks also for the reviewer suggestions.
@lwasser Would you be interested in being one of the reviewers?

[karthik]

I have also reached out to @jeffreyhanson by email.

[lbusett]

@karthik Great, thanks!
@lwasser Glad to hear about that!

[lwasser]

sure @karthik i'll review this. i wanted to spend some time playing with it after my student showed it to me. What is the deadline? i'll need a few weeks given i just got back from travel today.

[jeffreyhanson]

Sure, I'm happy to review this. I'm pretty busy at the moment though, so it might be a few weeks before I can get around to it if that's ok?

[karthik]

Editor checks:

• Fit: The package meets criteria for fit and overlap
• Automated tests: Package has a testing suite and is tested via Travis-CI or another CI service.
• License: The package has a CRAN or OSI accepted license
• Repository: The repository link resolves correctly
• Archive (JOSS only, may be post-review): The repository DOI resolves correctly
• Version (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?

---

Editor comments

Work in progress (dealing with installation issues)

---

Reviewers: @lwasser and @jeffreyhanson
Due date: Feb 10, 2018

[karthik]

@lwasser @jeffreyhanson Thanks very much! I've tentatively assigned Feb 10th. Please let me know if anything changes. 🙏

[lbusett]

@karthik Sorry to hear that you having problems in installation.... 😟 Unfortunately, as I reported in the submission, that is a common problem related to the requirements of the gWidgetsRGtk2 package. Instructions reported here: http://lbusett.github.io/MODIStsp/articles/installation.html are usually sufficient to solve the issue (unless you are working on a mac with os sierra, in which case it can be more dificult...).

Let me know if I can help in any way!

[lbusett]

Hi @karthik . Just wanted to ask if in the end you managed to solve the installation issues or you are still having problems. In case you are working on a mac and you are still struggling, these are the "latest" instructions I found regarding how to install packages "RGtk2" (and consequently gWidgetsRGtk2) on mac os Sierra:

https://stackoverflow.com/a/46748516/6871135

[jeffreyhanson]

Awesome work @lbusett! This package looks absolutely amazing. I've just started working on my review, but I thought I would let you know that some of the unit tests fail on my system in case you wanted to address this before I submit my review. I've included the output below and a copy of my session info.

> test()
Loading MODIStsp
Testing MODIStsp
✔ | OK F W S | Context
✔ |  4       | Check proper functioning of bbox_from_file [1.6 s]
✔ |  4       | get_yeardates works as expected when download_range == "Full"
✔ |  6       | get_yeardates works as expected when download_range == "Seasonal"
✔ |  7       | Check proper functioning of MODIStsp_addindex [0.2 s]
✔ | 21       | MODIStsp_extract works as expected [58.7 s]
⠏ |  0       | Check proper functioning of the main MODIStsp GUI
(R:19949): Gtk-WARNING **: Failed to set text from markup due to error parsing markup: Error on line 1 char 83: Element 'markup' was closed, but the currently open element is 'span'

(R:19949): Gtk-WARNING **: Failed to set text from markup due to error parsing markup: Error on line 1 char 83: Element 'markup' was closed, but the currently open element is 'span'

(R:19949): Gtk-WARNING **: Failed to set text from markup due to error parsing markup: Error on line 1 char 87: Element 'markup' was closed, but the currently open element is 'span'

(R:19949): Gtk-WARNING **: Failed to set text from markup due to error parsing markup: Error on line 1 char 87: Element 'markup' was closed, but the currently open element is 'span'

(R:19949): Gtk-WARNING **: Failed to set text from markup due to error parsing markup: Error on line 1 char 83: Element 'markup' was closed, but the currently open element is 'span'

(R:19949): Gtk-WARNING **: Failed to set text from markup due to error parsing markup: Error on line 1 char 83: Element 'markup' was closed, but the currently open element is 'span'

(R:19949): Gtk-WARNING **: Failed to set text from markup due to error parsing markup: Error on line 1 char 83: Element 'markup' was closed, but the currently open element is 'span'

(R:19949): Gtk-WARNING **: Failed to set text from markup due to error parsing markup: Error on line 1 char 83: Element 'markup' was closed, but the currently open element is 'span'
✔ |  0       | Check proper functioning of the main MODIStsp GUI [4.1 s]
✔ |  6   2   | MODIStsp Test 0: Gracefully fail on input problems [14.9 s]
────────────────────────────────────────────────────────────────────────────────
test_MODIStsp.R:13: warning: Tests on MODIStsp
input string 1 is invalid in this locale

test_MODIStsp.R:18: warning: Tests on MODIStsp
input string 1 is invalid in this locale
────────────────────────────────────────────────────────────────────────────────
⠏ |  0       | MODIStsp Test 1: Basic processing on bands and quality
⠋ |  1       | MODIStsp Test 1: Basic processing on bands and quality
⠙ |  2       | MODIStsp Test 1: Basic processing on bands and quality
✔ |  2       | MODIStsp Test 1: Basic processing on bands and quality
        indicators [5.3 s]
✔ |  4       | MODIStsp Test 2: Geometric operations [3.2 s]
⠏ |  0       | MODIStsp Test 3: Computation of spectral indices and
            creation of time seriesWarning 1: Metadata exceeding 32000 bytes cannot be written into GeoTIFF. Transferred to PAM instead.
⠋ |  1       | MODIStsp Test 3: Computation of spectral indices and
⠙ |  2       | MODIStsp Test 3: Computation of spectral indices and
⠹ |  3       | MODIStsp Test 3: Computation of spectral indices and
⠸ |  4       | MODIStsp Test 3: Computation of spectral indices and
⠼ |  5       | MODIStsp Test 3: Computation of spectral indices and
            creation of time seriesWarning 1: Metadata exceeding 32000 bytes cannot be written into GeoTIFF. Transferred to PAM instead.
⠴ |  6       | MODIStsp Test 3: Computation of spectral indices and
⠦ |  7       | MODIStsp Test 3: Computation of spectral indices and
✔ |  7       | MODIStsp Test 3: Computation of spectral indices and
            creation of time series [22.0 s]
✖ |  0 1     | MODIStsp Test 4: HTTP download from NSIDC (seasonal) [5.8 s]
────────────────────────────────────────────────────────────────────────────────
test_MODIStsp.R:193: error: Tests on MODIStsp
argument is of length zero
1: MODIStsp(test = 4) at /home/jeff/GitHub/MODIStsp/tests/testthat/test_MODIStsp.R:193
2: MODIStsp_process(sel_prod = general_opts$sel_prod, start_date = general_opts$start_date, 
       end_date = general_opts$end_date, out_folder = general_opts$out_folder, out_folder_mod = general_opts$out_folder_mod, 
       reprocess = general_opts$reprocess, delete_hdf = general_opts$delete_hdf, sensor = general_opts$sensor, 
       download_server = general_opts$download_server, user = general_opts$user, password = general_opts$password, 
       https = prod_opts$http, ftps = prod_opts$ftp, start_x = general_opts$start_x, 
       start_y = general_opts$start_y, end_x = general_opts$end_x, end_y = general_opts$end_y, 
       full_ext = general_opts$full_ext, bbox = general_opts$bbox, out_format = general_opts$out_format, 
       out_res_sel = general_opts$out_res_sel, out_res = as.numeric(general_opts$out_res), 
       native_res = prod_opts$native_res, tiled = prod_opts$tiled, resampling = general_opts$resampling, 
       ts_format = general_opts$ts_format, compress = general_opts$compress, mod_proj_str = mod_proj_str, 
       outproj_str = general_opts$user_proj4, nodata_in = prod_opts$nodata_in, nodata_out = prod_opts$nodata_out, 
       rts = general_opts$rts, nodata_change = general_opts$nodata_change, scale_val = general_opts$scale_val, 
       scale_factor = prod_opts$scale_factor, offset = prod_opts$offset, datatype = prod_opts$datatype, 
       bandsel = general_opts$bandsel, bandnames = prod_opts$bandnames, indexes_bandsel = c(general_opts$indexes_bandsel), 
       indexes_bandnames = c(prod_opts$indexes_bandnames, custom_idx$indexes_bandnames), 
       indexes_formula = c(prod_opts$indexes_formula, custom_idx$indexes_formulas), 
       indexes_nodata_out = c(prod_opts$indexes_nodata_out, custom_idx$indexes_nodata_out), 
       quality_bandnames = prod_opts$quality_bandnames, quality_bandsel = general_opts$quality_bandsel, 
       quality_bitN = prod_opts$quality_bitN, quality_source = prod_opts$quality_source, 
       quality_nodata_in = prod_opts$quality_nodata_in, quality_nodata_out = prod_opts$quality_nodata_out, 
       file_prefixes = prod_opts$file_prefix, main_out_folder = prod_opts$main_out_folder, 
       gui = gui, use_aria = general_opts$use_aria, download_range = general_opts$download_range, 
       n_retries = n_retries, verbose = verbose) at /home/jeff/GitHub/MODIStsp/R/MODIStsp.R:464
3: MODIStsp_download(modislist, out_folder_mod, download_server, http, ftp, n_retries, 
       use_aria, date_dirs[date], year, DOY, user, password, sens_sel, date_name, gui, 
       mess_lab, verbose) at /home/jeff/GitHub/MODIStsp/R/MODIStsp_process.R:388
────────────────────────────────────────────────────────────────────────────────
✖ |  0 1     | MODIStsp Test 5: HTTP download from USGS, resize and reproject [14.5 s]
────────────────────────────────────────────────────────────────────────────────
test_MODIStsp.R:249: error: Tests on MODIStsp
argument is of length zero
1: MODIStsp(test = 5) at /home/jeff/GitHub/MODIStsp/tests/testthat/test_MODIStsp.R:249
2: MODIStsp_process(sel_prod = general_opts$sel_prod, start_date = general_opts$start_date, 
       end_date = general_opts$end_date, out_folder = general_opts$out_folder, out_folder_mod = general_opts$out_folder_mod, 
       reprocess = general_opts$reprocess, delete_hdf = general_opts$delete_hdf, sensor = general_opts$sensor, 
       download_server = general_opts$download_server, user = general_opts$user, password = general_opts$password, 
       https = prod_opts$http, ftps = prod_opts$ftp, start_x = general_opts$start_x, 
       start_y = general_opts$start_y, end_x = general_opts$end_x, end_y = general_opts$end_y, 
       full_ext = general_opts$full_ext, bbox = general_opts$bbox, out_format = general_opts$out_format, 
       out_res_sel = general_opts$out_res_sel, out_res = as.numeric(general_opts$out_res), 
       native_res = prod_opts$native_res, tiled = prod_opts$tiled, resampling = general_opts$resampling, 
       ts_format = general_opts$ts_format, compress = general_opts$compress, mod_proj_str = mod_proj_str, 
       outproj_str = general_opts$user_proj4, nodata_in = prod_opts$nodata_in, nodata_out = prod_opts$nodata_out, 
       rts = general_opts$rts, nodata_change = general_opts$nodata_change, scale_val = general_opts$scale_val, 
       scale_factor = prod_opts$scale_factor, offset = prod_opts$offset, datatype = prod_opts$datatype, 
       bandsel = general_opts$bandsel, bandnames = prod_opts$bandnames, indexes_bandsel = c(general_opts$indexes_bandsel), 
       indexes_bandnames = c(prod_opts$indexes_bandnames, custom_idx$indexes_bandnames), 
       indexes_formula = c(prod_opts$indexes_formula, custom_idx$indexes_formulas), 
       indexes_nodata_out = c(prod_opts$indexes_nodata_out, custom_idx$indexes_nodata_out), 
       quality_bandnames = prod_opts$quality_bandnames, quality_bandsel = general_opts$quality_bandsel, 
       quality_bitN = prod_opts$quality_bitN, quality_source = prod_opts$quality_source, 
       quality_nodata_in = prod_opts$quality_nodata_in, quality_nodata_out = prod_opts$quality_nodata_out, 
       file_prefixes = prod_opts$file_prefix, main_out_folder = prod_opts$main_out_folder, 
       gui = gui, use_aria = general_opts$use_aria, download_range = general_opts$download_range, 
       n_retries = n_retries, verbose = verbose) at /home/jeff/GitHub/MODIStsp/R/MODIStsp.R:464
3: MODIStsp_download(modislist, out_folder_mod, download_server, http, ftp, n_retries, 
       use_aria, date_dirs[date], year, DOY, user, password, sens_sel, date_name, gui, 
       mess_lab, verbose) at /home/jeff/GitHub/MODIStsp/R/MODIStsp_process.R:388
────────────────────────────────────────────────────────────────────────────────
  |======================================================================| 100%
  |======================================================================| 100%
  |======================================================================| 100%
  |======================================================================| 100%
✔ |  6       | MODIStsp Test 6: FTP download and mosaicing of MODIS tiles [77.0 s]
✔ |  1       | MODIStsp Test 7: Passing the extent with a spatial file [3.0 s]
⠏ |  0       | MODIStsp Test 8: Fail gracefully on missing connectionError : GET http://e4ftl01.cr.usgs.gov/MOTA/MCD43A3.006/
In addition: Warning message:
`encoding` is deprecated; all files now assumed to be UTF-8 
Error : GET ftp://ladsweb.nascom.nasa.gov/allData/6/MCD43A3/2017/
⠋ |  1       | MODIStsp Test 8: Fail gracefully on missing connectionError : GET ftp://ladsweb.nascom.nasa.gov/allData/6/MCD15A2H/2016/
✔ |  2       | MODIStsp Test 8: Fail gracefully on missing connection [0.5 s]
✖ |  0 1     | Check proper functioning of MODIStsp_readxml [1.9 s]
────────────────────────────────────────────────────────────────────────────────
test_MODIStsp_readxml.R:8: error: MODIStsp_readxml works as expected
attempt to select less than one element in OneIndex
1: MODIStsp_read_xml(tmpfile, system.file("ExtData", "MODIStsp_ProdOpts.xml", package = "MODIStsp")) at /home/jeff/GitHub/MODIStsp/tests/testthat/test_MODIStsp_readxml.R:8
────────────────────────────────────────────────────────────────────────────────
✔ |  2       | MODIStsp_resetindexes

══ Results ═════════════════════════════════════════════════════════════════════
Duration: 213.2 s

OK:       72
Failed:   3
Warnings: 2
Skipped:  0

Session information

R version 3.4.3 (2017-11-30)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.3 LTS

Matrix products: default
BLAS: /usr/lib/libblas/libblas.so.3.6.0
LAPACK: /usr/lib/lapack/liblapack.so.3.6.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] httptest_3.0.0       gWidgetsRGtk2_0.0-85 cairoDevice_2.24    
[4] gWidgets_0.0-54      RGtk2_2.20.34        MODIStsp_1.3.3.9000 
[7] testthat_2.0.0       devtools_1.13.4     

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.15        compiler_3.4.3      R.methodsS3_1.7.1  
 [4] bitops_1.0-6        R.utils_2.6.0       iterators_1.0.9    
 [7] tools_3.4.3         xts_0.10-1          gdalUtils_2.0.1.7  
[10] digest_0.6.15       jsonlite_1.5        memoise_1.1.0      
[13] lattice_0.20-35     rlang_0.1.6         foreach_1.4.4      
[16] cli_1.0.0           commonmark_1.4      curl_3.1           
[19] rgdal_1.2-16        parallel_3.4.3      withr_2.1.1        
[22] stringr_1.2.0       httr_1.3.1          roxygen2_6.0.1     
[25] raster_2.6-7        xml2_1.2.0          rgeos_0.3-26       
[28] grid_3.4.3          data.table_1.10.4-3 R6_2.2.2           
[31] pacman_0.4.6        sp_1.2-7            magrittr_1.5       
[34] codetools_0.2-15    assertthat_0.2.0    mime_0.5           
[37] stringi_1.1.6       RCurl_1.95-4.10     crayon_1.3.4       
[40] zoo_1.8-1           R.oo_1.21.0  

Let me know if there's any other info I can provide.

[lbusett]

@jeffreyhanson

Thanks! Glad you like our package!

Concerning the tests, thanks for reporting the issues. They should now be fixed. One was due to a recent change in xml2 functionality, the other (apparently) to a change in the "standard" http response of NASA servers. If you pull the new github version, the tests should pass (and you should also be able to download data from http also outside of the tests, which I think ws not possible due to the NASA response change. I'll have to also test the CRAN version to see if it still works over http.

[lwasser]

hey @lbusett or @jeffreyhanson i'm a bit confused on the quartz install

i'm here:

next, edit the configure options for GTK to require x11 rather than Quartz:

brew edit gtk+

in the def install section, remove the reference to quartz and switch to:

--with-gdktarget=x11,
--enable-x11-backend

And would like to understand what part of the gtk+ file i need to edit. do i just replace "with-gdktarget=quartz", with two lines as follows:

"--with-gdktarget=x11",
"--enable-x11-backend",

it's taking me some time to get everything installed and this step is a bit confusing. thank you for any advice!

So i'm not sure if i did this right but another question. Does this message make sense:

library(RGtk2)
R session is headless; GTK+ not initialized.

[lbusett]

hi @lwasser ,

sorry you are having trouble... unfortunately, at the moent RGtk2 installation on mac is troublesome (and the solution to the trouble seems to change with time.. ;-(.
Moreover, I know almost nothing about mac, so I admit that I never really tested the instructions provided in the help.... However, those instructions mean that indeed you need to modify the gtk file so that it reads:

def install
args = ["--disable-dependency-tracking",
"--disable-silent-rules",
"--prefix=#{prefix}",
"--disable-glibtest",
"--enable-introspection=yes",
"--with-gdktarget=x11",
"--enable-x11-backend"
]

and then proceed with brew install.

Concerning the "R session is headless; GTK+ not initialized." : this does not bode well. It (should) mean that RGtk2 does not find a "graphical environment" to use. I fear that if you try to load gWidgetsRGtk2 you will have problems.

Again, sorry for the trouble. I hope you'll be able to solve it easily (this link may also help : https://gist.github.com/sebkopf/9405675). Otherwise, I'll try to get my hands on a (modern) mac pc and see if I manage to provide better instructions.

[lwasser]

ok so remove "--disable-visibility" as well?
Yes i'm having many issues now. i can't seem to get cairoDevice or gWidgetsRGtk2 to install correctly. i'm not sure what to try next to get them to install.

> install.packages("cairoDevice", lib="~/Library/Frameworks/R.framework/Versions/3.4/Resources/library/cairoDevice")
Warning in install.packages :
  'lib = "~/Library/Frameworks/R.framework/Versions/3.4/Resources/library/cairoDevice"' is not writable
Would you like to use a personal library instead?  (y/n) y
Package which is only available in source form, and may need compilation of
  C/C++/Fortran: ‘cairoDevice’
Do you want to attempt to install these from sources?
y/n: y
installing the source package ‘cairoDevice’

trying URL 'https://cran.rstudio.com/src/contrib/cairoDevice_2.24.tar.gz'
Content type 'application/x-gzip' length 39639 bytes (38 KB)
==================================================
downloaded 38 KB

* installing *source* package ‘cairoDevice’ ...
** package ‘cairoDevice’ successfully unpacked and MD5 sums checked
checking for pkg-config... pkg-config
ERROR: gtk+2.0 not found by pkg-config.
ERROR: configuration failed for package ‘cairoDevice’
* removing ‘/Users/lewa8222/Library/R/3.4/library/cairoDevice’
Warning in install.packages :
  installation of package ‘cairoDevice’ had non-zero exit status

The downloaded source packages are in
	‘/private/var/folders/43/4q82487d5xsfpxdx6nl_c1wmhckx08/T/RtmpJoK8cV/downloaded_packages’

does this help at all with troubleshooting? if i say "n" to install from source it just quits. i'm wondering if i get this error because i edited the install code above incorrectly (i left disable-visibility in the code).

[lbusett]

Just a check:

did you already do

export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:/usr/local/lib/pkgconfig/gtk+-2.0.pc:/opt/X11/lib/pkgconfig

which I think is setting the path to gtk+ and should solve to cairo installation issue (ERROR: gtk+2.0 not found by pkg-config.)?

[lwasser]

ok!! i think i just got it. THANK YOU @lbusett i just installed those packages manually but also i hadn't run that line above that you suggested so i did that too. phew! i'll play with the package more tomorrow. thank you for the help.

[jeffreyhanson]

Sorry for the late reply @lbusett, I just installed the latest GitHub version and all the tests pass now. Thank for you fixing this so quickly.

[lbusett]

Good. In the end, both problems were due to a breaking change in xml2. Since the cran version uses xml, I was not notified about it.

By the way: a user signalled us a bug in the gui that I probably introduced while refactoring the code for ropensci submission. I am going to commit a fix for it ASAP.

[lbusett]

@karthik Ok, thanks for the update!

[karthik]

@jeffreyhanson Quick ping/reminder to sign off or ask more questions for @lbusett

🙏

[jeffreyhanson]

I'm sorry for the late reply, this completely fell off my radar. @lbusett has done an amazing job and addressed all my issues/suggestions.

[lbusett]

@jeffreyhanson @lwasser Thanks! Glad you liked the improvements! I think the review process allowed to really improve the package!

@karthik How should we proceed from here? Consider that unfortunately i will be forced to make some changes with respect to the reviewed package due to some very revcent changes in NASA policies (e.g., discontinued support for FTP download, and discontinuation of most of version 005 products). I expect to be able to release a new version by the beginning of next week.

[karthik]

@lbusett 👋 I'm glad to see @jeffreyhanson and @lwasser have signed off. I suggest making those changes and pushing a new release before we proceed with acceptance. I'm painfully aware of such policy changes.

[lbusett]

@karthik

I finally managed to put out a new version (1.3.4), including changes required to address changes in NASA policies. The new version is available on github, and I plan to upload it to CRAN soon.

I'd like therefore to ask:

1. What are the next steps for approval?
2. Can I proceed with CRAN release or do you prefer I wait for the end of the review/approval process?

Lorenzo

[lbusett]

@karthik, @jeffreyhanson, @lwasser

Hi all, please note that MODIStsp v 1.3.4 is now on cran. Please let me know what would be the next steps to be done for final approval on ropensci.

Thanks in advance!

Lorenzo

[lwasser]

woo hoo! great work @lbusett ... seems like it's ready for final acceptance into ropensci by @karthik ?

[lbusett]

Hi,

just a quick ping for @karthik, since I saw you are back to work: How can we further proceed on this?

Lorenzo

[lbusett]

Hi @karthik

Sorry for bringing this up again, but it seems this submission got stuck in some kind of limbo, recently, and i would like to close this task. ☺️

Could we have any updates regarding next steps?

[lwasser]

... i was just talking to someone about it and looking for this on ropensci and didn't see it and was wondering the same thing? @maelle @karthik just curious.

[sckott]

Sorry about the delay, i'll ping Karthik

[karthik]

@lbusett Apologies for the delay -- will follow up in a few.

[karthik]

@lbusett Sincere apologies for the delay. Your last notification came during my holiday and got lost in a giant pile of unread GitHub notifications 😢But now that I see all the Is have been dotted and T's crossed, we are ready to go.

Congrats @lbusett, your submission has been approved! 🎉 Thank you for submitting and @lwasser and @jeffreyhanson for thorough and timely reviews. To-dos:

• Transfer the repo to the rOpenSci organization under "Settings" in your repo. I have invited you to a team that should allow you to do so. You'll be made admin once you do.

• Add the rOpenSci footer to the bottom of your README

[![ropensci_footer](https://ropensci.org/public_images/ropensci_footer.png)](https://ropensci.org)

• Fix any links in badges for CI and coverage to point to the ropensci URL. (We'll turn on the services on our end as needed)

Welcome aboard! We'd also love a blog post about your package, either a short-form intro to it (https://ropensci.org/technotes/) or long-form post with more narrative about its development. ((https://ropensci.org/blog/). If you are, @stefaniebutland will be in touch about content and timing.

[karthik]

@lbusett You can also now add the status badge (I should have alerted you earlier) to your README. It should switch to accepted shortly:

[![](https://badges.ropensci.org/184_status.svg)](https://github.com/ropensci/onboarding/issues/184)

[lbusett]

@karthik

thanks! No worries about the delay: I know it happens. I will do those last steps ASAP.
Concerning the blog post, although I already have some blog posts regarding the package here: https://lbusett.netlify.com/categories/modis/, I think that making also a technote on rOpensci would be nice.

@lwasser @jeffreyhanson

Thanks again for your very usefeul reviews: I think they allowed me to really improve the package!
I would like to include you as reviewers in the package DECRIPTION: would that be ok with you ?

Lorenzo

[stefaniebutland]

Hello @lbusett. Glad to hear you're interested in contributing a technote. It might get even more eyes on your work.

Examples of technotes are here: https://ropensci.org/technotes/
Here are some technical and editorial guidelines. https://github.com/ropensci/roweb2#contributing-a-blog-post. Difference for technote is categories: technotes in YAML

We ask that you submit your draft post via pull request a week before the planned publication date so we can give you some feedback.

Happy to answer any questions.

[lbusett]

Hi @karthik ,

I just transferred the repo to https://github.com/ropensci/MODIStsp after making the necessary changes.

(Note that the build on Travis is currently failing apparently due to a downtime of the NSIDC server)

[lwasser]

@lbusett happy to be included as a reviewer in the description. thank you for your effort on this awesome package!! it's a great community contribution. i'm also happy to see it being embraced within the ropensci community!! yay!

[lbusett]

@lwasser Thanks! I will add you in the description ASAP!
@jeffreyhanson What about you? Isit ok also for you to be included as a reviewer ?

[lbusett]

@karthik

I am currently unable to modify the package settings (i.e., to modify the link to the website at the top of the repo page). Could you grant me admin settings on the repository ?

Lorenzo

[VasilisNtoufas]

Great package! I have just 1 question, while running the package I saw written that QA's were computed. In my PC several folders were created. In which folder the 'corrected' images of MODIS are? Or should I make something else (e.g. use another package) in order to take the good quality images?

[karthik]

@vanto1994 Please post this question in the project's repo (https://github.com/ropensci/MODIStsp/issues), not this review thread. Thanks!

[lbusett]

@vanto1994 I took the liberty of moving the issue in the proper issues page. I will reply you there.