Problem with subassignment operator in version 3.1.0

[agila5]

Dear package developers, I want to report a problematic interaction between tibble and tmap that was reported by @Robinlovelace a few hours ago in the tmap repository. I think that the problem is not really related to tmap, but it can be summarised as follows:

# CRAN version (i.e. 3.1.0)
remove.packages("tibble")
#> Removing package from 'C:/Users/Utente/Documents/R/win-library/3.6'
#> (as 'lib' is unspecified)
remotes::install_version("tibble", "3.1.0", quiet = TRUE)

# test
library(tibble)
x <- tibble(a = "1")
x[FALSE, ] <- NA
#> Error: Assigned data `NA` must be compatible with existing data.
#> i Error occurred for column `a`.
#> x Can't convert <logical> to <character>.
x[FALSE, ] <- NA_character_
x
#> # A tibble: 1 x 1
#>   a    
#>   <chr>
#> 1 1

^{Created on 2021-03-02 by the reprex package (v0.3.0)}

Session info

devtools::session_info()
#> - Session info ---------------------------------------------------------------
#>  setting  value                       
#>  version  R version 3.6.3 (2020-02-29)
#>  os       Windows 10 x64              
#>  system   x86_64, mingw32             
#>  ui       RTerm                       
#>  language (EN)                        
#>  collate  Italian_Italy.1252          
#>  ctype    Italian_Italy.1252          
#>  tz       Europe/Berlin               
#>  date     2021-03-02                  
#> 
#> - Packages -------------------------------------------------------------------
#>  package     * version date       lib source        
#>  assertthat    0.2.1   2019-03-21 [1] CRAN (R 3.6.0)
#>  bit           4.0.4   2020-08-04 [1] CRAN (R 3.6.3)
#>  bit64         4.0.5   2020-08-30 [1] CRAN (R 3.6.3)
#>  blob          1.2.1   2020-01-20 [1] CRAN (R 3.6.2)
#>  callr         3.5.1   2020-10-13 [1] CRAN (R 3.6.3)
#>  cli           2.3.1   2021-02-23 [1] CRAN (R 3.6.3)
#>  cranlike      1.0.2   2018-11-26 [1] CRAN (R 3.6.3)
#>  crayon        1.4.1   2021-02-08 [1] CRAN (R 3.6.3)
#>  curl          4.3     2019-12-02 [1] CRAN (R 3.6.1)
#>  DBI           1.1.1   2021-01-15 [1] CRAN (R 3.6.3)
#>  debugme       1.1.0   2017-10-22 [1] CRAN (R 3.6.3)
#>  desc          1.2.0   2018-05-01 [1] CRAN (R 3.6.0)
#>  devtools      2.3.2   2020-09-18 [1] CRAN (R 3.6.3)
#>  digest        0.6.27  2020-10-24 [1] CRAN (R 3.6.3)
#>  ellipsis      0.3.1   2020-05-15 [1] CRAN (R 3.6.3)
#>  evaluate      0.14    2019-05-28 [1] CRAN (R 3.6.0)
#>  fansi         0.4.2   2021-01-15 [1] CRAN (R 3.6.3)
#>  fs            1.5.0   2020-07-31 [1] CRAN (R 3.6.3)
#>  glue          1.4.2   2020-08-27 [1] CRAN (R 3.6.3)
#>  highr         0.8     2019-03-20 [1] CRAN (R 3.6.0)
#>  htmltools     0.5.0   2020-06-16 [1] CRAN (R 3.6.3)
#>  knitr         1.30    2020-09-22 [1] CRAN (R 3.6.3)
#>  lifecycle     1.0.0   2021-02-15 [1] CRAN (R 3.6.3)
#>  magrittr      2.0.1   2020-11-17 [1] CRAN (R 3.6.3)
#>  memoise       1.1.0   2017-04-21 [1] CRAN (R 3.6.0)
#>  parsedate     1.2.0   2019-05-08 [1] CRAN (R 3.6.3)
#>  pillar        1.5.0   2021-02-22 [1] CRAN (R 3.6.3)
#>  pkgbuild      1.1.0   2020-07-13 [1] CRAN (R 3.6.3)
#>  pkgconfig     2.0.3   2019-09-22 [1] CRAN (R 3.6.1)
#>  pkgload       1.1.0   2020-05-29 [1] CRAN (R 3.6.3)
#>  prettyunits   1.1.1   2020-01-24 [1] CRAN (R 3.6.2)
#>  processx      3.4.4   2020-09-03 [1] CRAN (R 3.6.3)
#>  ps            1.4.0   2020-10-07 [1] CRAN (R 3.6.3)
#>  purrr         0.3.4   2020-04-17 [1] CRAN (R 3.6.3)
#>  R6            2.5.0   2020-10-28 [1] CRAN (R 3.6.3)
#>  rappdirs      0.3.1   2016-03-28 [1] CRAN (R 3.6.0)
#>  Rcpp          1.0.6   2021-01-15 [1] CRAN (R 3.6.3)
#>  remotes       2.2.0   2020-07-21 [1] CRAN (R 3.6.3)
#>  rlang         0.4.10  2020-12-30 [1] CRAN (R 3.6.3)
#>  rmarkdown     2.7     2021-02-19 [1] CRAN (R 3.6.3)
#>  rprojroot     2.0.2   2020-11-15 [1] CRAN (R 3.6.3)
#>  RSQLite       2.2.1   2020-09-30 [1] CRAN (R 3.6.3)
#>  rstudioapi    0.13    2020-11-12 [1] CRAN (R 3.6.3)
#>  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 3.6.0)
#>  stringi       1.5.3   2020-09-09 [1] CRAN (R 3.6.3)
#>  stringr       1.4.0   2019-02-10 [1] CRAN (R 3.6.0)
#>  testthat      3.0.0   2020-10-31 [1] CRAN (R 3.6.3)
#>  tibble      * 3.1.0   2021-02-25 [1] CRAN (R 3.6.3)
#>  usethis       2.0.0   2020-12-10 [1] CRAN (R 3.6.3)
#>  utf8          1.1.4   2018-05-24 [1] CRAN (R 3.6.0)
#>  vctrs         0.3.6   2020-12-17 [1] CRAN (R 3.6.3)
#>  withr         2.3.0   2020-09-22 [1] CRAN (R 3.6.3)
#>  xfun          0.19    2020-10-30 [1] CRAN (R 3.6.3)
#>  yaml          2.2.1   2020-02-01 [1] CRAN (R 3.6.2)
#> 
#> [1] C:/Users/Utente/Documents/R/win-library/3.6
#> [2] C:/Program Files/R/R-3.6.3/library

The same code should work with version 3.0.6:

# version 3.0.6
remove.packages("tibble")
#> Removing package from 'C:/Users/Utente/Documents/R/win-library/3.6'
#> (as 'lib' is unspecified)
remotes::install_version("tibble", "3.0.6", quiet = TRUE)

# test
library(tibble)
x <- tibble(a = "1")
x[FALSE, ] <- NA
x
#> # A tibble: 1 x 1
#>   a    
#>   <chr>
#> 1 1

^{Created on 2021-03-02 by the reprex package (v0.3.0)}

Session info

devtools::session_info()
#> - Session info ---------------------------------------------------------------
#>  setting  value                       
#>  version  R version 3.6.3 (2020-02-29)
#>  os       Windows 10 x64              
#>  system   x86_64, mingw32             
#>  ui       RTerm                       
#>  language (EN)                        
#>  collate  Italian_Italy.1252          
#>  ctype    Italian_Italy.1252          
#>  tz       Europe/Berlin               
#>  date     2021-03-02                  
#> 
#> - Packages -------------------------------------------------------------------
#>  package     * version date       lib source        
#>  assertthat    0.2.1   2019-03-21 [1] CRAN (R 3.6.0)
#>  bit           4.0.4   2020-08-04 [1] CRAN (R 3.6.3)
#>  bit64         4.0.5   2020-08-30 [1] CRAN (R 3.6.3)
#>  blob          1.2.1   2020-01-20 [1] CRAN (R 3.6.2)
#>  callr         3.5.1   2020-10-13 [1] CRAN (R 3.6.3)
#>  cli           2.3.1   2021-02-23 [1] CRAN (R 3.6.3)
#>  cranlike      1.0.2   2018-11-26 [1] CRAN (R 3.6.3)
#>  crayon        1.4.1   2021-02-08 [1] CRAN (R 3.6.3)
#>  curl          4.3     2019-12-02 [1] CRAN (R 3.6.1)
#>  DBI           1.1.1   2021-01-15 [1] CRAN (R 3.6.3)
#>  debugme       1.1.0   2017-10-22 [1] CRAN (R 3.6.3)
#>  desc          1.2.0   2018-05-01 [1] CRAN (R 3.6.0)
#>  devtools      2.3.2   2020-09-18 [1] CRAN (R 3.6.3)
#>  digest        0.6.27  2020-10-24 [1] CRAN (R 3.6.3)
#>  ellipsis      0.3.1   2020-05-15 [1] CRAN (R 3.6.3)
#>  evaluate      0.14    2019-05-28 [1] CRAN (R 3.6.0)
#>  fansi         0.4.2   2021-01-15 [1] CRAN (R 3.6.3)
#>  fs            1.5.0   2020-07-31 [1] CRAN (R 3.6.3)
#>  glue          1.4.2   2020-08-27 [1] CRAN (R 3.6.3)
#>  highr         0.8     2019-03-20 [1] CRAN (R 3.6.0)
#>  htmltools     0.5.0   2020-06-16 [1] CRAN (R 3.6.3)
#>  knitr         1.30    2020-09-22 [1] CRAN (R 3.6.3)
#>  lifecycle     1.0.0   2021-02-15 [1] CRAN (R 3.6.3)
#>  magrittr      2.0.1   2020-11-17 [1] CRAN (R 3.6.3)
#>  memoise       1.1.0   2017-04-21 [1] CRAN (R 3.6.0)
#>  parsedate     1.2.0   2019-05-08 [1] CRAN (R 3.6.3)
#>  pillar        1.5.0   2021-02-22 [1] CRAN (R 3.6.3)
#>  pkgbuild      1.1.0   2020-07-13 [1] CRAN (R 3.6.3)
#>  pkgconfig     2.0.3   2019-09-22 [1] CRAN (R 3.6.1)
#>  pkgload       1.1.0   2020-05-29 [1] CRAN (R 3.6.3)
#>  prettyunits   1.1.1   2020-01-24 [1] CRAN (R 3.6.2)
#>  processx      3.4.4   2020-09-03 [1] CRAN (R 3.6.3)
#>  ps            1.4.0   2020-10-07 [1] CRAN (R 3.6.3)
#>  purrr         0.3.4   2020-04-17 [1] CRAN (R 3.6.3)
#>  R6            2.5.0   2020-10-28 [1] CRAN (R 3.6.3)
#>  rappdirs      0.3.1   2016-03-28 [1] CRAN (R 3.6.0)
#>  Rcpp          1.0.6   2021-01-15 [1] CRAN (R 3.6.3)
#>  remotes       2.2.0   2020-07-21 [1] CRAN (R 3.6.3)
#>  rlang         0.4.10  2020-12-30 [1] CRAN (R 3.6.3)
#>  rmarkdown     2.7     2021-02-19 [1] CRAN (R 3.6.3)
#>  rprojroot     2.0.2   2020-11-15 [1] CRAN (R 3.6.3)
#>  RSQLite       2.2.1   2020-09-30 [1] CRAN (R 3.6.3)
#>  rstudioapi    0.13    2020-11-12 [1] CRAN (R 3.6.3)
#>  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 3.6.0)
#>  stringi       1.5.3   2020-09-09 [1] CRAN (R 3.6.3)
#>  stringr       1.4.0   2019-02-10 [1] CRAN (R 3.6.0)
#>  testthat      3.0.0   2020-10-31 [1] CRAN (R 3.6.3)
#>  tibble      * 3.0.6   2021-01-29 [1] CRAN (R 3.6.3)
#>  usethis       2.0.0   2020-12-10 [1] CRAN (R 3.6.3)
#>  utf8          1.1.4   2018-05-24 [1] CRAN (R 3.6.0)
#>  vctrs         0.3.6   2020-12-17 [1] CRAN (R 3.6.3)
#>  withr         2.3.0   2020-09-22 [1] CRAN (R 3.6.3)
#>  xfun          0.19    2020-10-30 [1] CRAN (R 3.6.3)
#>  yaml          2.2.1   2020-02-01 [1] CRAN (R 3.6.2)
#> 
#> [1] C:/Users/Utente/Documents/R/win-library/3.6
#> [2] C:/Program Files/R/R-3.6.3/library

More details about the original issue: https://github.com/mtennekes/tmap/issues/551

EDIT:

The same code works with regular data.frame:

x <- data.frame(a = "1")
x[FALSE, ] <- NA
x
#>   a
#> 1 1

^{Created on 2021-03-02 by the reprex package (v0.3.0)}

[Robinlovelace]

I have just responded to an email from CRAN saying my pct package will be removed on 18th March (in 3 days) unless fallout from this issue was resolved. I replied as follows:

I have checked the tests: https://cran.r-project.org/web/checks/check_results_pct.html
This is the result of a known issue in the latest release of tibble and its interaction the tmap:

• tibble issue: Problem with subassignment operator in version 3.1.0 #868
• tmap issue: https://github.com/mtennekes/tmap/issues/551

Is it OK to wait for these issues to be fixed upstream?
All the best,
Robin

In case anyone else is seeing CRAN checks fail because of this issue, it may be helpful to get an update on this, e.g. 'wip' or 'wontfix'. Heads up @krlmlr and other tibble developers. Thanks.

[krlmlr]

Thanks. Is this a duplicate of #773?

Please work around for now.

[agila5]

Thanks. Is this a duplicate of #773?

I think it is more or less the same problem. Moreover, I don't know exactly the rules behind subset-assignment for tibble(s), but I think that the behaviour is not always consistent. For example:

# packages
library(tibble)

# data
x <- tibble(
  a = TRUE, 
  b = 1L, 
  c = sqrt(2), 
  d = 3i + 1, 
  e = "e", 
  f = raw(1), 
)
x
#> # A tibble: 1 x 6
#>   a         b     c d     e     f    
#>   <lgl> <int> <dbl> <cpl> <chr> <raw>
#> 1 TRUE      1  1.41 1+3i  e     00

# subset-assign 0 row
x[FALSE, "a"] <- NA
x[FALSE, "b"] <- NA
x[FALSE, "c"] <- NA
x[FALSE, "d"] <- NA
x[FALSE, "e"] <- NA
#> Error: Assigned data `NA` must be compatible with existing data.
#> i Error occurred for column `e`.
#> x Can't convert <logical> to <character>.
x[FALSE, "f"] <- NA
#> Error: Assigned data `NA` must be compatible with existing data.
#> i Error occurred for column `f`.
#> x Can't convert <logical> to <raw>.
x
#> # A tibble: 1 x 6
#>   a         b     c d     e     f    
#>   <lgl> <int> <dbl> <cpl> <chr> <raw>
#> 1 TRUE      1  1.41 1+3i  e     00

while, when I consider subset-assignment with 1 row,

x[1, "a"] <- NA
x[1, "b"] <- NA
x[1, "c"] <- NA
x[1, "d"] <- NA
x[1, "e"] <- NA
x[1, "f"] <- NA
x 
#> # A tibble: 1 x 6
#>   a         b     c d     e     f    
#>   <lgl> <int> <dbl> <cpl> <chr> <raw>
#> 1 NA       NA    NA NA    <NA>  00

^{Created on 2021-03-18 by the reprex package (v1.0.0)}

[krlmlr]

It's a different issue, #773 is about assigning a value to an all-NA vector.

[krlmlr]

I have a PR ready that fixes this and the sister issue #773. Before actually merging, I'd like to rethink if we really want this, or if we'd rather prefer consistent failure in all cases.

Requiring typed NA adds a bit of safety for programming. I imagine this will be used mostly when populating/tweaking individual cells in an existing data frame. If we fail consistently, this means that the onus is on the programmer to double-check and ensure that the NA value they stuffed into this column is actually intended, and not just a programming error.

What do you think?

CC @lionel-.

[krlmlr]

On the other hand, in #773 (comment), @hadley suggests:

In general, this is a deliberate change, so that you don't accidentally change the type of your columns. However, this specific case seems to be a bug, since generally we treat vectors consisting only of missing values as compatible with any other vector.

Following that, this means that we might not need this extra safety, but also raises the following questions that should be resolved in vctrs:

• Are NA_integer_ and "x" compatible?
• Is tibble(a = integer(), b = logical()) compatible to tibble(a = 3, b = "x") ?
• Is tibble(a = 1, b = NA) compatible to tibble(a = 3, b = "x") ?

The answer might be "No", "Yes" and "Yes", with severe implications, but again it should be decided elsewhere.

[github-actions]

This old thread has been automatically locked. If you think you have found something related to this, please open a new issue and link to this old issue if necessary.