Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Add derived subvar #453

Open
wants to merge 20 commits into
base: master
Choose a base branch
from
Open

[WIP] Add derived subvar #453

wants to merge 20 commits into from

Conversation

gergness
Copy link
Contributor

Definitely not ready for primetime yet, but wanted to get some notes down.

Weird things:

  • absolutifyVariables (which modifies the derivation expression before user sees it) is wrong in some cases, likely having to do with array variables. It seems to assume that the variable is given by relative URL, but within map zcl function, variables are given by ID only. I have gotten around this by not converting to absolute paths, but I'd like to actually fix.
  • For subvariables that are created when the array is first derived, they get their metadata automatically filled in, but we don't get that when adding new ones via derivation. Is it possible to get this? Or a limitation of the API?
  • Haven't gotten the MRs to work yet because they have an extra layer of nesting in zcl. Need a better way to do the derivation modification that fails more gracefully and is flexible for at least these 2 cases.
library(crunch)
set.seed(2020-05-18)

levels <- c("Good", "Okay", "Bad")
data <- data.frame(
    x1 = factor(sample(c(levels, NA), 10, TRUE), levels),
    x2 = factor(sample(c(levels, NA), 10, TRUE), levels),
    x3 = factor(sample(c(levels, NA), 10, TRUE), levels)
)

ds <- newDataset(data, name = "combine into array test")
ds$xcat <- deriveArray(ds[1:2], name = "x cat")
ds$xmr <- deriveArray(ds[1:2], name = "x mr", selections = "Good")


# Works in POC
ds$xcat <- addSubvariable(ds$xcat, ds["x3"])

# Does not yet work 
ds$xmr <- addSubvariable(ds$xmr, ds["x3"])

@lintr-bot
Copy link

R/add-subvariable.R:160:1: style: Variable and function names should not be longer than 30 characters.

addSubvarsToSelectCatDerivation <- function(deriv, new_vars, existing_cats) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

R/add-subvariable.R:207:1: style: Lines should not be more than 100 characters.

"cannot add subvariables.\n  ", paste0(var_aliases, "(", cats_for_vars, ")", collapse = ", ")
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

tests/testthat/test-add-subvariable.R:71:1: style: Lines should not be more than 100 characters.

​    test_that("Cannot add new categories when adding existing variable to derived select_cat array", {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

@codecov
Copy link

codecov bot commented May 26, 2020

Codecov Report

Merging #453 into master will decrease coverage by 0.44%.
The diff coverage is 63.02%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #453      +/-   ##
==========================================
- Coverage   90.55%   90.11%   -0.45%     
==========================================
  Files         121      121              
  Lines        6932     7048     +116     
==========================================
+ Hits         6277     6351      +74     
- Misses        655      697      +42     
Impacted Files Coverage Δ
R/delete.R 64.06% <5.12%> (-25.94%) ⬇️
R/add-subvariable.R 89.00% <90.90%> (+9.00%) ⬆️
R/variable-definition.R 100.00% <100.00%> (ø)
R/shoji-catalog.R 93.28% <0.00%> (+0.67%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5201389...5615950. Read the comment docs.

@gergness gergness requested a review from malecki May 26, 2020 15:47
@gergness gergness marked this pull request as ready for review May 26, 2020 15:47
Copy link
Contributor Author

@gergness gergness left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

expanding out some of the TODOs

derivation(variable) <- new_deriv
# We don't get metadata from original variable like we would if we were creating
# subvariable during original derivation...
# TODO: It would be nice to update name in the same POST as updating derivation
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a derivation can have a subreferences field? Is that the best way?

# subvariable during original derivation...
# TODO: It would be nice to update name in the same POST as updating derivation
# to ensure consistency, but I don't see how yet
# TODO: Also, the alias is really ugly for newly created subvariables
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to get the server's naming logic? I tried just adding the alias explicitly, hoping that the server would dedup for me, but this resulted in a failure.


with_test_authentication({
# adding to derived array
# TODO: Use existing fixture? Add this to fixtures?
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This simple example works well for these tests, but we usually use newDatasetFromFixture(), worth changing to that (either by using existing fixture or adding this one there)?

@lintr-bot
Copy link

R/add-subvariable.R:96:1: style: Lines should not be more than 100 characters.

new_deriv <- addToSelectCatDerivation(old_deriv, subvariable, new_map_ids, categories(variable))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

@lintr-bot
Copy link

R/add-subvariable.R:96:1: style: Lines should not be more than 100 characters.

new_deriv <- addToSelectCatDerivation(old_deriv, subvariable, new_map_ids, categories(variable))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants