Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Head and Tail methods #176

Open
wants to merge 34 commits into
base: master
Choose a base branch
from
Open

Implement Head and Tail methods #176

wants to merge 34 commits into from

Conversation

gshotwell
Copy link
Contributor

Creates head and tail methods for Crunch Datasets, Dataframes, and Variables, this also implements numeric subsetting on dataset rows.

@codecov
Copy link

codecov bot commented Dec 12, 2017

Codecov Report

Merging #176 into master will increase coverage by 0.23%.
The diff coverage is 83.72%.

Impacted file tree graph

@@           Coverage Diff            @@
##           master   #176      +/-   ##
========================================
+ Coverage   89.77%    90%   +0.23%     
========================================
  Files         102    104       +2     
  Lines        6151   6533     +382     
========================================
+ Hits         5522   5880     +358     
- Misses        629    653      +24
Impacted Files Coverage Δ
R/AllGenerics.R 91.94% <100%> (-0.54%) ⬇️
R/dataset-extract.R 92% <80%> (-6.67%) ⬇️
R/as-data-frame.R 97.5% <85.71%> (+3.38%) ⬆️
R/variable.R 82.66% <88.88%> (+0.05%) ⬆️
R/folders.R 85.18% <0%> (-4.82%) ⬇️
R/members.R 77.55% <0%> (-4.27%) ⬇️
R/show.R 86.26% <0%> (-0.48%) ⬇️
R/abstract-categories.R 100% <0%> (ø) ⬆️
... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 93b4abb...d51d9ec. Read the comment docs.

@gshotwell gshotwell changed the title Head tail Implement Head and Tail methods Dec 12, 2017
@gshotwell gshotwell closed this Dec 13, 2017
@gshotwell gshotwell reopened this Dec 13, 2017
@nealrichardson
Copy link
Contributor

This is currently blocked due to overly strict validation on the csv export endpoint on the server.

@gshotwell
Copy link
Contributor Author

The ticket for the validation is: https://www.pivotaltracker.com/story/show/154086828

Remaining todos on the R side are:

  • Generate mocks for numeric row subsetting
  • Check test coverage

Copy link
Contributor

@jonkeane jonkeane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few changes to the methods of dispatch.

I also added some tests that fail when using with an exclusion filter. This is another call to adding the ability in the backend to return arbitrary row (orders) https://www.pivotaltracker.com/story/show/151013797 Thought it's possible we can still work around the exclusion filter here with similar clever harmonizing magic like is here with filters.

#' @export
setMethod("tail", "CrunchDataFrame", function (x, n=6L, ...) {
return(tail(attr(x, "crunchDataset")))
})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't look like these would respect the ordering of the CrunchDataFrame which is in the row.order attribute.

I don't think it would be too difficult to add that, though we haven't been using that feature too much (it was designed to make merging onto simple feature style geographies possible, but I haven't had the time to push forward on that recently)

#' @aliases head tail
NULL

#' @rdname head-tail
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the head/tail methods that we are imitating are based on S3 classes, we should probably match those and define this (and similar for the following method definitions) as:

head.CrunchDataset <- function ...

Then remove the lines in AllGenerics.R that make the generics. This will prevent masking when the package is loaded.

@gshotwell
Copy link
Contributor Author

It looks like this PR is blocked by some odd behaviour around datasets which have an exclusion. When you have a dataset with an exclusion, then try to pass a logical row filter, the API returns incorrect data. We think that this is because the row filter is being applied to the unexcluded data.

Decision is to wait on this method until we implement https://www.pivotaltracker.com/n/projects/931610/stories/151013797 at which point we can refactor all of the row_number filtering to use a more concise version of numeric subsetting.

@nealrichardson
Copy link
Contributor

If row filtering on the server doesn't respect the exclusion filter, that sounds like a bug that needs fixing (independent of this PR).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants