Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update CramCompressionRecord.isPlaced() to make it APDelta-aware #1284

Merged
merged 20 commits into from
Feb 21, 2019

Conversation

jmthibault79
Copy link
Contributor

@jmthibault79 jmthibault79 commented Feb 14, 2019

Description

isPlaced() was not checking for the APDelta flag, thus inappropriately marking some records as unplaced.

Error introduced on 1/30 with #1266

Checklist

  • Code compiles correctly
  • New tests covering changes and new functionality
  • All tests passing
  • Extended the README / documentation, if necessary
  • Is not backward compatible (breaks binary or source compatibility)

@codecov-io
Copy link

codecov-io commented Feb 14, 2019

Codecov Report

Merging #1284 into master will increase coverage by 0.278%.
The diff coverage is 97.222%.

@@              Coverage Diff               @@
##             master     #1284       +/-   ##
==============================================
+ Coverage     67.49%   67.768%   +0.278%     
- Complexity     8150      8345      +195     
==============================================
  Files           558       561        +3     
  Lines         33365     33929      +564     
  Branches       5608      5752      +144     
==============================================
+ Hits          22518     22993      +475     
- Misses         8658      8733       +75     
- Partials       2189      2203       +14
Impacted Files Coverage Δ Complexity Δ
...ava/htsjdk/samtools/cram/build/CramNormalizer.java 81.25% <100%> (+0.107%) 57 <0> (+1) ⬆️
...ain/java/htsjdk/samtools/cram/structure/Slice.java 63.008% <100%> (+2.76%) 76 <1> (+38) ⬆️
...samtools/cram/structure/CramCompressionRecord.java 82.308% <100%> (+3.141%) 96 <21> (+8) ⬆️
...ava/htsjdk/samtools/CRAMContainerStreamWriter.java 70.612% <50%> (-0.289%) 54 <0> (ø)
...c/main/java/htsjdk/samtools/util/SnappyLoader.java 78.571% <0%> (-2.198%) 9% <0%> (ø)
src/main/java/htsjdk/samtools/SamReader.java 81.818% <0%> (-0.94%) 0% <0%> (ø)
...va/htsjdk/samtools/SAMSequenceDictionaryCodec.java 86.957% <0%> (-0.543%) 8% <0%> (+2%)
src/main/java/htsjdk/samtools/util/CigarUtil.java 30.303% <0%> (-0.466%) 14% <0%> (ø)
...java/htsjdk/samtools/cram/ref/ReferenceSource.java 46.364% <0%> (-0.425%) 19% <0%> (ø)
...htsjdk/tribble/readers/LongLineBufferedReader.java 29.944% <0%> (-0.342%) 15% <0%> (ø)
... and 32 more

Copy link
Collaborator

@cmnbroad cmnbroad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments/questions.

@@ -379,8 +379,8 @@ protected void flushContainer() throws IllegalArgumentException, IllegalAccessEx
while (last.next != null) last = last.next;

if (cramRecord.isFirstSegment() && last.isLastSegment()) {

final int templateLength = CramNormalizer.computeInsertSize(cramRecord, last);
final boolean coordinateSorted = (samFileHeader.getSortOrder() == SAMFileHeader.SortOrder.unsorted);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one seems inverted - it will be false for coordinate sorted. Since no tests failed, we should add one somewhere that verifies this is working correctly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops, good catch.

Unfortunately, I believe that it would take me a substantial amount of time to make sense of CramNormalizer and write a reasonable test for it, so I don't think that's feasible with the quick turnaround time we want for this PR.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any test in this PR that would have failed before without the fix for APDelta, but passes now so at least we can be sure that we've fixed it ? I assume we didn't have one before this bug was introduced, since no test caught the issue ? Without that, I'm not even sure I'd recognize the failure if I saw it. Ideally we'd create a ticket describing it (that will be fixed by this PR) so we have a record in case anyone sees it using the interim version that had the issue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. The pair of SliceTests cases starting with the comment on line 97 calls this out explicitly.

On a lower level, the combinations in CramCompressionRecordTest.placedTests show that the APDelta flag does affect results, and some would fail without this fix.

@jmthibault79 jmthibault79 marked this pull request as ready for review February 15, 2019 21:54
Copy link
Collaborator

@cmnbroad cmnbroad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think its worth investing in some test cleanup for this one - lots of test comments.

Copy link
Member

@lbergelson lbergelson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nicer if CramCompressionRecord knew if it was delta encoded, but this is fine too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants