Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BetaIntegerCodecTest and bugfixes #1199

Merged
merged 11 commits into from
Oct 19, 2018
Merged

Conversation

jmthibault79
Copy link
Contributor

@jmthibault79 jmthibault79 commented Oct 18, 2018

Checklist

  • Code compiles correctly
  • New tests covering changes and new functionality
  • All tests passing
  • Extended the README / documentation, if necessary
  • Is not backward compatible (breaks binary or source compatibility)

@codecov-io
Copy link

codecov-io commented Oct 18, 2018

Codecov Report

Merging #1199 into master will increase coverage by 0.074%.
The diff coverage is 95.238%.

@@              Coverage Diff               @@
##             master     #1199       +/-   ##
==============================================
+ Coverage     68.45%   68.524%   +0.074%     
- Complexity     8022      8035       +13     
==============================================
  Files           542       542               
  Lines         32704     32714       +10     
  Branches       5530      5534        +4     
==============================================
+ Hits          22386     22417       +31     
+ Misses         8113      8095       -18     
+ Partials       2205      2202        -3
Impacted Files Coverage Δ Complexity Δ
...tsjdk/samtools/cram/encoding/BetaIntegerCodec.java 91.667% <95.238%> (+48.81%) 8 <8> (+6) ⬆️
...va/htsjdk/samtools/sra/SRAIndexedSequenceFile.java 64.865% <0%> (+2.703%) 7% <0%> (+1%) ⬆️
...samtools/util/AsyncBlockCompressedInputStream.java 76% <0%> (+4%) 13% <0%> (+1%) ⬆️
...sjdk/samtools/util/Md5CalculatingOutputStream.java 74.359% <0%> (+7.692%) 8% <0%> (+1%) ⬆️
...tsjdk/samtools/cram/io/DefaultBitOutputStream.java 53.409% <0%> (+9.091%) 16% <0%> (+4%) ⬆️

Copy link
Contributor

@pshapiro4broad pshapiro4broad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few minor comments


int[] actual = new int[values.length];
try (InputStream is = new ByteArrayInputStream(os.toByteArray())) {
BitInputStream bis = new DefaultBitInputStream(is);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should declare bis in the try (...) as well, since it's a closeable resource.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bis is not AutoCloseable (we should fix that but not today) but I can make it a DefaultBitInputStream which is.

Copy link
Member

@lbergelson lbergelson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jmthibault79 Your changes look good. I have some refactoring suggestions and some things that I think also probably be error cases. If you want to keep this as simple bugfix PR I can do my refactoring suggestions in another branch. Class / constructor javadoc would be really useful here since I had no idea what the point of this was without reading the code and looking at the cram spec.

@@ -47,8 +47,8 @@ public final long write(final BitOutputStream bitOutputStream, final Integer val

@Override
public final long numberOfBits(final Integer value) {
if (value > (1L << readNofBits))
throw new IllegalArgumentException("Value written is bigger then allowed: value=" + value
if (value + offset >= (1L << readNofBits))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like the correct case. Do we not have any code that actually sets an offset so we never noticed the bug?

It seems like there needs to be an additional constrain here which is value + offset >= 0

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There were 2 errors here: in addition to the offset case, the > meant we allowed e.g. 0x100 for 8 bits. I don't know whether we would have hit these cases before.

It's not clear to me that we should be disallowing negative values here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Considering this more thoroughly: yes, this codec is intended to store only positive values (though the source data series may be negative). I'll add a check, a test, and javadoc about this.

@jmthibault79
Copy link
Contributor Author

Ready for re-review @lbergelson

Copy link
Member

@lbergelson lbergelson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jmthibault79 This looks good now. Thanks for making those changes, I think it's easier to understand and more robust now.

@jmthibault79 jmthibault79 merged commit bbc674f into master Oct 19, 2018
@jmthibault79 jmthibault79 deleted the jt_BetaIntegerCodecTest branch October 19, 2018 21:59
lbergelson pushed a commit that referenced this pull request Dec 3, 2018
*  allow BetaIntegerEncoding to be specified with 0 bits.  
    This was disabled in #1199, as a probable error state, but it turns out that it's used in some crams created with htslib.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants