Add InputStream support to VCFFileReader

[btiernay]

This is an enabler for remote streaming of VCF files from remote locations or even stdin. A common use case would be reading from an S3 bucket a slice of a VCF file without having to first transfer the entire file locally. The current File requirement is quite limiting for these types of usages.

[lbergelson]

Hi @btiernay. One alternative to streaming from stdin would be to implement or find a java.nio.FileSystemProvider for S3 buckets. We've implemented support for reading VCFs from anything that can be represented by a java.nio.Path. There's an existing implementation for Google's object store. The nice thing about doing it that way is that it will automatically have support for things like the vcf index so you can read just the chunks of the file you want.

[lindenb]

@btiernay @lbergelson related PR: #837

[delocalizer]

@lbergelson a while back you mentioned above functionality to read VCFs (and indexes?) from Path; is this the stuff in yf_vcf_from_path branch and is there a timeline on merging to master? We'd love this feature.

[lbergelson]

@delocalizer You can currently load a vcf file and index from a path by using something along the lines of

AbstractFeatureReader.getFeatureReader(path.toAbsolutePath().toUri().toString(), new VCFCodec())

It takes a String for complicated legacy reasons, but it will resolve the string with the appropriate FileSystem. That overload will try to autodiscover the index, but there are other overloads that let you explicitly pass in an index, or not require one, or pass in other options.

We use this in gatk's FeatureDataSource which also does other things like automatic codec discovery and asynchronous prefetching from remote paths.

[delocalizer]

Thanks @lbergelson; this is good!