UpdateVcfSequenceDictionary cannot write index

[nh13]

@ronlevine looks like you introduced a bug, can you fix? Try writing it to a file with CREATE_INDEX=true. Since you wrapped it with an output stream :/. I am curious as to why the change was made at all, since shouldn't VariantContextWriterBuilder.determineOutputTypeFromFile handle this? What was the original problem?

https://github.com/broadinstitute/picard/blob/master/src/main/java/picard/vcf/UpdateVcfSequenceDictionary.java#L92-L99

[nh13]

@ronlevine even better, if the output file is output.vcf.gz, it doesn't write a gzipped file, but rather a text file named output.vcf.gz.

[nh13]

#741

[nh13]

@ronlevine I have added a potential fix into htsjdk so that setOutputFile should work: samtools/htsjdk#933

[ronlevine]

Reverted the non-test changes from #741 and fixed/expanded tests on https://github.com/broadinstitute/picard/compare/rhl_uvsd_fix_stdout. samtools/htsjdk#933 should fix the stdout test failure.

[nh13]

I checked out both your htsjdk and picard branches and got the following when testing it:

To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp
Exception in thread "main" java.lang.NullPointerException
	at htsjdk.variant.variantcontext.writer.VCFWriter.writeAndResetBuffer(VCFWriter.java:125)
	at htsjdk.variant.variantcontext.writer.VCFWriter.writeHeader(VCFWriter.java:136)
	at picard.vcf.UpdateVcfSequenceDictionary.doWork(UpdateVcfSequenceDictionary.java:94)
	at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:205)
	at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:94)
	at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:104)

[ronlevine]

The Picard branch should get that NPE since the htsjdk fix from nh_vcf_to_stdout is not yet in. Update nh_vcf_to_stdout and try again, it should work fine. nh_vcf_to_stdout runs without error locally for me as well as on travis, the automated testing framework.

[nh13]

Still getting the NPE. I checked out the branch in htsjdk, ran ./gradlew install printVersion, then ran $ ./gradlew shadowJar -Dhtsjdk.version=2.10.1-17-g269f0c1-SNAPSHOT in Picard. Can you send me an executable jar for me to verify?

[ronlevine]

I am getting the same error. VCFWriter is trying to write a buffer to a null outputStream.This needs to be set. I added your original change back into htsjdk and am still getting the same error. The htsjdk snapshot is now 2.10.1-18-gc24bd07-SNAPSHOT.

[ronlevine]

I made a change in VariantContextWriter.build() so the VCF_STREAM case does not use a null stream:

final OutputStream writerStream = outStream != null ? outStream : outStreamFromFile;

The following command successfully wrote to stdout without error:
java -jar ./build/libs/picard.jar UpdateVcfSequenceDictionary INPUT=testdata/picard/vcf/vcfFormatTest.vcf OUTPUT=/dev/stdout SD=testdata/picard/vcf/dummy.reference.dict. I need to fix the Picard test for stdout because the output is not being captured to the file used for diffing.

[nh13]

@ronlevine can you take a look at this?

[yfarjoun]

@cmnbroad feel like taking this one too?