Sample name is not recognized in "get_sample_ploidy_metadata" method when generating "segments" VCF file using PostprocessGermlineCNVCalls

[shulik7]

Hi,

I am trying to call germline CNVs for a set of samples. After running DetermineGermlineContigPloidy and GermlineCNVCaller, I am using PostprocessGermlineCNVCalls to generate the VCF files with CNV calls. The "interval" VCF files are generated successfully. But I got the following error message when segmenting contigs:

org.broadinstitute.hellbender.utils.python.PythonScriptExecutorException:
python exited with 1
Command Line: python /tmp/shulik7/segment_gcnv_calls.2338024416841754264.py --ploidy_calls_path /scratch/users/shulik7/test_GATK_CNV/Postprocess/../DetermineGermlineContigPloidy/model/test_run-calls/ --model_shards /scratch/shulik7/test_GATK_CNV/Postprocess/../GermlineCNVCaller/cnvs/test_run-model --calls_shards /scratch/shulik7/test_GATK_CNV/Postprocess/../GermlineCNVCaller/cnvs/test_run-calls --output_path /tmp/shulik7/gcnv-segmented-calls28280883609685538 --sample_index 0
Stdout: 11:32:16.728 INFO segment_gcnv_calls - Loading ploidy calls...
11:32:16.729 INFO gcnvkernel.io.io_metadata - Loading germline contig ploidy and global read depth metadata...
11:32:16.730 INFO segment_gcnv_calls - Instantiating the Viterbi segmentation engine...
11:32:18.585 INFO gcnvkernel.postprocess.viterbi_segmentation - Assembling interval list and copy-number class posterior from model shards...
11:32:25.158 INFO gcnvkernel.structs.metadata - Generating intervals metadata...
11:32:27.543 INFO gcnvkernel.postprocess.viterbi_segmentation - Compiling theano forward-backward function...
11:32:34.406 INFO gcnvkernel.postprocess.viterbi_segmentation - Compiling theano Viterbi function...
11:32:40.598 INFO gcnvkernel.postprocess.viterbi_segmentation - Compiling theano variational HHMM...
11:32:42.862 INFO gcnvkernel.postprocess.viterbi_segmentation - Processing sample index: 0, sample name: test_sample_0...
11:32:43.631 INFO gcnvkernel.postprocess.viterbi_segmentation - Segmenting contig (1/24) (contig name: 1)...

Stderr: Traceback (most recent call last):
File "/tmp/shulik7/segment_gcnv_calls.2338024416841754264.py", line 73, in
viterbi_engine.write_copy_number_segments_for_single_sample(args.sample_index)
File "/home/shulik7/miniconda3/envs/gatk/lib/python3.6/site-packages/gcnvkernel/postprocess/viterbi_segmentation.py", line 265, in write_copy_number_segments_for_single_sample
for segment in self._viterbi_segments_generator_for_single_sample(sample_index):
File "/home/shulik7/miniconda3/envs/gatk/lib/python3.6/site-packages/gcnvkernel/postprocess/viterbi_segmentation.py", line 160, in _viterbi_segments_generator_for_single_sample
.get_sample_ploidy_metadata(sample_name)
File "/home/shulik7/miniconda3/envs/gatk/lib/python3.6/site-packages/gcnvkernel/structs/metadata.py", line 278, in get_sample_ploidy_metadata
return self.sample_ploidy_metadata_dict[sample_name]
KeyError: 'test_sample_0'

    at org.broadinstitute.hellbender.utils.python.PythonExecutorBase.getScriptException(PythonExecutorBase.java:75)
    at org.broadinstitute.hellbender.utils.runtime.ScriptExecutor.executeCuratedArgs(ScriptExecutor.java:126)
    at org.broadinstitute.hellbender.utils.python.PythonScriptExecutor.executeArgs(PythonScriptExecutor.java:170)
    at org.broadinstitute.hellbender.utils.python.PythonScriptExecutor.executeScript(PythonScriptExecutor.java:151)
    at org.broadinstitute.hellbender.utils.python.PythonScriptExecutor.executeScript(PythonScriptExecutor.java:121)
    at org.broadinstitute.hellbender.tools.copynumber.PostprocessGermlineCNVCalls.executeSegmentGermlineCNVCallsPythonScript(PostprocessGermlineCNVCalls.java:500)
    at org.broadinstitute.hellbender.tools.copynumber.PostprocessGermlineCNVCalls.generateSegmentsVCFFileFromAllShards(PostprocessGermlineCNVCalls.java:436)
    at org.broadinstitute.hellbender.tools.copynumber.PostprocessGermlineCNVCalls.traverse(PostprocessGermlineCNVCalls.java:297)
    at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:893)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:134)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:179)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:198)
    at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
    at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
    at org.broadinstitute.hellbender.Main.main(Main.java:289)

The sample_name.txt file in ../DetermineGermlineContigPloidy/model/test_run-calls/SAMPLE_0/ folder has the file name in it:
$cat ../DetermineGermlineContigPloidy/model/test_run-calls/SAMPLE_0/sample_name.txt
test_sample_0

The version of GATK4 I am running is 4.0.3.0

[mbabadi]

Dear @shulik7, according to the error message, gcnvkernel is expecting ploidy calls at the following path: /scratch/users/shulik7/test_GATK_CNV/Postprocess/../DetermineGermlineContigPloidy/model/test_run-calls/.

Could you please assert that the above path is indeed the ploidy calls paths, and if so, whether it contains test_sample_0 under one of SAMPLE_x subdirs? if the path is valid, please try running PostprocessGermlineCNVCalls with an absolute path and report back.

[shulik7]

Hi @mbabadi ,
Yes, there are sub directories SAMPLE_0 to SAMPLE_39 under the path I provided. I tried to use the absolute path and the problem resolved. Thanks for the suggestion!

[samuelklee]

Thanks for bringing this issue to our attention, @shulik7. @mbabadi, does it resolve if we pass a canonical path, rather than just an absolute path?

[mbabadi]

@samuelklee we have to use .getCanonicalPath() instead of .getAbsolutePath() when passing paths to Python. It looks like python does not like foo/../../bar too much.