Freeze 7

[ch-kr]

Code updating 455k vcf export for return to UKBB.

Updates:

• Created sample and vcf (variant) data dictionaries and summaries (required by UKBB): sample_data_dicts.py, vcf_data_dicts.py, data_dict_summaries.md
• Updated raw MT path (old path has been moved from standard storage to nearline): basics.py
• Updated excluded samples path to point to most recent file for samples who withdrew consent: basics.py
• Updated get_ukbb_data to remove new sample IDs with withdrawn consent. Also updated get_ukbb_data to exclude samples with undefined batch if ukbb_samples_only is set: basics.py
• Updated index dict functions to remove subset, subpop information: utils.py
• Updated validity checks to remove subset/subpop information and remove unnecessary checks on gnomAD data: sanity_checks.py. NOTE that the code is still expecting an older version of the validity checks from gnomAD methods -- this is the commit that will work (cannot use a more recent commit): d43951086bfea1b80b1bad34b3193f7c031dbc8d
• Removed code adding/unfurling gnomAD fields for export: prepare_vcf_data_release.py
• Added homalt release patch frequency to release VCF HT, MT: prepare_vcf_data_release.py
• Updated homalt hotfix GT adjustment: prepare_vcf_data_release.py
• Added code to get vcf shard start/end positions: prepare_vcf_data_release.py
• Added code to generate manifest (required by DNAnexus for transfer): manifest.py
• Added batch script to repackage VCF (required by DNAnexus): vcf_repackage.py
• Added header repackage code from DNAnexus: ukbb_header_reformat.sh
• Edited meta HT to remove control samples/samples with withdrawn consent/samples with undefined batch, rekey using UKBB ID, and rename ukbb_meta to ukb_meta: create_meta_ht.py

I think these are all the changes we need to return data -- please let me know if I missed anything!

[jkgoodrich]

This is looking good so far. I didn't get through it all because it's very big, but I wanted to put in my current comments because I will be in meetings all day tomorrow and won't get to the rest of the files until Friday.