Allow missing vcf samples or gvcfs in Module00c

[mwalker174]

For large cohorts, it is useful in practice to be able to omit certain samples from BAF generation (in many cases owing to issues with SNP/indel calling on a small subset of samples). This typically does not affect results, as BAF is used only for training in Module03.

This PR adds a ignore_missing_baf_samples option to 00c that is off by default. Setting to true skips cross-checks between the sample list and provided VCF/gVCF samples (an error is thrown if an expected sample is not present in the snp data). When using gVCFs for BAF generation, the input type has been changed from Array[File]? to Array[File?]? to allow for null inputs. For the sharded VCF method of BAF generation, samples can simply be absent.

This branch was tested using the default 00c test_small, and a modified version of the Module00cTest.test_baf_from_vcf.json input where the first sample gVCF was omitted.

[epiercehoffman]

This looks good to me. I think your reorganization of GenerateBAF and GatherBAF makes sense. I assume the intention behind passing samples to GenerateBAF for GVCFs is so that it will error out for missing samples if ignore_missing_gvcfs is not set?

I noticed there are some linting errors that should be addressed, but could probably be merged without remaking the dockers for now.

To provide gvcfs for most but not all samples, would the input look like this? It might be worth documenting this as I doubt most people have worked with Array[File?]? inputs and I didn't see anything about it in the WDL spec.

"Module00c.gvcfs": ["gvcf.sample1.vcf.gz", , "gvcf.sample3.vcf.gz"]

Has Module03 been tested with missing BAF for some samples in the past?

[epiercehoffman]

Another thought - should we provide guidance on a minimum % of samples that need BAF?

[mwalker174]

This looks good to me. I think your reorganization of GenerateBAF and GatherBAF makes sense. I assume the intention behind passing samples to GenerateBAF for GVCFs is so that it will error out for missing samples if ignore_missing_gvcfs is not set?

Yes it is for cross-checking with the SNP calls.

I noticed there are some linting errors that should be addressed, but could probably be merged without remaking the dockers for now.

Fixed!

To provide gvcfs for most but not all samples, would the input look like this? It might be worth documenting this as I doubt most people have worked with Array[File?]? inputs and I didn't see anything about it in the WDL spec.

"Module00c.gvcfs": ["gvcf.sample1.vcf.gz", , "gvcf.sample3.vcf.gz"]

Should look like ["Module00c.gvcfs": ["gvcf.sample1.vcf.gz", null, "gvcf.sample3.vcf.gz"]. Added a note in the documentation.

Has Module03 been tested with missing BAF for some samples in the past?

Yes we ran gnomAD-v3 with some samples missing BAF. Anything around ~1% should be very tolerable, but we haven't tested beyond that. Added a quick note in the documentation.