TSPS-269 Speed up CountVariantsInChunksBeagle by using bedtools #1335
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Can one of the admins verify this patch? |
jsotobroad
reviewed
Jul 17, 2024
| @@ -161,7 +161,7 @@ task CountVariantsInChunks { | |||
| set -e -o pipefail | |||
|
|
|||
| echo $(gatk --java-options "-Xms~{command_mem}m -Xmx~{max_heap}m" CountVariants -V ~{vcf} | sed 's/Tool returned://') > var_in_original | |||
| echo $(gatk --java-options "-Xms~{command_mem}m -Xmx~{max_heap}m" CountVariants -V ~{vcf} -L ~{panel_vcf} | sed 's/Tool returned://') > var_in_reference | |||
| echo $(gatk --java-options "-Xms~{command_mem}m -Xmx~{max_heap}m" CountVariants -V ~{vcf} -L ~{panel_vcf} | sed 's/Tool returned://') > var_in_reference | |||
There was a problem hiding this comment.
should we also update this task to be more efficient too?
There was a problem hiding this comment.
i don't think so, this is used in the minimac wdl and i don't really want to touch that one
| input { | ||
| File ref_panel_interval_list | ||
|
|
||
| Int disk_size_gb = ceil(2*size(ref_panel_interval_list, "GiB")) + 50 # not sure how big the disk size needs to be since we aren't downloading the entire VCF here |
There was a problem hiding this comment.
this comment is probably copy pasta'd?
|
oh also just double checking that workflows that used to fail the QC still continue to fail the QC check (if you use the simulated data as input it should fail the QC check) |
jsotobroad
approved these changes
Jul 18, 2024
| @@ -161,7 +161,7 @@ task CountVariantsInChunks { | |||
| set -e -o pipefail | |||
|
|
|||
| echo $(gatk --java-options "-Xms~{command_mem}m -Xmx~{max_heap}m" CountVariants -V ~{vcf} | sed 's/Tool returned://') > var_in_original | |||
| echo $(gatk --java-options "-Xms~{command_mem}m -Xmx~{max_heap}m" CountVariants -V ~{vcf} -L ~{panel_vcf} | sed 's/Tool returned://') > var_in_reference | |||
| echo $(gatk --java-options "-Xms~{command_mem}m -Xmx~{max_heap}m" CountVariants -V ~{vcf} -L ~{panel_vcf} | sed 's/Tool returned://') > var_in_reference | |||
mmorgantaylor
merged commit 95c3941
into
TSPS-183_mma_beagle_imputation_hg38
1 of 2 checks passed
mmorgantaylor
added a commit
that referenced
this pull request
Feb 13, 2025
* try creating bed files * try again * try again again * a different thing * use bedtools and bed ref panel files * oops update the correct task * fix * use the right freaking file name * remove comment
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Stats for this task before and after the change:
1000 sample input, 1000G ref panel:
This PR also includes an update to the Ref Panel generation wdl to include creating the ref panel bed files used in the new task.
Also tested that the QC check for overlapping sites between input and reference still works: https://app.terra.bio/#workspaces/morgan-fieldeng/Imputation_pipeline_testing/job_history/d116884a-d918-44d1-900e-b91691a2b3b1
old branch: 500 sample input, 1000G ref panel. shard 0 / ... / CountVariantsInChunksBeagle / shard 0
var_in_original: 7574
var_also_in_reference: 7076
this branch: 500 sample input, sim 10k ref panel. shard 0 / .../ CountVariantsInChunksBeagle / shard 0
var_in_original: 7574
var_also_in_reference: 426
Checklist
If you can answer "yes" to the following items, please add a checkmark next to the appropriate checklist item(s) and notify our WARP documentation team by tagging either @ekiernan or @kayleemathews in a comment on this PR.