Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNVQ recalibration for flow based reads #8697

Merged
Merged
Changes from 1 commit
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
0ce6afe
ApplySNVQR tool squashed
dror27 Nov 14, 2023
bf5d2d8
[BIOIN-1385] FlowBasedRead now can estimate filling_value from the read
ilyasoifer Dec 27, 2023
876a4e6
[BIOIN-1385] Removed probabilityRatioThreshold
ilyasoifer Dec 27, 2023
86d3ae4
[BIOIN-1385] Removed mistaken usage by fillingValue
ilyasoifer Dec 28, 2023
c5e2ad8
[BIOIN-1385] Fixed bug in t0 parsing
ilyasoifer Dec 30, 2023
fcbbd3c
[BIOIN-1385] Reverted the aggressive change in filling probability in…
ilyasoifer Jan 1, 2024
ba29b3c
[BIOIN-1385] Minor update of poorly_modelded_reads
ilyasoifer Jan 1, 2024
ac66953
[BIOIN-1385] FlowBasedRead class cleanup
ilyasoifer Jan 2, 2024
8d9782f
[BIOIN-1385] Updated ground truth scorer for the case when the fillin…
ilyasoifer Jan 2, 2024
a032030
Fixed typing error + missed file
ilyasoifer Jan 3, 2024
626a458
ceil -> round
ilyasoifer Jan 3, 2024
b4d8341
Set phred score for bins with no zeros to maximum rather than zero
ilyasoifer Jan 5, 2024
2d69e85
Updated tests
ilyasoifer Jan 5, 2024
92aab19
Updated tests
ilyasoifer Jan 5, 2024
c564dc6
Updated tests
ilyasoifer Jan 5, 2024
51ebdb5
-10 qual fix
dror27 Jan 18, 2024
a7947b9
Guard against negative qualities
dror27 Jan 21, 2024
5e6da62
(not final) reuse AddFlowBaseQuality code to accelerate
dror27 Jan 22, 2024
fcb1f83
--debug-use-add-flow-base-quality-algorithm flasg
dror27 Jan 23, 2024
8f17ea5
use of add_base_q now default
dror27 Jan 23, 2024
8634c11
ApplySNVQR test data rebuilt
dror27 Jan 23, 2024
9ba77d3
add_base_q integration in progress ... missing mapping function
dror27 Jan 23, 2024
44b6763
allow unsorted/unmapped input to ApplySNVQR when no model specified
dror27 Jan 24, 2024
2f478de
SNVQ of called base adjusted to sum to 1
dror27 Jan 24, 2024
13aa6ee
update called base snvq
dror27 Jan 25, 2024
5850557
--limit-phred-score replaces --limit-score
dror27 Jan 28, 2024
d6707d2
flow flags passed to add base quality
dror27 Jan 29, 2024
d0e4a0f
BQ adjusted
dror27 Jan 29, 2024
f79da59
Fixed boundary case
ilyasoifer Jan 31, 2024
b2337dd
Snvq modes added. see --snvq-mode param
dror27 Feb 4, 2024
78818b4
SNVq corrected (all 80 issue)
dror27 Feb 5, 2024
f081cb5
Add missing X_ADJACENT_REF_DIFF
dror27 Feb 19, 2024
1a8b734
Tool renamed to AddFlowSNVQuality and model/conf removed
dror27 Feb 26, 2024
1461c82
further cleanup tests
dror27 Feb 26, 2024
1170d4b
FlowFeatureMapper changes removed
dror27 Feb 26, 2024
7da3aa6
expected results adjusted
dror27 Feb 26, 2024
2b25b21
snvq-mode removed. Geometric mode is one used
dror27 Mar 3, 2024
3cb409c
some comments addressed
dror27 Mar 3, 2024
94d4fe7
Update src/main/java/org/broadinstitute/hellbender/tools/walkers/feat…
dror27 Mar 3, 2024
4afc042
Update src/main/java/org/broadinstitute/hellbender/tools/walkers/feat…
dror27 Mar 3, 2024
4f8320e
0.0001 to keep consistent with GATK
dror27 Mar 3, 2024
9c17210
Merge branch 'ultima.snv-qual.develop.rebase' of github.com:Ultimagen…
dror27 Mar 3, 2024
5577f79
includeDupReads removed
dror27 Mar 3, 2024
ef7ea5e
additional updates
dror27 Mar 3, 2024
eacc100
DEFAULT_FILLING_VALUE restored to 0.001
dror27 Mar 11, 2024
a2b7f54
snvq-mode restored
dror27 Mar 11, 2024
06284b0
AddFlowSNVQuality integeration test expected data adjusted
Mar 16, 2024
4ee1255
fix comment spec on expected output
dror27 Mar 16, 2024
bc0ff83
some of the comments addressed
dror27 Mar 18, 2024
ff06e36
additional comments addressed
dror27 Mar 21, 2024
403bd29
additional comments addressed
dror27 Mar 25, 2024
a6afd8e
Clarified function
ilyasoifer Mar 28, 2024
516c152
limit scores precision to regain platform independence
dror27 Mar 31, 2024
4ee5b6d
documented generateHmerBaseErrorProbabilities
dror27 Mar 31, 2024
d9c090b
AddFlowSNVQuality now overrides read base quality by default
dror27 Mar 31, 2024
0c73917
additional doc comments addressed
dror27 Apr 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
--limit-phred-score replaces --limit-score
dror27 committed Jan 28, 2024
commit 58505575ee45d01f229f225bd911a227ca0be45a
Original file line number Diff line number Diff line change
@@ -20,12 +20,18 @@ public final class AddFlowSNVQuality {
public static final int ERROR_PROB_BAND_1MORE = 2;
public static final int ERROR_PROB_BANDS = 3;

public double minErrorRate = 1e-6;
public double maxErrorRate = 1 - minErrorRate;
public double minLikelihoodProbRate = 1e-6;
public int maxQualityScore = 60;

public FlowBasedArgumentCollection fbargs = new FlowBasedArgumentCollection();

public void addBaseQuality(final GATKRead read, final SAMFileHeader hdr) {
public void addBaseQuality(final GATKRead read, final SAMFileHeader hdr, double limitPhoreScore) {

// take in phred score limit
if ( !Double.isNaN(limitPhoreScore) ) {
maxQualityScore = (int)limitPhoreScore;
minLikelihoodProbRate = Math.pow(10, -limitPhoreScore / 10.0);
}

// convert to a flow base read
final FlowBasedReadUtils.ReadGroupInfo rgInfo = FlowBasedReadUtils.getReadGroupInfo(hdr, read);
@@ -73,7 +79,7 @@ private double[] generateBaseErrorProbability(final FlowBasedRead fbRead, final
* for a description of the flow probabilities see {@link FlowBasedRead#flowMatrix}
*/
final int[] key = fbRead.getKey();
final double[][] errorProbBands = extractErrorProbBands(fbRead, minErrorRate);
final double[][] errorProbBands = extractErrorProbBands(fbRead, minLikelihoodProbRate);
final double[] result = new double[fbRead.getBasesNoCopy().length];

final double[][] snvResult = new double[flowOrderLength][];
@@ -110,7 +116,7 @@ private double[] generateBaseErrorProbability(final FlowBasedRead fbRead, final
if ( allBaseProb0.containsKey(flowOrder[i]) ) {
snvResult[i][base - 1] = allBaseProb0.get(flowOrder[i]);
} else if ( i != flow_i ) {
snvResult[i][base - 1] = minErrorRate;
snvResult[i][base - 1] = minLikelihoodProbRate;
}
}
}
@@ -128,11 +134,11 @@ private double[] generateBaseErrorProbability(final FlowBasedRead fbRead, final
if ( allBaseProb1.containsKey(flowOrder[i]) ) {
final double p = allBaseProb1.get(flowOrder[i]);
for ( int j = 0 ; j < hmerLength - 1 ; j++ ) {
snvResult[i][base - 1 - j] = p;
snvResult[i][base - 1 - j] = (j == 0) ? p : minLikelihoodProbRate; // all but last get the min prob
}
} else if ( i != flow_i ) {
for ( int j = 0 ; j < hmerLength - 1 ; j++ ) {
snvResult[i][base - 1 - j] = minErrorRate;
snvResult[i][base - 1 - j] = minLikelihoodProbRate;
}
}
}
Original file line number Diff line number Diff line change
@@ -104,7 +104,7 @@ public void onTraversalStart() {
utilArgs = new FlowFeatureMapperUtils.Args();
utilArgs.header = getHeaderForReads();
utilArgs.fbArgs = fbargs;
utilArgs.limitScore = aqArgs.limitScore;
utilArgs.limitScore = !Double.isNaN(aqArgs.limitPhoreScore) ? (aqArgs.limitPhoreScore / 10.0) : Double.NaN;
utilArgs.debugNegatives = aqArgs.debugNegatives;
utilArgs.keepNegatives = aqArgs.keepNegatives;
utilArgs.negativeScoreOverride = aqArgs.negativeScoreOverride;
@@ -156,7 +156,7 @@ public void apply(final GATKRead read, final ReferenceContext referenceContext,
collectInputStats(read);

// add flow SNV
addFlowSNVQuality.addBaseQuality(read, getHeaderForReads());
addFlowSNVQuality.addBaseQuality(read, getHeaderForReads(), aqArgs.limitPhoreScore);

// if more complex, go do it the hard way
if ( aqArgs.model != null || aqArgs.conf != null || !read.hasAttribute(attrNameForNonCalledBase('A')) ) {
Original file line number Diff line number Diff line change
@@ -16,8 +16,8 @@ public class ApplySNVQRArgumentCollection implements Serializable{
/**
* maximum value for delta in score
**/
@Argument(fullName = "limit-score", doc = "Limit value for score", optional = true)
public double limitScore = Double.NaN;
@Argument(fullName = "limit-phred-score", doc = "Limit value for phred scores", optional = true)
public double limitPhoreScore = Double.NaN;

/**
* include duplicate read?