Segmentation fault in bulk mode with .bam files

[archnakj]

I see existing open issues with no workaround listed. Filing new one in the hopes of at getting a workaround. I have 48 files in all and could quantify one by one but not sure how to collate the results from all of them. Help is much appreciated.

I am very happy to help reproduce the problem and gather whatever information is needed to fix this problem.

Bulk mode was used with aligned files. I do plan to try this with fastq files (instead of already aligned files).

Core is dumped if I use more than one file. Here is how I used it. I tried this while letting it pick the number of thread automatically (no change) and specifying the number of threads as 8 (went a little farther with that).

v 1.9.0

Salmon was installed using the command conda install -c bioconda salmon
Arabidopsis Thaliana reference genome was used - TAIR10_chr_all.fas - from here: https://www.arabidopsis.org/download/index-auto.jsp?dir=%2Fdownload_files%2FGenes%2FTAIR10_genome_release%2FTAIR10_chromosome_files

Alignment was done using STAR.

See command below for options used.

Linux login01 3.10.0-1160.el7.x86_64 #1 SMP Mon Oct 19 16:18:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

6. Output, for success case, one file only

salmon quant -t /rsstu/users/c/cjdohert/NSF_Tomato/AT_RefGenomeFiles/TAIR10_chr_all.fas -l A -a leaf_mock_t6_rep3_S64_R1_001Aligned.out.bam -o salmonquant
Version Info: This is the most recent version of salmon.
# salmon (alignment-based) v1.9.0
# [ program ] => salmon 
# [ command ] => quant 
# [ targets ] => { /rsstu/users/c/cjdohert/NSF_Tomato/AT_RefGenomeFiles/TAIR10_chr_all.fas }
# [ libType ] => { A }
# [ alignments ] => { leaf_mock_t6_rep3_S64_R1_001Aligned.out.bam }
# [ output ] => { salmonquant }
Logs will be written to salmonquant/logs
[2023-01-29 16:02:11.267] [jointLog] [info] setting maxHashResizeThreads to 8
[2023-01-29 16:02:11.267] [jointLog] [info] Fragment incompatibility prior below threshold.  Incompatible fragments will be ignored.
Library format { type:single end, relative orientation:none, strandedness:unstranded }
[2023-01-29 16:02:11.308] [jointLog] [info] numQuantThreads = 4
parseThreads = 4
Checking that provided alignment files have consistent headers . . . done
Populating targets from aln = "leaf_mock_t6_rep3_S64_R1_001Aligned.out.bam", fasta = "/rsstu/users/c/cjdohert/NSF_Tomato/AT_RefGenomeFiles/TAIR10_chr_all.fas" . . .done


processed 0 reads in current round[2023-01-29 16:02:12.216] [jointLog] [info] replaced 186,207 non-ACGT nucleotides with random nucleotides
[2023-01-29 16:02:12.668] [jointLog] [info] Automatically detected most likely library type as U

processed 2000000 reads in current round[2023-01-29 16:02:13.116] [jointLog] [info] 

The alignment group queue pool has been exhausted.  1842 extra fragments were allocated on the heap to saturate the pool.  No new fragments will be allocated


processed 13423395 reads in current round
killing thread 3 . . . done

Freeing memory used by read queue . . . 00000
Joined parsing thread . . . "leaf_mock_t6_rep3_S64_R1_001Aligned.out.bam" 
Closed all files . . . 
Emptied frag queue. . . 
Emptied Alignment Group Pool. . 
Emptied Alignment Group Queue. . . done
[2023-01-29 16:02:59.265] [jointLog] [info] 


Completed first pass through the alignment file.
Total # of mapped reads : 13423395
# of uniquely mapped reads : 13423394
# ambiguously mapped reads : 1



[2023-01-29 16:02:59.271] [jointLog] [info] Computed 169 rich equivalence classes for further processing
[2023-01-29 16:02:59.272] [jointLog] [info] Counted 13,423,395 total reads in the equivalence classes 
[2023-01-29 16:02:59.272] [jointLog] [info] starting optimizer
[2023-01-29 16:02:59.274] [jointLog] [info] Marked 0 weighted equivalence classes as degenerate
[2023-01-29 16:02:59.274] [jointLog] [info] iteration = 0 | max rel diff. = 237.557
[2023-01-29 16:02:59.282] [jointLog] [info] iteration = 100 | max rel diff. = 3.78077e-16
[2023-01-29 16:02:59.282] [jointLog] [info] finished optimizer
[2023-01-29 16:02:59.282] [jointLog] [info] writing output

Output for failure case - 48 files:

salmon quant -t /rsstu/users/c/cjdohert/NSF_Tomato/AT_RefGenomeFiles/TAIR10_chr_all.fas -l A -a $file_list -o ../SalmonQuantFiles
Version Info: This is the most recent version of salmon.
# salmon (alignment-based) v1.9.0
# [ program ] => salmon 
# [ command ] => quant 
# [ targets ] => { /rsstu/users/c/cjdohert/NSF_Tomato/AT_RefGenomeFiles/TAIR10_chr_all.fas }
# [ libType ] => { A }
# [ alignments ] => { apex_infe_t1_rep1_S29_R1_001Aligned.out.bam apex_infe_t1_rep3_S53_R1_001Aligned.out.bam apex_infe_t2_rep1_S30_R1_001Aligned.out.bam apex_infe_t2_rep3_S54_R1_001Aligned.out.bam apex_infe_t3_rep1_S31_R1_001Aligned.out.bam apex_infe_t3_rep3_S55_R1_001Aligned.out.bam apex_infe_t4_rep1_S32_R1_001Aligned.out.bam apex_infe_t4_rep3_S56_R1_001Aligned.out.bam apex_infe_t5_rep1_S33_R1_001Aligned.out.bam apex_infe_t5_rep3_S57_R1_001Aligned.out.bam apex_infe_t6_rep1_S34_R1_001Aligned.out.bam apex_infe_t6_rep3_S58_R1_001Aligned.out.bam apex_mock_t1_rep1_S23_R1_001Aligned.out.bam apex_mock_t1_rep3_S47_R1_001Aligned.out.bam apex_mock_t2_rep1_S24_R1_001Aligned.out.bam apex_mock_t2_rep3_S48_R1_001Aligned.out.bam apex_mock_t3_rep1_S25_R1_001Aligned.out.bam apex_mock_t3_rep3_S49_R1_001Aligned.out.bam apex_mock_t4_rep1_S26_R1_001Aligned.out.bam apex_mock_t4_rep3_S50_R1_001Aligned.out.bam apex_mock_t5_rep1_S27_R1_001Aligned.out.bam apex_mock_t5_rep3_S51_R1_001Aligned.out.bam apex_mock_t6_rep1_S28_R1_001Aligned.out.bam apex_mock_t6_rep3_S52_R1_001Aligned.out.bam leaf_infe_t1_rep1_S41_R1_001Aligned.out.bam leaf_infe_t1_rep3_S65_R1_001Aligned.out.bam leaf_infe_t2_rep1_S42_R1_001Aligned.out.bam leaf_infe_t2_rep3_S66_R1_001Aligned.out.bam leaf_infe_t3_rep1_S43_R1_001Aligned.out.bam leaf_infe_t3_rep3_S67_R1_001Aligned.out.bam leaf_infe_t4_rep1_S44_R1_001Aligned.out.bam leaf_infe_t4_rep3_S68_R1_001Aligned.out.bam leaf_infe_t5_rep1_S45_R1_001Aligned.out.bam leaf_infe_t5_rep3_S69_R1_001Aligned.out.bam leaf_infe_t6_rep1_S46_R1_001Aligned.out.bam leaf_infe_t6_rep3_S70_R1_001Aligned.out.bam leaf_mock_t1_rep1_S35_R1_001Aligned.out.bam leaf_mock_t1_rep3_S59_R1_001Aligned.out.bam leaf_mock_t2_rep1_S36_R1_001Aligned.out.bam leaf_mock_t2_rep3_S60_R1_001Aligned.out.bam leaf_mock_t3_rep1_S37_R1_001Aligned.out.bam leaf_mock_t3_rep3_S61_R1_001Aligned.out.bam leaf_mock_t4_rep1_S38_R1_001Aligned.out.bam leaf_mock_t4_rep3_S62_R1_001Aligned.out.bam leaf_mock_t5_rep1_S39_R1_001Aligned.out.bam leaf_mock_t5_rep3_S63_R1_001Aligned.out.bam leaf_mock_t6_rep1_S40_R1_001Aligned.out.bam leaf_mock_t6_rep3_S64_R1_001Aligned.out.bam }
# [ output ] => { ../SalmonQuantFiles }
Logs will be written to ../SalmonQuantFiles/logs
[2023-01-29 16:06:31.513] [jointLog] [info] setting maxHashResizeThreads to 8
[2023-01-29 16:06:31.513] [jointLog] [info] Fragment incompatibility prior below threshold.  Incompatible fragments will be ignored.
Library format { type:single end, relative orientation:none, strandedness:unstranded }
[2023-01-29 16:06:31.580] [jointLog] [info] numQuantThreads = 4
parseThreads = 4
Checking that provided alignment files have consistent headers . . . done
Populating targets from aln = "apex_infe_t1_rep1_S29_R1_001Aligned.out.bam", fasta = "/rsstu/users/c/cjdohert/NSF_Tomato/AT_RefGenomeFiles/TAIR10_chr_all.fas" . . .done

processed 0 reads in current round[2023-01-29 16:06:34.583] [jointLog] [info] replaced 186,207 non-ACGT nucleotides with random nucleotides
processed 2000000 reads in current round[2023-01-29 16:06:35.068] [jointLog] [info] Automatically detected most likely library type as U

[2023-01-29 16:06:35.443] [jointLog] [info] 

The alignment group queue pool has been exhausted.  1955 extra fragments were allocated on the heap to saturate the pool.  No new fragments will be allocated


processed 26000000 reads in current roundSegmentation fault (core dumped)

Output for failure case - four files

salmon quant -t /rsstu/users/c/cjdohert/NSF_Tomato/AT_RefGenomeFiles/TAIR10_chr_all.fas -l A -a leaf_mock_t6_rep3_S64_R1_001Aligned.out.bam leaf_mock_t6_rep1_S40_R1_001Aligned.out.bam leaf_mock_t5_rep3_S63_R1_001Aligned.out.bam leaf_mock_t5_rep1_S39_R1_001Aligned.out.bam -p 8 -o ../SalmonQuantFiles
Version Info: This is the most recent version of salmon.
# salmon (alignment-based) v1.9.0
# [ program ] => salmon 
# [ command ] => quant 
# [ targets ] => { /rsstu/users/c/cjdohert/NSF_Tomato/AT_RefGenomeFiles/TAIR10_chr_all.fas }
# [ libType ] => { A }
# [ alignments ] => { leaf_mock_t6_rep3_S64_R1_001Aligned.out.bam leaf_mock_t6_rep1_S40_R1_001Aligned.out.bam leaf_mock_t5_rep3_S63_R1_001Aligned.out.bam leaf_mock_t5_rep1_S39_R1_001Aligned.out.bam }
# [ threads ] => { 8 }
# [ output ] => { ../SalmonQuantFiles }
Logs will be written to ../SalmonQuantFiles/logs
[2023-01-29 16:52:41.666] [jointLog] [info] setting maxHashResizeThreads to 8
[2023-01-29 16:52:41.666] [jointLog] [info] Fragment incompatibility prior below threshold.  Incompatible fragments will be ignored.
Library format { type:single end, relative orientation:none, strandedness:unstranded }
[2023-01-29 16:52:41.668] [jointLog] [info] numQuantThreads = 4
parseThreads = 4
Checking that provided alignment files have consistent headers . . . done
Populating targets from aln = "leaf_mock_t6_rep3_S64_R1_001Aligned.out.bam", fasta = "/rsstu/users/c/cjdohert/NSF_Tomato/AT_RefGenomeFiles/TAIR10_chr_all.fas" . . .done

processed 0 reads in current round[2023-01-29 16:52:42.565] [jointLog] [info] replaced 186,207 non-ACGT nucleotides with random nucleotides
processed 2000000 reads in current round[2023-01-29 16:52:43.137] [jointLog] [info] Automatically detected most likely library type as U

[2023-01-29 16:52:43.276] [jointLog] [info] 

The alignment group queue pool has been exhausted.  1759 extra fragments were allocated on the heap to saturate the pool.  No new fragments will be allocated


processed 25000000 reads in current roundSegmentation fault (core dumped)