Many 'RuntimeWarning: invalid value encountered in true_divide' warnings in analysis

[tahorst]

I've noticed there are a lot of warning messages printed during analysis scripts that look like they're coming from taking log(0) of some of the data or dividing by 0.

Several multigen plots:

/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/multigen/charging_molecules.py:151: RuntimeWarning: invalid value encountered in true_divide
  normalized_synthetase_counts = synthetase_counts / initial_synthetase_counts
/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/multigen/charging_molecules.py:151: RuntimeWarning: invalid value encountered in true_divide
  normalized_synthetase_counts = synthetase_counts / initial_synthetase_counts
/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/multigen/charging_molecules.py:185: RuntimeWarning: divide by zero encountered in log2
  plot_ax(uncharged_trna_ax, time, np.log2(normalized_uncharged_trna_counts))
/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/multigen/kineticsFluxComparison.py:100: RuntimeWarning: invalid value encountered in true_divide
  categorization[actualAve / targetAve > threshold] = i + 1
/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/multigen/replication.py:68: RuntimeWarning: divide by zero encountered in true_divide
  criticalMassEquivalents = totalMass / criticalInitiationMass
/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/multigen/replication.py:68: RuntimeWarning: divide by zero encountered in true_divide
  criticalMassEquivalents = totalMass / criticalInitiationMass
/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/multigen/tf_binding.py:27: RuntimeWarning: invalid value encountered in true_divide
  frac_promoters_occupied = bound / promoters
/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/multigen/tf_binding.py:29: RuntimeWarning: invalid value encountered in true_divide
  frac_tf_bound = bound / total_active
/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/multigen/growthAffectingPolymerases.py:143: RuntimeWarning: divide by zero encountered in true_divide
  extraRibosomes = (ribosomeCounts - actualElongations / 21.) / (actualElongations / 21.) * 100
/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/multigen/growthAffectingPolymerases.py:163: RuntimeWarning: invalid value encountered in true_divide
  allocatedElongationRate = aaUsed.sum(axis=1) / allocatedRibosomes * timeStep.asNumber(units.s)
/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/multigen/growthAffectingPolymerases.py:307: RuntimeWarning: invalid value encountered in true_divide
  ax14.plot(time.asNumber(units.min), aaRequested / aaPoolsize)
/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/multigen/growthAffectingPolymerases.py:312: RuntimeWarning: invalid value encountered in true_divide
  ax15.plot(time.asNumber(units.min), aaUsed / aaAllocated)
/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/multigen/ribosomeProduction.py:193: RuntimeWarning: invalid value encountered in true_divide
  hist_rrn16S_init_prob = removeNanReshape(rrn16S_init_prob / total_rna_init)
/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/multigen/ribosomeProduction.py:203: RuntimeWarning: invalid value encountered in true_divide
  hist_rrn23S_init_prob = removeNanReshape(rrn23S_init_prob / total_rna_init)
/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/multigen/ribosomeProduction.py:213: RuntimeWarning: invalid value encountered in true_divide
  hist_rrn5S_init_prob = removeNanReshape(rrn5S_init_prob / total_rna_init)
/scratch/groups/mcovert/jenkins/workspace@2/wholecell/utils/units.py:33: RuntimeWarning: divide by zero encountered in true_divide
  return Unum(unit, self._value / other._value)

And what looks like thousands of lines from allReactionFluxes single:

/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/single/allReactionFluxes.py:50: RuntimeWarning: invalid value encountered in true_divide
  meanNormFlux = runningMeanFlux / np.mean(runningMeanFlux)
/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/single/allReactionFluxes.py:57: RuntimeWarning: divide by zero encountered in log10
  self.subplot(2,2,3).plot(time / 60., np.log10(reactionFlux), label=reactionID, color=idToColor[reactionID])

@1fish2 did you say Theano suppressed all of these warnings when we were using it?

[1fish2]

Yes. Theano suppressed all of these warnings from NumPy. I relied on the PR build to test the analysis scripts but didn't look for warnings.

With so many warnings, I'm tempted to suppress them all in the analysis scripts unless anyone wants to go through and fix them or suppress the warnings more narrowly so we'd see new ones.

[tahorst]

With so many warnings, I'm tempted to suppress them all in the analysis scripts unless anyone wants to go through and fix them or suppress the warnings more narrowly so we'd see new ones.

That sounds reasonable. It could be useful to add a class attribute that let's us turn it on/off for certain scripts and maybe have it off to begin with so we aren't silently suppressing a bunch of these warnings like Theano was doing. Maybe something like _suppress_warnings below:

class Plot(variantAnalysisPlot.VariantAnalysisPlot):
	_suppress_warnings = True  # Could be set to false in base class and checked before calling the do_plot function

	def do_plot(self, inputDir, plotOutDir, plotOutFileName, simDataFile, validationDataFile, metadata):

We could just add the _suppress_warnings line to every existing analysis plot so we don't need to check which ones are actually throwing the errors or just set the default to True and have the template analysis files suggest adding _suppress_warnings = False.

[1fish2]

That's a good tradeoff. I'll try that.