Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove nan/inf values for pearsonr #1209

Merged
merged 1 commit into from
Oct 28, 2021
Merged

Remove nan/inf values for pearsonr #1209

merged 1 commit into from
Oct 28, 2021

Conversation

tahorst
Copy link
Member

@tahorst tahorst commented Oct 27, 2021

This handles inf/nan values that might have been passed to the pearsonr function to prevent errors like:

Traceback (most recent call last):
  File "/scratch/groups/mcovert/jenkins/workspace@2/wholecell/fireworks/firetasks/analysisBase.py", line 222, in run_plot
    plot_class.main(*args, cpus=1)
  File "/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/analysisPlot.py", line 164, in main
    instance.plot(inputDir, plotOutDir, plotOutFileName, simDataFile,
  File "/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/analysisPlot.py", line 155, in plot
    do_plot()
  File "/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/analysisPlot.py", line 142, in do_plot
    self.do_plot(inputDir, plotOutDir, plotOutFileName, simDataFile,
  File "/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/variant/growth_correlations.py", line 150, in do_plot
    self.plot_bar(plt.subplot(gs[row, col + 1]), property[removed_mask, :], all_growth, label, aa_ids[removed_mask])
  File "/scratch/groups/mcovert/jenkins/workspace@2/models/ecoli/analysis/variant/growth_correlations.py", line 40, in plot_bar
    all_r[i, j] = stats.pearsonr(property[:len(growth)], growth)[0]
  File "/home/groups/mcovert/pyenv/versions/wcEcoli3/lib/python3.8/site-packages/scipy/stats/stats.py", line 3867, in pearsonr
    normxm = linalg.norm(xm)
  File "/home/groups/mcovert/pyenv/versions/wcEcoli3/lib/python3.8/site-packages/scipy/linalg/misc.py", line 140, in norm
    a = np.asarray_chkfinite(a)
  File "/home/groups/mcovert/pyenv/versions/wcEcoli3/lib/python3.8/site-packages/numpy/lib/function_base.py", line 485, in asarray_chkfinite
    raise ValueError(
ValueError: array must not contain infs or NaNs

Copy link
Contributor

@1fish2 1fish2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Is it worth making a filtered-stats.pearsonr subroutine that other analysis classes might call in the future?

@tahorst
Copy link
Member Author

tahorst commented Oct 27, 2021

Is it worth making a filtered-stats.pearsonr subroutine that other analysis classes might call in the future?

Possibly. I think I've run into this issue in other analysis scripts. I was surprised the pearsonr function doesn't have an arg to handle this when I looked at the docs.

@tahorst tahorst merged commit d591eeb into master Oct 28, 2021
@tahorst tahorst deleted the fix-pearsonr branch October 28, 2021 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants