plot_input.py: ModuleNotFoundError

[alyssadai]

Getting the below error when I try to run plot_input.py with the concatenated metric matrix:
python ../cobra-nmf/vertex/plot_input.py --input wholebrain_ct-sa-mc-gi.mat --output ct-sa-mc-gi_matrix.png

Traceback (most recent call last):
  File "/scinet/intel/2019u4/intelpython3/lib/python3.6/site-packages/matplotlib/backends/backend_webagg.py", line 27, in <module>
    import tornado
ModuleNotFoundError: No module named 'tornado'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "../cobra-nmf/vertex/plot_input.py", line 16, in <module>
    from matplotlib import pyplot as plt
  File "/scinet/intel/2019u4/intelpython3/lib/python3.6/site-packages/matplotlib/pyplot.py", line 2372, in <module>
    switch_backend(rcParams["backend"])
  File "/scinet/intel/2019u4/intelpython3/lib/python3.6/site-packages/matplotlib/pyplot.py", line 207, in switch_backend
    backend_mod = importlib.import_module(backend_name)
  File "/scinet/intel/2019u4/intelpython3/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/scinet/intel/2019u4/intelpython3/lib/python3.6/site-packages/matplotlib/backends/backend_webagg.py", line 29, in <module>
    raise RuntimeError("The WebAgg backend requires Tornado.")
RuntimeError: The WebAgg backend requires Tornado.

[raihaan]

will check this out. just to confirm @justineziolkowski you didnt run into this?

[justineziolkowski]

No, I haven't seen this error

[raihaan]

i wasnt able to replicate this. could you try again to confirm. make sure you're on the argparse branch

[gdevenyi]

This seems to be due to remote access on Niagara. The default python install doesn't include tornado but matplotlib tries to use it. You can change the backend or install tornado

[alyssadai]

Can confirm that I still get the same error (after module load cobralab/2019b), and that I'm on the argparse branch (see below). Adding mpl.use('tkagg') after line 13 seems to fix the issue - I believe this is changing the backend like Gabe mentioned. I also tested the non-argparse plot_input.py on Niagara, and it gives me the same error (but works fine on the CIC with anaconda/5.1.0-python3, which is where I was previously running the visualization scripts). Not sure if this is helpful, but I use Windows and MobaXterm to access the CIC/Niagara.

daialy@nia-login03:/scratch/m/mchakrav/daialy/PS_Dimensions/NMF$ cd ../cobra-nmf
daialy@nia-login03:/scratch/m/mchakrav/daialy/PS_Dimensions/cobra-nmf$ git status
On branch argparse
Your branch is up to date with 'origin/argparse'.

[raihaan]

@alyssadai whenever you have a moment can you log onto niagara and do

module load cobralab
python
import matplotlib as mpl
mpl.get_backend()

and share output

[alyssadai]

Sure, here it is:

>>> mpl.get_backend()
'WebAgg'

[gdevenyi]

The intelpython that Niagara provides doesn't have Tornado installed, which causes the error.

[raihaan]

ok, ill either switch backend or install Tornado. i cant wrap my head around what allows some users (me, Justine) but not others to run the following without error though

module load cobralab
python
import matplotlib as mpl
mpl.get_backend()
from matplotlib import pyplot as plt

any ideas?

[gdevenyi]

you probably have toronado installed under pip --user: ls ~/.local/lib/python*/site-packages

Edit:
Your example fails with me with the tornado error.

[gdevenyi]

I've got it. Its how you're invoking ssh.

If ssh is invoked with -X

>>> import matplotlib as mpl
>>> mpl.get_backend()
'WebAgg'
>>> from matplotlib import pyplot as plt
Traceback (most recent call last):
  File "/scinet/intel/2019u4/intelpython3/lib/python3.6/site-packages/matplotlib/backends/backend_webagg.py", line 27, in <module>
    import tornado
ModuleNotFoundError: No module named 'tornado'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/scinet/intel/2019u4/intelpython3/lib/python3.6/site-packages/matplotlib/pyplot.py", line 2372, in <module>
    switch_backend(rcParams["backend"])
  File "/scinet/intel/2019u4/intelpython3/lib/python3.6/site-packages/matplotlib/pyplot.py", line 207, in switch_backend
    backend_mod = importlib.import_module(backend_name)
  File "/scinet/intel/2019u4/intelpython3/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/scinet/intel/2019u4/intelpython3/lib/python3.6/site-packages/matplotlib/backends/backend_webagg.py", line 29, in <module>
    raise RuntimeError("The WebAgg backend requires Tornado.")
RuntimeError: The WebAgg backend requires Tornado.

If you don't use -X

$ python
Python 3.6.8 |Intel Corporation| (default, Apr 12 2019, 17:34:58) 
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> import matplotlib as mpl

>>> mpl.get_backend()
'WebAgg'
>>> from matplotlib import pyplot as plt
>>> 

matplotlib is detecting $DISPLAY and initializing the backend differently.

[raihaan]

aha. thanks Gabe! in that case think ill go with a try except block and switch backend if RuntimeError

[alyssadai]

WOW @gdevenyi that is some Gabe the Technowizard-level sleuthing. Just checked and can confirm that I have X11 forwarding turned on for my Niagara session, but not for my CIC session, on MobaXterm.