npm run virtualenv doesn't activate virtual environment

[miketaylr]

Noticed this when I was running thru the Simple Setup instructions:

https://github.com/webcompat/webcompat.com/blob/master/docs/dev-env-setup.md#simple-setup-mac-os-and-linux

If you run those, everything gets installed as I would expect, and I can see that a virtual env is created, but it never gets activated.

"virtualenv": "pip install virtualenv && virtualenv env && source env/bin/activate || . env/bin/activate && npm run pip",

I'm not really sure how to fix this (or how npm commands should expect to behave when they're chained).

 npm run setup

> webcompat@ setup /private/tmp/webcompat.com
> npm run virtualenv && npm install && npm run config


> webcompat@ virtualenv /private/tmp/webcompat.com
> pip install virtualenv && virtualenv env && source env/bin/activate || . env/bin/activate && npm run pip

Requirement already satisfied: virtualenv in /Users/mitaylor/.venvburrito/lib/python2.7/site-packages
New python executable in /private/tmp/webcompat.com/env/bin/python
Installing setuptools, pip, wheel...done.

> webcompat@ pip /private/tmp/webcompat.com
> pip install -r config/requirements.txt

Collecting Babel==2.3.4 (from -r config/requirements.txt (line 1))
  Using cached Babel-2.3.4-py2.py3-none-any.whl
Collecting blinker==1.4 (from -r config/requirements.txt (line 2))
Collecting Flask==0.12 (from -r config/requirements.txt (line 3))
  Using cached Flask-0.12-py2.py3-none-any.whl
Collecting Flask-Limiter==0.9.3 (from -r config/requirements.txt (line 4))
Collecting Flask-SQLAlchemy==2.1 (from -r config/requirements.txt (line 5))
Collecting Flask-WTF==0.14.2 (from -r config/requirements.txt (line 6))
  Using cached Flask_WTF-0.14.2-py2.py3-none-any.whl
Collecting GitHub-Flask==3.1.5 (from -r config/requirements.txt (line 7))
Collecting mock==2.0.0 (from -r config/requirements.txt (line 8))
  Using cached mock-2.0.0-py2.py3-none-any.whl
Collecting nose==1.3.7 (from -r config/requirements.txt (line 9))
  Using cached nose-1.3.7-py2-none-any.whl
Collecting pep8==1.7.0 (from -r config/requirements.txt (line 10))
  Using cached pep8-1.7.0-py2.py3-none-any.whl
Collecting Pillow==4.0.0 (from -r config/requirements.txt (line 11))
  Using cached Pillow-4.0.0-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Collecting requests==2.13.0 (from -r config/requirements.txt (line 12))
  Using cached requests-2.13.0-py2.py3-none-any.whl
Collecting ua-parser==0.7.2 (from -r config/requirements.txt (line 13))
  Using cached ua_parser-0.7.2-py2.py3-none-any.whl
Collecting WTForms==2.1 (from -r config/requirements.txt (line 14))
Collecting pytz>=0a (from Babel==2.3.4->-r config/requirements.txt (line 1))
  Using cached pytz-2017.3-py2.py3-none-any.whl
Collecting Jinja2>=2.4 (from Flask==0.12->-r config/requirements.txt (line 3))
  Using cached Jinja2-2.10-py2.py3-none-any.whl
Collecting Werkzeug>=0.7 (from Flask==0.12->-r config/requirements.txt (line 3))
  Using cached Werkzeug-0.12.2-py2.py3-none-any.whl
Collecting click>=2.0 (from Flask==0.12->-r config/requirements.txt (line 3))
  Using cached click-6.7-py2.py3-none-any.whl
Collecting itsdangerous>=0.21 (from Flask==0.12->-r config/requirements.txt (line 3))
Collecting limits (from Flask-Limiter==0.9.3->-r config/requirements.txt (line 4))
  Using cached limits-1.2.1-py2-none-any.whl
Requirement already satisfied: six>=1.4.1 in /Users/mitaylor/.venvburrito/lib/python2.7/site-packages (from Flask-Limiter==0.9.3->-r config/requirements.txt (line 4))
Collecting SQLAlchemy>=0.7 (from Flask-SQLAlchemy==2.1->-r config/requirements.txt (line 5))
Requirement already satisfied: pbr>=0.11 in /Users/mitaylor/.venvburrito/lib/python2.7/site-packages (from mock==2.0.0->-r config/requirements.txt (line 8))
Collecting funcsigs>=1; python_version < "3.3" (from mock==2.0.0->-r config/requirements.txt (line 8))
  Using cached funcsigs-1.0.2-py2.py3-none-any.whl
Collecting olefile (from Pillow==4.0.0->-r config/requirements.txt (line 11))
Collecting MarkupSafe>=0.23 (from Jinja2>=2.4->Flask==0.12->-r config/requirements.txt (line 3))
Installing collected packages: pytz, Babel, blinker, MarkupSafe, Jinja2, Werkzeug, click, itsdangerous, Flask, limits, Flask-Limiter, SQLAlchemy, Flask-SQLAlchemy, WTForms, Flask-WTF, requests, GitHub-Flask, funcsigs, mock, nose, pep8, olefile, Pillow, ua-parser
Successfully installed Babel-2.3.4 Flask-0.12 Flask-Limiter-0.9.3 Flask-SQLAlchemy-2.1 Flask-WTF-0.14.2 GitHub-Flask-3.1.5 Jinja2-2.10 MarkupSafe-1.0 Pillow-4.0.0 SQLAlchemy-1.1.15 WTForms-2.1 Werkzeug-0.12.2 blinker-1.4 click-6.7 funcsigs-1.0.2 itsdangerous-0.24 limits-1.2.1 mock-2.0.0 nose-1.3.7 olefile-0.44 pep8-1.7.0 pytz-2017.3 requests-2.13.0 ua-parser-0.7.2
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: Use uuid module instead

> [email protected] install /private/tmp/webcompat.com/node_modules/husky
> node ./bin/install.js

husky
setting up hooks
done


> [email protected] install /private/tmp/webcompat.com/node_modules/intern
> node support/fixdeps.js


> [email protected] postinstall /private/tmp/webcompat.com/node_modules/gifsicle
> node lib/install.js

  ✔ gifsicle pre-build test passed successfully

> [email protected] postinstall /private/tmp/webcompat.com/node_modules/jpegtran-bin
> node lib/install.js

  ✔ jpegtran pre-build test passed successfully

> [email protected] postinstall /private/tmp/webcompat.com/node_modules/optipng-bin
> node lib/install.js

  ✔ optipng pre-build test passed successfully

> [email protected] postinstall /private/tmp/webcompat.com/node_modules/lint-staged
> echo "🚫💩 lint-staged installed! 
Do not forget to configure it. See https://github.com/okonet/lint-staged/blob/master/README.md" && exit 0

🚫💩 lint-staged installed! 
Do not forget to configure it. See https://github.com/okonet/lint-staged/blob/master/README.md
npm notice created a lockfile as package-lock.json. You should commit this file.
added 1111 packages in 19.858s

> webcompat@ config /private/tmp/webcompat.com
> cp config/secrets.py.example config/secrets.py

mitaylor at omg-5 in /tmp/webcompat.com on master
🐓 

Notice how my virtualenv env isn't activated after that command.

@zoepage this might be an issue for the community sprint, the workaround is to manually activate the virtualenv: source env/bin/activate. Otherwise you end up with the following error and it's confusing:

Traceback (most recent call last):
  File "run.py", line 41, in <module>
    raise ImportError('{0}\n\n{1}'.format(e, IMPORT_ERROR))
ImportError: No module named flask_github


==============================================
It seems like you don't have all dependencies.
Please re-run:
    pip install -r config/requirements.txt
==============================================

[miketaylr]

Notice how my virtualenv env isn't activated after that command.

(I'm not sure if this is because I'm using virtualenvburrito -- it would be good if someone could test on their system (which doesn't have virtualenv-burrito installed)):

https://github.com/brainsik/virtualenv-burrito

[karlcow]

the instructions never worked for me either and I don't virtualenv-burrito but I do have virtualenvwrapper

[karlcow]

@miketaylr did you see that line

Requirement already satisfied: virtualenv in /Users/mitaylor/.venvburrito/lib/python2.7/site-packages

I have a macbook air around, that I need to reformat/clean install for Austin soon. It will be too late for the Berlin sprint but it might be worth to test the install instructions from zero.

[miketaylr]

I did some more looking into this. Basically, I can't get the server to run using just npm scripts. I don't think we should use it to run Python stuff that requires an active virtualenv.

npm run virtualenv <-- seems to work?
npm start <-- seems to work, but then barfs because it can't find pip deps

> webcompat@ start /Users/mitaylor/dev/webcompat.com
> source env/bin/activate || . env/bin/activate && python run.py

Traceback (most recent call last):
  File "run.py", line 41, in <module>
    raise ImportError('{0}\n\n{1}'.format(e, IMPORT_ERROR))
ImportError: No module named flask


==============================================
It seems like you don't have all dependencies.
Please re-run:
    pip install -r config/requirements.txt
==============================================

Somehow this is all related to the fact that npm runs these in a new process, and I guess the environment variables that the env/bin/activate script relies on are lost when it exits to the parent shell/process.

Easiest thing to do here would be write some docs explaining that you need to start and activate a virtual enviroment before running any of the npm scripts, then removing all the virtualenv stuff from them. I'm not sure how to solve it otherwise. As-is it just doesn't work.

[karlcow]

@miketaylr Does #2476 fix your issue?

[miketaylr]

@miketaylr Does #2476 fix your issue?

Yep!

In a fresh clone, if i run npm run setup, then npm run start -- everything works as expected. Thanks @karlcow!