‘make lint’ is broken under python2

[TaoLv]

Description

https://github.com/dmlc/dmlc-core/blob/master/scripts/lint.py was changed to python3 by dmlc/dmlc-core#396

So if one is using python2 and pylint/cpplint are not presented in his python3 environment, make lint will fail.

Environment info (Required)

What to do:
1. Download the diagnosis script from https://raw.githubusercontent.com/apache/incubator-mxnet/master/tools/diagnose.py
2. Run the script using `python diagnose.py` and paste its output here.

Package used (Python/R/Scala/Julia):
python

For Scala user, please provide:

1. Java version: (java -version)
2. Maven version: (mvn -version)
3. Scala runtime if applicable: (scala -version)

For R user, please provide R sessionInfo():

Build info (Required if built from source)

MXNet commit hash:
latest master branch

Build config:
make lint

Error Message:

Makefile:166: "USE_LAPACK disabled because libraries were not found"
Makefile:264: WARNING: Significant performance increases can be achieved by installing and enabling gperftools or jemalloc development packages
3rdparty/dmlc-core/scripts/lint.py mxnet cpp include src plugin cpp-package tests \
--exclude_path src/operator/contrib/ctc_include
Traceback (most recent call last):
  File "3rdparty/dmlc-core/scripts/lint.py", line 13, in <module>
    import cpplint
ImportError: No module named 'cpplint'
make: *** [cpplint] Error 1

Minimum reproducible example

(If you are using your own code, please provide a short script that reproduces the error. Otherwise, please provide link to the existing example.)

Steps to reproduce

(Paste the commands you ran that produced the error.)

What have you tried to solve it?

[kalyc]

@TaoLv thanks for submitting this issue.
@sandeep-krishnamurthy could you add label "Bug" & "Python" to this issue?

[apeforest]

@TaoLv I am looking into this issue. Could you please update the "Steps to reproduce". The error I got when running "make lint" is below:

Traceback (most recent call last):
  File "scripts/lint.py", line 13, in <module>
    import cpplint
ModuleNotFoundError: No module named 'cpplint'
make: *** [lint] Error 1

Sorry if my question sounds too silly, I am still new to this community. Thanks!

[TaoLv]

@apeforest Thank you for asking that. That's the exact error I got. I have no idea why your traceback didn't give a full path of scripts/lint.py file.

[apeforest]

@TaoLv Thanks for your reply. I could reproduce the error you reported now. I will start working on the fix.

[apeforest]

@TaoLv The fix is actually in dmlc_core repo. I have opened a PR there: dmlc/dmlc-core#411. Maybe we should link this issue with the PR?

[TaoLv]

@apeforest Thanks for your effort. Please feel free to do that.

[TaoLv]

@apeforest The problem is still there for me. Seems it happens in dmlc-core, so why you change mshadow?

[apeforest]

@TaoLv please ignore my previous comment. That fix was for CI only. Sorry for the confusion.

Nonetheless, I still don't seem to have the issue using python2 in my MXNet virtual env.

Here is my virtual env package list:

certifi==2018.4.16
chardet==3.0.4
coremltools==0.8
graphviz==0.8.4
idna==2.6
-e git+https://github.com/apeforest/incubator-mxnet.git@da6b714e20f22800c5f632c7e32984b25436e3cb#egg=mxnet&subdirectory=python
nose==1.3.7
numpy==1.14.5
protobuf==3.6.0
PyYAML==3.13
requests==2.18.4
six==1.10.0
urllib3==1.22

And if I run make lint at the root directory of mxnet, it just runs smoothly. Could you please let me know the packages in your virtualenv?

[TaoLv]

I guess you also have a py3 environment which has cpplint and pylint installed there. Could you try to uninstall cpplint and pylint in py3 firstly before run make lint in py2?

pip3 uninstall cpplint
pip3 uninstall pylint
make lint

[apeforest]

@TaoLv I could reproduce the error following your comments. I have fixed it and verified in my local environment. PR is dmlc/dmlc-core#433

[apeforest]

@TaoLv PR has been merged. Please pull the latest dmlc-core module and verify. Thanks.

[TaoLv]

Thanks, @apeforest . I would like to close this issue after dmlc-core is updated to mxnet master branch.

[vandanavk]

dmlc/dmlc-core#433 fixes the issue. Will mxnet's dmlc-core be updated any time soon?

[lupesko]

@Roshrini since you are working on the release of MXNet 1.3 - are you planning to pull in the updated dmlc-core into the new release?

[Roshrini]

Yes I will be updating dmlc-core in the new release.

[vandanavk]

@TaoLv dmlc-core has been pulled in. The fix is in incubator-mxnet now.

@sandeep-krishnamurthy Could you close this issue?

[TaoLv]

Thank you all for fixing this issue. @sandeep-krishnamurthy feel free to close.