Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch CI build to use Python 3.7 #494

Closed
desilinguist opened this issue Jan 12, 2021 · 3 comments
Closed

Switch CI build to use Python 3.7 #494

desilinguist opened this issue Jan 12, 2021 · 3 comments

Comments

@desilinguist
Copy link
Member

desilinguist commented Jan 12, 2021

xlrd has dropped support for XLSX files starting with v2.0. Therefore, when using an XLSX file as input, the current RSMTool package will raise the following error:

Traceback (most recent call last):
  File "/private/scratch/nmadnani/rsmtool/rsmdev/bin/rsmtool", line 33, in <module>
    sys.exit(load_entry_point('rsmtool', 'console_scripts', 'rsmtool')())
  File "/private/scratch/nmadnani/rsmtool/rsmtool/rsmtool.py", line 372, in main
    run_experiment(abspath(args.config_file),
  File "/private/scratch/nmadnani/rsmtool/rsmtool/rsmtool.py", line 145, in run_experiment
    data_container = reader.read()
  File "/private/scratch/nmadnani/rsmtool/rsmtool/reader.py", line 370, in read
    dataframe = self.read_from_file(set_path, converter, **kwargs)
  File "/private/scratch/nmadnani/rsmtool/rsmtool/reader.py", line 263, in read_from_file
    df = do_read(filename, **kwargs)
  File "/private/scratch/nmadnani/rsmtool/rsmdev/lib/python3.8/site-packages/pandas/util/_decorators.py", line 296, in wrapper
    return func(*args, **kwargs)
  File "/private/scratch/nmadnani/rsmtool/rsmdev/lib/python3.8/site-packages/pandas/io/excel/_base.py", line 304, in read_excel         sh %s'
    io = ExcelFile(io, engine=engine)
  File "/private/scratch/nmadnani/rsmtool/rsmdev/lib/python3.8/site-packages/pandas/io/excel/_base.py", line 867, in __init__
    self._reader = self._engines[engine](self._io)
  File "/private/scratch/nmadnani/rsmtool/rsmdev/lib/python3.8/site-packages/pandas/io/excel/_xlrd.py", line 22, in __init__
    super().__init__(filepath_or_buffer)
  File "/private/scratch/nmadnani/rsmtool/rsmdev/lib/python3.8/site-packages/pandas/io/excel/_base.py", line 353, in __init__
    self.book = self.load_workbook(filepath_or_buffer)
  File "/private/scratch/nmadnani/rsmtool/rsmdev/lib/python3.8/site-packages/pandas/io/excel/_xlrd.py", line 37, in load_workbook
    return open_workbook(filepath_or_buffer)
  File "/private/scratch/nmadnani/rsmtool/rsmdev/lib/python3.8/site-packages/xlrd/__init__.py", line 170, in open_workbook
    raise XLRDError(FILE_FORMAT_DESCRIPTIONS[file_format]+'; not supported')
xlrd.biffh.XLRDError: Excel xlsx file; not supported
@desilinguist
Copy link
Member Author

It looks like pandas.read_excel() should use openpyxl automatically if it is installed and needed.

@desilinguist
Copy link
Member Author

desilinguist commented Jan 12, 2021

The reason that the Travis CI build failed is because conda-forge currently has no conda package for pandas=1.2.0 -- the version that automatically uses openpyxl for XLSX files instead of xlrd -- for Python 3.6.

$> conda search -c conda-forge pandas=1.2.0
Loading channels: done
# Name                       Version           Build  Channel             
pandas                         1.2.0  py37hdc94413_0  conda-forge         
pandas                         1.2.0  py38h51da96c_0  conda-forge         
pandas                         1.2.0  py39hde0f152_0  conda-forge

This is because pandas 1.2.0 explicitly dropped support for Python 3.6.

It might be time to do the same for RSMTool going forward. For now, the workaround for Python 3.6 is to install the older version of xlrd:

conda install xlrd=1.2.0

@desilinguist desilinguist changed the title Switch to openpyxl to read XLSX files Switch Travis CI build to use Python 3.7 Jan 12, 2021
@desilinguist desilinguist pinned this issue Jan 13, 2021
@desilinguist desilinguist changed the title Switch Travis CI build to use Python 3.7 Switch CI build to use Python 3.7 Jan 13, 2021
@desilinguist
Copy link
Member Author

Done in #498 and #499.

@desilinguist desilinguist unpinned this issue Feb 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant