Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add license, docs, authors to manifest, exclude spa files #580

Merged
merged 7 commits into from
Sep 17, 2018

Conversation

wholmgren
Copy link
Member

Closes #579. @mikofski does this look ok to you? I wasn't sure about spa_c_files so I left it alone. Or maybe we can merge this and then you can make further updates in your PR if it's important.

@wholmgren wholmgren added this to the 0.6.0 milestone Sep 16, 2018
@mikofski
Copy link
Member

mikofski commented Sep 16, 2018

LGTM!

  1. switched to wholmgren/manifest branch

  2. I created a new python3 virtualenv

  3. installed the requirements

  4. and ran python setup.py sdist

  5. listed the contents of the source distribution

    tar -t -f pvlib-0.6.0a2+6.g5b679c4.dirty.tar.gz > sdist.txt
    

The following files from the manifest are in the source:

  • pvlib-0.6.0a2+6.g5b679c4.dirty/AUTHORS.md

  • pvlib-0.6.0a2+6.g5b679c4.dirty/LICENSE

  • pvlib-0.6.0a2+6.g5b679c4.dirty/README.md

  • pvlib-0.6.0a2+6.g5b679c4.dirty/versioneer.py

  • pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/_version.py <-- see my comment

  • pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/

    pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/12839.tm2
    pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/703165TY.csv
    pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/adr-library-2013-10-01.csv
    pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/BIRD_08_16_2012.csv
    pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/BIRD_08_16_2012_patm.csv
    pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/bishop88_numerical_precision.csv
    pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/detect_clearsky_data.csv
    pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/LinkeTurbidities.h5
    pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/sam-library-cec-inverters-2018-3-18.csv
    pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/sam-library-cec-modules-2017-6-5.csv
    pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/sam-library-sandia-modules-2015-6-30.csv
    pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/variables_style_rules.csv
    
  • spa_c_files/*.h <-- see my comments in review

  • no __pycache__, *.pyc, or *.pyo

  • pvlib-0.6.0a2+6.g5b679c4.dirty/docs/

    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/environment.yml
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/Makefile
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/_images/
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/_images/clonebutton.png
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/_static/
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/_static/.gitignore
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/_static/no_scrollbars.css
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/api.rst
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/clearsky.rst
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/comparison_pvlib_matlab.rst
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/conf.py
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/contributing.rst
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/forecasts.rst
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/index.rst
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/installation.rst
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/introexamples.rst
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/modelchain.rst
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/package_overview.rst
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/pvsystem.rst
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/singlediode.rst
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/timetimezones.rst
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/variables_style_rules.rst
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.1.0.txt
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.2.0.txt
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.2.1.txt
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.2.2.txt
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.3.0.txt
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.3.1.txt
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.3.2.txt
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.3.3.txt
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.4.0.txt
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.4.1.txt
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.4.2.txt
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.4.3.txt
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.4.4.txt
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.4.5.txt
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.5.0.rst
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.5.1.rst
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.5.2.rst
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.6.0.rst
    pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew.rst
    
  • no docs/tutorials, docs/sphinx/build, docs/sphinx/source/generated, or docs/sphinx/source/savefig <-- see my comment about tutorials

  • no Jupyter .ipynb_checkpoints or macOS .DS_store files

here's the full contents of the source distribution:

pvlib-0.6.0a2+6.g5b679c4.dirty/
pvlib-0.6.0a2+6.g5b679c4.dirty/AUTHORS.md
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/environment.yml
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/Makefile
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/_images/
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/_images/clonebutton.png
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/_static/
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/_static/.gitignore
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/_static/no_scrollbars.css
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/api.rst
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/clearsky.rst
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/comparison_pvlib_matlab.rst
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/conf.py
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/contributing.rst
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/forecasts.rst
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/index.rst
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/installation.rst
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/introexamples.rst
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/modelchain.rst
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/package_overview.rst
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/pvsystem.rst
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/singlediode.rst
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/timetimezones.rst
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/variables_style_rules.rst
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.1.0.txt
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.2.0.txt
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.2.1.txt
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.2.2.txt
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.3.0.txt
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.3.1.txt
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.3.2.txt
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.3.3.txt
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.4.0.txt
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.4.1.txt
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.4.2.txt
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.4.3.txt
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.4.4.txt
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.4.5.txt
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.5.0.rst
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.5.1.rst
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.5.2.rst
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew/v0.6.0.rst
pvlib-0.6.0a2+6.g5b679c4.dirty/docs/sphinx/source/whatsnew.rst
pvlib-0.6.0a2+6.g5b679c4.dirty/LICENSE
pvlib-0.6.0a2+6.g5b679c4.dirty/MANIFEST.in
pvlib-0.6.0a2+6.g5b679c4.dirty/PKG-INFO
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/__init__.py
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/_deprecation.py
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/_version.py
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/atmosphere.py
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/clearsky.py
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/12839.tm2
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/703165TY.csv
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/adr-library-2013-10-01.csv
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/BIRD_08_16_2012.csv
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/BIRD_08_16_2012_patm.csv
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/bishop88_numerical_precision.csv
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/detect_clearsky_data.csv
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/LinkeTurbidities.h5
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/sam-library-cec-inverters-2018-3-18.csv
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/sam-library-cec-modules-2017-6-5.csv
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/sam-library-sandia-modules-2015-6-30.csv
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/data/variables_style_rules.csv
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/forecast.py
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/irradiance.py
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/location.py
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/modelchain.py
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/pvsystem.py
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/singlediode.py
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/solarposition.py
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/spa.py
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/tmy.py
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/tools.py
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/tracking.py
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib/version.py
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib.egg-info/
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib.egg-info/dependency_links.txt
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib.egg-info/not-zip-safe
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib.egg-info/PKG-INFO
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib.egg-info/requires.txt
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib.egg-info/SOURCES.txt
pvlib-0.6.0a2+6.g5b679c4.dirty/pvlib.egg-info/top_level.txt
pvlib-0.6.0a2+6.g5b679c4.dirty/README.md
pvlib-0.6.0a2+6.g5b679c4.dirty/setup.cfg
pvlib-0.6.0a2+6.g5b679c4.dirty/setup.py
pvlib-0.6.0a2+6.g5b679c4.dirty/versioneer.py

Copy link
Member

@mikofski mikofski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason for including the SPA header files? IMO based on their license agreement, pvlib should not redistribute any of their files

MANIFEST.in Outdated
recursive-include pvlib/data *
include README.md

include pvlib/spa_c_files/*.h
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this should be removed, b/c I don't think there are any header files are in spa_c_files that should be distributed, and this might accidentally cause the NREL SPA C-files to be distributed without their consent.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. Do we need to replace it with anything else so that the subpackage is properly distributed (minus the NREL files)?

For the record... the only way this would cause the NREL files to be distributed is if the files were manually added to the pvlib/spa_c_files directory before making a release. I usually make a fresh clone of pvlib/pvlib-python before making a release so that things like this don't happen. It's possible that the files are in a few releases in the 0.2-0.3 era, but I'd be surprised if they are in any of the 0.4-0.5 releases.

MANIFEST.in Outdated
include AUTHORS.md
include LICENSE
include README.md

include versioneer.py
include pvlib/_version.py
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK, this line is not necessary, since pvlib is a package, and is already listed in setup.py

Copy link
Member Author

@wholmgren wholmgren Sep 16, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, but I believe this was part of the versioneer setup instructions. Other projects include this line as well. So I recommend we leave it for now.

MANIFEST.in Outdated
recursive-exclude * *.py[co]

recursive-include docs *
prune docs/tutorials
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just curious, why prune the Jupyter notebook tutorials? IMO these could be run locally more interactively, than viewing them in nbviewer or GitHub. IMO pvlib should include these in the sdist.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, better to include them.

@wholmgren
Copy link
Member Author

wholmgren commented Sep 16, 2018

I think the MANIFEST now properly handles the spa_c_files directory. With the NREL spa.c/h files present in the spa_c_files directory...

$ ls -1a pvlib/spa_c_files
.
..
README.md
SPA_NOTICE.md
__init__.py
__init__.pyc
__pycache__
build
cspa_py.pxd
setup.py
spa.c
spa.h
spa_py.cpython-36m-darwin.so
spa_py.pyx
spa_py_example.py

$ python setup.py sdist

$ ls -1a dist/pvlib-0.6.0a2+10.g01e8c3e/pvlib/spa_c_files
.
..
README.md
SPA_NOTICE.md
__init__.py
cspa_py.pxd
setup.py
spa_py.pyx
spa_py_example.py

@mikofski
Copy link
Member

mikofski commented Sep 16, 2018 via email

@wholmgren
Copy link
Member Author

wholmgren commented Sep 16, 2018 via email

@mikofski
Copy link
Member

Oh ok sorry, my bad, either way. I'm not at a computer so I can't test

@wholmgren
Copy link
Member Author

Reworked the manifest to use graft. The behavior now is to include everything under pvlib/ except for the specifically excluded items. This follows the prevailing attitude online of "better to include too much than not enough."

Contents of distributed spa_c_files is now:

README.md
SPA_NOTICE.md
__init__.py
cspa_py.pxd
setup.py
spa_py.pyx
spa_py_example.py

@mikofski
Copy link
Member

I like this a lot, I just tested, and it excludes a lot more cruft than previously (.git*, etc.)

maybe also global-exclude *.egg-info

@wholmgren
Copy link
Member Author

Thanks @mikofski. The global-exclude *.egg-info and variants doesn't seem to work for some reason. Did it work for you?

I see that I've accidentally been including eggs in some of the previous source distributions.

@mikofski
Copy link
Member

something weird, dunno if/how you want to handle - I had put spa.h and spa.c in a subfolder of spa_c_files, and since they weren't explicitly excluded, they ended up in the tarball.

adding this seems to work:

global-exclude */spa.c
global-exclude */spa.h

@mikofski
Copy link
Member

Thanks @mikofski. The global-exclude *.egg-info and variants doesn't seem to work for some reason. Did it work for you?
I see that I've accidentally been including eggs in some of the previous source distributions.

no, didn't work for me either, frustrating

@wholmgren
Copy link
Member Author

Good idea to make a more flexible spa exclusion.

I can't find much guidance online for how to exclude the egg. I'd say better to look the other way if it's not easy to exclude. Note that the egg is included in the v0.6.0a2 release that TravisCI pushed to PyPI last week.

Copy link
Member

@mikofski mikofski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks Will!

@mikofski
Copy link
Member

SO Q/A say:

This directory is created intentionally as part of the build process for a source distribution. A little gander at the developer guide for setuptools gives you a hint as to why

But, be sure to ignore any part of the distutils documentation that deals with MANIFEST or how it's generated from MANIFEST.in; setuptools shields you from these issues and doesn't work the same way in any case. Unlike the distutils, setuptools regenerates the source distribution manifest file every time you build a source distribution, and it builds it inside the project's .egg-info directory, out of the way of your main project directory. You therefore need not worry about whether it is up-to-date or not.

None of the workarounds seem to work for me, I agree with your conclusion, AFAICT this folder was meant to be included in any distribution.

@wholmgren wholmgren changed the title add license, docs, authors to manifest add license, docs, authors to manifest, exclude spa files Sep 17, 2018
@wholmgren wholmgren merged commit cbd7800 into pvlib:master Sep 17, 2018
@wholmgren wholmgren deleted the manifest branch September 17, 2018 17:45
@mikofski
Copy link
Member

rabbit hole that egg-info, glad you moved on 🐰

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants