Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use poetry to manage Python packaging #167

Merged
merged 7 commits into from
Jan 17, 2022
Merged

Use poetry to manage Python packaging #167

merged 7 commits into from
Jan 17, 2022

Conversation

bostonrwalker
Copy link

Use poetry to manage building Python extension instead of shell script

cd pkg/pypi
poetry build

@bostonrwalker bostonrwalker changed the title Feature poetry Use poetry to manage Python packing Jan 11, 2022
@bostonrwalker bostonrwalker changed the title Use poetry to manage Python packing Use poetry to manage Python packaging Jan 11, 2022
@bostonrwalker
Rename zt_wrap.cxx to zt_wrap.cpp
601032c 
Reason: Python distutils invocation of SWIG will output "zt_wrap.cpp" instead of "zt_wrap.cxx", and if both files are present the compiler will get confused.
@bostonrwalker
Copy link
Author

This PR is not dependent on #161, but #161 really should be merged first to avoid issues with names of generated files.

@joseph-henry
Copy link
Contributor

Thanks. I've tested all of your PRs and everything seems to work and I can build the wheel with poetry. This is excellent work! Before I merge everything into dev I still need to figure out how to tie in poetry with the current use of cbuildwheel in .github/workflows/wheels.yml. Let me know if you have any thoughts on this or if there is another tool you think would be better.

@bostonrwalker
Copy link
Author

Looks like there are actions available for GitHub workflows that install poetry (e.g. https://github.com/marketplace/actions/setup-poetry). Then you can run poetry env use 3.x where x is the version of Python you want to build for and poetry build to build the wheel.

I played around with this and I was able to build wheels this way for multiple versions of Python. A caveat is there seems to be a bug in the latest version of poetry that causes the name of the output wheel to always reflect the same version of Python (i.e. the wheel I got was always called libzt-1.8.4-cp310..., even when I built for 3.9, 3.8, etc.). The wheel is still built against the correct version of Python so this is fixed by just renaming the file.

@bostonrwalker
Copy link
Author

Reference: python-poetry/poetry#3509. I wouldn't hold my breath for a resolution as poetry extension module support is still very alpha-stage for some reason even after 3 years.

@bostonrwalker
Copy link
Author

RE: cibuildwheel, it looks like Poetry might already be supported: pypa/cibuildwheel#916. Also, it would be a good idea to add Python 3.10 support now.

@bostonrwalker
Update wheels.yml
6070370 
- Pre-install poetry
- Remove redundant build steps (now handled by build.py)
- Add Python 3.10 to builds
@bostonrwalker
Copy link
Author

I took a stab at retooling wheels.yml. It's not tested though.

@bostonrwalker
Copy link
Author

Note: I just removed support for Python 3.5 since it is not supported by Poetry and it is currently being phased out. I don't think we can migrate to Poetry without making this change.

joseph-henry added a commit that referenced this pull request Jan 17, 2022
@joseph-henry
Resolved conflict for PR #167
f601602
@joseph-henry joseph-henry merged commit ef66042 into zerotier:dev Jan 17, 2022
@bostonrwalker
Copy link
Author

setup.py needs to be removed as well for this one to work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@bostonrwalker @joseph-henry