Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated GitHub Actions for Python 3.13, added new deploy Action and fixed mac universal2 build #7

Merged
merged 3 commits into from
Nov 6, 2024

Conversation

Zuzu-Typ
Copy link
Collaborator

@Zuzu-Typ Zuzu-Typ commented Oct 21, 2024

Hi @ranjian0

I've noticed that assimp_py doesn't have a release for Python 3.12 and Python 3.13 yet.
Looking at your profile I assumed you currently don't have a lot of time to work on your projects, so I decided to update the project accordingly myself.

I'll walk you through the changes I have made:

Updated Python to 3.9 - 3.13

  • Listed support for 3.12 and 3.13 in setup.py
  • I've updated the Python versions tested in python-package.yml to include 3.12 and 3.13

Updated workflow actions

  • Updated actions/checkout to v4
  • Updated actions/setup-python to v5

Switched to building with pip install

Running python setup.py install directly is deprecated by now.
You can use the build module or pip install.
Using build builds sdist and a wheel, but it doesn't install it automatically.

pip install build
python -m build
pip install .

I've updated python-package.yml accordingly.

Created a working deploy action

When updating the project I noticed you were in the process of automating the build and deployment of assimp_py.
python-publish-manylinux.yml seemed to be working, though using some deprecated modules and python-publish-winmac.yml seemed to only create a source distribution, presumably because the build didn't run successfully.

Previously you were using RalfG/python-wheels-manylinux-build to build your manylinux wheels, which is superseded by cibuildwheel.

pypa/cibuildwheel is a great tool for wheel building automation.
Using cibuildwheel you no longer need multiple GitHub Actions for different operating systems, thus I created a new deployment action, simply called python-publish.yml.

New Publish Action

The new action might look a bit complicated at first.
I'll walk you through it.

on:
  workflow_dispatch:
  release:
    types: [released]

I've set the trigger to released releases (i.e. the workflow runs when you create a release that isn't a prerelease) and to workflow_dispatch, which means you can run the Action from its respective page in your GitHub repository.
I've added workflow_dispatch for testing purposes. You can keep it or remove it, whichever way you like.

env:
  CIBW_BUILD: cp3*
  CIBW_SKIP: cp36-* cp37-* cp38-*
  CIBW_TEST_REQUIRES: pytest
  CIBW_TEST_COMMAND: pytest {project}/tests

These are environment variables for cibuildwheel.

  1. CIBW_BUILD: cp3* tells cibuildwheel to build any CPython 3.X wheels
  2. CIBW_SKIP: cp36-* cp37-* cp38-* tells cibuildwheel not to build 3.6, 3.7, 3.8 wheels. (I think by now cibuildwheel might no longer build these by default, but I kept them to be sure)
  3. CIBW_TEST_REQUIRES: pytest tells cibuildwheel to install pytest for testing
  4. CIBW_TEST_COMMAND: pytest {project}/tests tells cibuildwheel how to run the test command. If the test fails, the build will stop and nothing gets uploaded. This makes sure you don't upload any broken builds.

Jobs

There are two layers of jobs:

  1. Build wheels
  • build_wheels_windows[AMD64, ARM64]
  • build_wheels_mac (universal2)
  • build_wheels_manylinux[2014, 2_28][x86_64, aarch64, s390x, ppc64le, (i686)]
  • build_wheels_musllinux[x86_64, i686, aarch64, s390x, ppc64le, armv7l]
  • build_sdist
  1. Upload wheels (only runs if all steps of the previous layer were successful)
  • upload_pypi

The first layer creates the wheels for Python 3.9 - 3.13 on windows, mac, manylinux2014, manylinux2_28, musllinux1_2 and a source distribution. The wheels are created for different architectures.
cibuildwheel handles this.
Building the wheels for aarch64, s390x, ppc64le and armv7l takes a while, because those architectures need to be emulated.

It was quite a challenge to get the universal2 builds to happen. I had to slightly adjust CMakeLists.txt to get it to work. I'm not an expert on cmake, so please double check that my changes are okay. Since the builds on mac run and the tests succeed too I think it's fine.

Every build step generates an artifact (zip file) which you can see in the summary section of the Action. These contain the generated wheels. The second layer uses these to collect the wheels.

The second layer only executes if all wheels were successfully built and tested.
Its purpose is to download the artifacts and upload them all to PyPI.
This requires some setup on your end.
I believe authenticating with username and password from GitHub Actions is no longer permitted.
You have to use an API token or a trusted publisher.
A Trusted Publisher means telling PyPI that its safe to accept deployment from a specific CI workflow (such as python-publish.yml). There are no API keys or passwords that can leak.
To set up an API token, log in to PyPI and generate one in the settings.
You also need to have two factor authentication enabled on PyPI. Otherwise you're not allowed to publish new releases.

For the time being I've set up python-publish.yml so that it publishes to the Test PyPI, so you can test the setup before actually publishing a new version.
Use this setup to publish to the production PyPI:

# Uncomment to publish to production PyPI:
password: ${{ secrets.pypi_password }}

# Uncomment to publish to test PyPI (https://test.pypi.org):
# password: ${{ secrets.testpypi_password }}
# repository-url: https://test.pypi.org/legacy/

Feel free to rename the secret to your liking, such as pypi_token.

All of these changes should make it trivial for you to add support for a new Python version or improve the project with minimal effort.

PS.: I kept python-package-manylinux.yml and python-package-winmac.yml for the time being. If everything works as expected, feel free to remove them.
Also, make sure to update the GitHub Actions status badge in the Readme.

+ Added Python versions 3.12 and 3.13
+ Updated actions/checkout to v4
+ Updated actions/setup-python to v5
+ Using python -m build instead of setup.py install
+ Added a new publish action and disabled the old ones.

Updated python-publish.yml name

Fixed action indentation for python-publish.yml

Fixed action indentation for python-publish.yml

Removed windows x86 build (not supported)

Updated python versions in setup.py

Try CMakeLists change for macos

Try CMakeLists change for macos

Test CMAKE_OSX_ARCHITECTURES param

Missed a comma

Try building mac images natively

Fixed macos build images

Try building universal2

Added osx_architectures option to cmakelists.txt

Pass OSX_ARCHITECTURES down to assimp

Moved set_property

Try different target_link_libraries option for mac

Only test arm

Fixed syntax error

Try building on macos12

Test macos12 only

Update python-publish.yml

Update python-publish.yml

Update python-publish.yml

Update CMakeLists.txt

Update CMakeLists.txt

Update CMakeLists.txt

Update python-publish.yml

Update python-publish.yml

Removed test edits

Update python-publish.yml
Using `pip install .` instead of `python -m build`
@ranjian0
Copy link
Owner

ranjian0 commented Nov 3, 2024

Sorry it took me a while to get to this PR. Seems like you put some reasonable effort getting everything updated.

You seem pretty keen on this little project seeing as you have contributed before, let me know if you are interested in being added as a maintainer. I don't anticipate having time to work on this any more.

@Zuzu-Typ
Copy link
Collaborator Author

Zuzu-Typ commented Nov 3, 2024

No worries. I understand that you currently don't have the time to work on this.

Sure, I would be interested to maintain the project.

@Zuzu-Typ
Copy link
Collaborator Author

Zuzu-Typ commented Nov 6, 2024

Hi @ranjian0

Thank you for making me a maintainer of assimp_py.

There is still an issue that I need your help with.
Since PyPI no longer supports uploading new releases with username and password it is required to either set up trusted publishing or an API token.
Unfortunately, I do not have access to the settings of the repository nor the PyPI project, so I would like to ask you to do it.
The easiest would be to create an API token:

  1. Sign in to pypi.org and enable 2FA, if you haven't already.
  2. Click on your profile and navigate to your projects
  3. Click on the "Manage" button for assimp-py
  4. Go to Settings
  5. Generate an API token
  6. Give the token any name and make sure to restrict it to assimp-py
  7. Copy the entire token
  8. Return to this repository and go to settings -> secrets -> actions (https://github.com/ranjian0/assimp_py/settings/secrets/actions)
  9. Create a new repository secret called PYPI_TOKEN and paste the token.
  10. You may remove the old secrets (PYPI_USERNAME and PYPI_PASSWORD) as they are no longer required.

It would be great if you could repeat the process for test.pypi.org so I can test the release before the production release. It's not absolutely necessary though. Make sure to save it as TEST_PYPI_TOKEN.

Thanks a lot for your support!

@ranjian0
Copy link
Owner

ranjian0 commented Nov 6, 2024

OK, I have added both tokens.
Also invited you as a maintainer to the pypi projects

@Zuzu-Typ
Copy link
Collaborator Author

Zuzu-Typ commented Nov 6, 2024

OK, I have added both tokens. Also invited you as a maintainer to the pypi projects

Thanks a ton!

@Zuzu-Typ Zuzu-Typ merged commit b266f5d into ranjian0:master Nov 6, 2024
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants