Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish releases to PyPI via a GitHub Action #674

Closed
wants to merge 12 commits into from
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ jobs:

- name: Build Python package
run: |
make build_pypi
make build


docs:
Expand Down
49 changes: 49 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# This github action builds Alibi Detect and publishes it to PyPI.
# It is only run when a new "release" is published via GitHub here:
# https://github.com/SeldonIO/alibi-detect/releases
name: Publish release to PyPI

# Only run when tags starting with "v" are pushed
on:
release:
types: [published]

jobs:
build-and-publish:
name: Build and publish
runs-on: ubuntu-18.04

steps:
# Setup Python
- uses: actions/checkout@master
- name: Set up Python 3.9
uses: actions/setup-python@v1
with:
python-version: 3.9

# Build
- name: Build a binary wheel and a source tarball
run: make build

# Get the PEP440 compliant version number (so we can check version is installable)
- name: Get 📦 version number
id: get_version
run: echo "PACKAGE_VERS=$(python setup.py --version)" >> $GITHUB_ENV

# Publish the associated tag to PyPI
- name: Publish 📦 to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
password: ${{ secrets.PYPI_API_TOKEN }}

# Wait for a while to give PyPI time to update (otherwise the pip install might fail)
- name: Sleep for 10 minutes
shell: bash
run: sleep 10m
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

10 minutes sounds long, usually I can install almost straight away. Not sure though if worth trying programmatically installing (e.g. in a while loop) until it succeeds.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would a while loop not be essentially the same thing though? i.e. wouldn't we have to set some sort of break condition to prevent an infinite loop, so we'd still have to set a time or number of attempts limit?

I've reduced it to 5 mins now. Perhaps we reduce it even more to say 1 or 2 mins, and then increase it if we regularly get failures? An occasional failure would not be the end of the world since the publish still would still have been done, and the user could check manually.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Annoyingly 1 minute wasn't enough!
https://github.com/ascillitoe/alibi-detect/actions/runs/3496674516/jobs/5854858932

I've settled on 3 for now... we can add a fetch a cup of tea 🍵 stage to the instructions?!


# Install from PyPI and test
- name: Install 📦 from PyPI and test
run: |
make clean
python -m pip install alibi-detect==${{ env.PACKAGE_VERS }}
python -c "from alibi_detect.cd import KSDrift"
50 changes: 50 additions & 0 deletions .github/workflows/publish_test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# This github action builds Alibi Detect and publishes it to Test PyPI.
# It is only run when a new tag is pushed to github.
name: Publish release to Test PyPI

# Only run when tags starting with "v" are pushed
on:
push:
tags:
- 'v*'

jobs:
build-and-publish:
name: Build and publish
runs-on: ubuntu-18.04

steps:
# Setup Python
- uses: actions/checkout@master
- name: Set up Python 3.9
uses: actions/setup-python@v1
with:
python-version: 3.9

# Build
- name: Build a binary wheel and a source tarball
run: make build

# Publish to Test PyPI
- name: Publish 📦 to Test PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
password: ${{ secrets.TEST_PYPI_API_TOKEN }}
repository_url: https://test.pypi.org/legacy/

# Get the PEP440 compliant version number (so we can check version is installable)
- name: Get 📦 version number
id: get_version
run: echo "PACKAGE_VERS=$(python setup.py --version)" >> $GITHUB_ENV

# Wait for a while to give PyPI time to update (otherwise the pip install might fail)
- name: Sleep for 10 minutes
shell: bash
run: sleep 10m

# Install from Test PyPI and test
- name: Install 📦 from PyPI and test
run: |
make clean
python -m pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple alibi-detect==${{ env.PACKAGE_VERS }}
python -c "from alibi_detect.cd import KSDrift"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would change this to have imports like import alibi_detect, from alibi_detect.cd import * etc. Also, does it capture any stdout/stderr and print it out? Asking as I didn't see it in the example CI run here: https://github.com/ascillitoe/alibi-detect/actions/runs/3490747918/jobs/5842565978

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re stdout I think this line was stdout:

None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.

In b3aa94e I've changed to the imports you suggested and added print()'s so there is more obvious stdout. CI result here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re stderr I'm not sure a python error would actually fail the GA step. Will look into this. Think we're pretty much there after that!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

27 changes: 14 additions & 13 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,14 @@
install: ## Install package in editable mode with all the dependencies
pip install -e .

.PHONY: clean
clean: # Clean up install
pip uninstall -y alibi-detect
rm -r alibi_detect.egg-info/ dist/ build/

.PHONY: test
test: ## Run all tests
python setup.py test
pytest alibi_detect

.PHONY: lint
lint: ## Check linting according to the flake8 configuration in setup.cfg
Expand All @@ -14,6 +19,14 @@ lint: ## Check linting according to the flake8 configuration in setup.cfg
mypy: ## Run typeckecking according to mypy configuration in setup.cfg
mypy .

.PHONY: build
build: ## Build the Python package (for debugging, building for release is done via a GitHub Action)
# Below commands are a duplicate of those in the publish.yml "Build" step
# (virtualenv added due to Debian issue https://github.com/pypa/build/issues/224)
python -m pip install --upgrade pip virtualenv
python -m pip install build~=0.9
python -m build --sdist --wheel --outdir dist/ .

.PHONY: build_docs
build_docs:
# readthedocs.org build command
Expand All @@ -32,18 +45,6 @@ clean_docs: ## Clean the documentation build
$(MAKE) -C doc clean
rm -r doc/source/api

.PHONY: build_pypi
build_pypi: ## Build the Python package
python setup.py sdist bdist_wheel

.PHONY: push_pypi_test
push_pypi_test: ## Upload the Python package to the test PyPI registry
twine upload --repository-url https://test.pypi.org/legacy/ dist/*

.PHONY: push_pypi
push_pypi: ## Upload the Python package to the PyPI registry
twine upload dist/*

.PHONY: help
help: ## Print out help message on using these commands
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
Expand Down
2 changes: 1 addition & 1 deletion alibi_detect/version.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@
# 2) we can import it in setup.py for the same reason
# 3) we can import it into your module module

__version__ = "0.11.0dev"
__version__ = "0.10.5-alpha5"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: revert before merging

6 changes: 6 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[build-system]
requires = [
"setuptools~=65.5",
"wheel~=0.38",
]
build-backend = "setuptools.build_meta"