This repository exists to show a working example of Python formatting, linting and import sorting configurations with continuous integration.
This is a very simple, single Python module. It is not a Python package and does not contain the structure or tooling required for packaging / installing a Python project.
Formatting is handled by black.
Black is an uncompromising Python code formatting tool. It takes a Python file as an input, and provides a reformatted Python file as an output, using rules that are a strict subset of PEP 8. It offers very little in the way of configuration (line length being the main exception) in order to achieve formatting consistency. It is deterministic - it will always produce the same output from the same inputs.
The line length configuration is stored in pyproject.toml.
Configuring Black to automatically run every time code is modified allows developers to forget about how code should be formatted and stay focused on functionality. Many IDEs and text editors allow this. Git hooks can also be used to execute Black before Python code is committed to a repository - this is not configured here as use of autosave and continuous integration checks makes it redundant.
In VSCode, you can set the default Python formatter using the setting: "python.formatting.provider": "black", and then turn on autosave using "editor.formatOnSave": true. VSCode will pick up the line length configuration from a pyproject.toml file per repository.
Linting is handled by pylint.
Pylint checks Python files in order to detect syntax errors and potential bugs (unreachable code / unused variables), provide refactoring help,
The configuration is stored in pyproject.toml.
Pylint is enabled by default in VSCode. VSCode will respect Pylint configurations stored in a pyproject.toml file per repository.
Import sorting is handled by isort.
isort sorts Python imports alphabetically within their respective sections:
- Standard library imports
- Related third party imports
- Local application / library specific imports
isort has many configuration options but these can cause inconsistencies with Black, so must be carefully assessed. The configurations within this repository will provide consistently ordered / formatted imports.
The configuration is stored in pyproject.toml.
It is possible to configure VSCode to run isort every time you save, using the codeActionsOnSave": { "source.organizeImports": true } setting. This currently causes a race condition when formatting by black is also configured to occur on save ("editor.formatOnSave": true), leading to lines being erroneously deleted. The VSCode Python team can't do anything about this.
The configuration stored here works with black - that is to say, black will not reformat changes made by isort - but it is still unsafe to apply both commands simultaneously.
In VSCode, you can manually trigger import sorting by:
- right clicking anywhere within a file and selecting
Sort Imports. - command palette (
Ctrl + Shift + P), thenPython Refactor: Sort Imports. - creating a shortcut key for
python.sortImports
There is also some discussion over whether black should just handle import sorting itself.
Line length needs to be consistently configured for formatting, linting and while sorting imports. In order to change the line length for a specific project, it will need to be updated in pyproject.toml.
Commit messages must conform to conventional commits:
type(optional-scope): description
where type is one of build, chore, ci, docs, feat, fix, perf, refactor, revert, style, test as specified in .gitlint.
For example:
ci(github-actions): add new matrix jobs for Python 3.8
The GitHub Actions workflow is stored at .github/workflows/push.yml.
This configuration contains two jobs, with each containing a build matrix with two different versions of Python. The second job depends on the first job being completed successfully by using the needs key within the second job.
The first job, linter, tests formatting, linting and correct sorting of imports.
gitlintThis command runs gitlint on the most recent commit, to check that commit message titles conform to the conventional commits specification. Earlier commits may not conform, but all commits that are included in a Pull Request are checked by the next command. Pull Requests are the recommended code review workflow before merging into master.
gitlint --commits origin/${{ github.base_ref }}..${{ github.event.pull_request.head.sha }}This command runs gitlint over all commits in a Pull Request, to check that commit message titles conform to the conventional commits specification.
black . --check --diffThis command runs the black formatter over the Python files in the current directory. The --check flag ensures that the command will error out if any changes are required, rather than making changes. --diff will show the difference.
black will respect the line length configuration stored in pyproject.toml.
pylint salutation.py test_salutation.pyThis command runs the pylint checks for errors, bugs, non-conforming code style and potential code structure improvements over our Python module and tests.
pylint uses the configurations stored in pyproject.toml.
isort -rc . --check --diffThis command runs the isort import sorter recursively over the Python files in the current directory. The --check flag ensures that the command will error out if any changes are required, rather than making changes. --diff will show the difference.
isort will respect the configurations stored in pyproject.toml.
The second job, test, runs tests using pytest and verifies test coverage.
pytestSimply execute pytest and it will find things that look like tests and run them.
Tests run in a random order thanks to pytest-randomly. This helps to detect interdependencies between tests. A test sequence can be reproduced by using the same --randomly-seed=VALUE option. pytest will always print this value, for example:
Using --randomly-seed=1548714831
This randomness can also be used to generate fuzzing input.
Kodiak is enabled on this repository, configured by .kodiak.toml.
Kodiak is a GitHub Bot that automatically merges pull requests that have been labelled automerge :rocket: once CI and required approvals have passed. Kodiak is free for open source repositories, but requires a per user subscription for private repositories.
Dependabot is enabled on this repository, configured by .github/dependabot.yml.
Dependabot checks the dependencies listed in requirements-dev.txt daily to check if any new versions of those dependencies have been released. If Dependabot finds a new release, it opens a new Pull Request with a version bump, and attempts to show the changelog for that change. It also provides a compatibility prediction based on all of the Pull Requests that it has opened on other open source repositories and whether continous integration was successful.
Dependabot is owned by GitHub and free for both open source and private repositories.
LGTM is enabled on this repository. There is no special configuration here, and enabling LGTM occurs at the GitHub organisation level.
LGTM provides automated code review and security analysis.
LGTM is owned by GitHub and free for open source repositories. It can only be enabled on private repositories by building your own instance of LGTM.
Prerequisites:
- Python 3.7 through 3.11 (this can be changed in
.python-versionandpyproject.toml) - Poetry
Optional dependencies:
- Pyenv to use the reference Python version in
pyproject.tomlwith a simplepyenv install
Install the project dependencies:
poetry installInstall commit-msg git hook. It runs on every local commit to check if the commit message conforms to the convention specified in .gitlint
pre-commit install --hook-type commit-msg --overwrite
pre-commit install --hook-type=pre-commit --overwrite