Migrate project metadata from setup.py to pyproject.toml following PEP621 #1848

seisman · 2022-03-26T15:33:54Z

Description of proposed changes

Reminders

Run make format and make check to make sure the code follows the style guide.
Add tests for new features or tests that would have caught the bug that you're fixing.
Add new public functions/methods/classes to doc/api/index.rst.
Write detailed docstrings for all functions/methods.
If wrapping a new module, open a 'Wrap new GMT module' issue and submit reasonably-sized PRs.
If adding new functionality, add an example to docstrings or tutorials.

Slash Commands

You can write slash commands (/command) in the first line of a comment to perform
specific operations. Supported slash commands are:

/format: automatically format and lint the code
/test-gmt-dev: run full tests on the latest GMT development version

…P621

weiji14

Thanks @seisman! Just some quick comments for now. There seems to be 166 in CI due to missing baseline images for some reason, is there something missing?

pyproject.toml

setup.py

seisman · 2022-03-27T15:56:29Z

pyproject.toml

@@ -1,7 +1,44 @@
 [build-system]
-requires = ["setuptools>=45", "setuptools_scm[toml]>=6.2"]
+requires = ["setuptools>=61", "setuptools_scm[toml]>=6.2"]


Please note that changes in this PR require setuptools>=61.

setuptools v61.0.0 was released on Mar. 24, 2022. This version starts to support storing project metadata in pyproject.toml following PEP621 but it's still experimental and may change in future versions. I think we should not merge this PR unless the pyproject.toml support is stable.

Yes, let's wait and see first. I've had to pin to setuptools=58.4.0 on one of my projects 3 weeks ago because one of the dependencies didn't like setuptools v60.9. There might be some other packages pinning to a lower setuptools version too.

leouieda · 2022-03-28T08:16:10Z

Having recently gone through all of this, there are 2 options that would work well here:

Use setup.cfg and pyproject.toml (see Pooch) so you don't need setup.py and have everything in text format already (though in 2 files). Once setuptools support is stable, move the config to pyproject.toml only. This can take a year or more since setuptools underpins so many projects. So keeping this PR open until then will make it stale.
Migrate from setuptools to flit which already supports pyproject.toml. It's a different built backend and is only compatible with pure Python packages. Since PyGMT doesn't have compiled code, this would work. But it would also make it impossible to distribute libgmt with PyPI wheels in the future (which would be awesome but difficult to achieve).

Either way, getting rid of setup.py means that source and wheel packages have to be built by a different tool. I'm using build which is pretty straight forward (replace python setup.py sdist bdist_wheel with python -m build . ).

pyproject.toml

seisman · 2022-03-29T01:51:36Z

Use setup.cfg and pyproject.toml (see Pooch) so you don't need setup.py and have everything in text format already (though in 2 files). Once setuptools support is stable, move the config to pyproject.toml only. This can take a year or more since setuptools underpins so many projects. So keeping this PR open until then will make it stale.

We are also trying to get rid of the setup.cfg file in #1847. So moving from setup.py to setup.cfg is not an option.

Migrate from setuptools to flit which already supports pyproject.toml. It's a different built backend and is only compatible with pure Python packages. Since PyGMT doesn't have compiled code, this would work. But it would also make it impossible to distribute libgmt with PyPI wheels in the future (which would be awesome but difficult to achieve).

I also noticed flit, but it seems flit doesn't support a dynamic version like what setuptools_scm does (pypa/flit#257).

Either way, getting rid of setup.py means that source and wheel packages have to be built by a different tool. I'm using build which is pretty straight forward (replace python setup.py sdist bdist_wheel with python -m build . ).

Yes, we have switched to build (#1823) a few weeks ago. 🎆

leouieda · 2022-03-29T07:17:43Z

Ah right, if keeping setup.cfg is out of the question then it would a matter of waiting until setuptools support is stable. It underpins so many packages that generating conflicts with it will likely be a nightmare.

pyproject.toml

seisman · 2022-08-12T05:13:24Z

setup.py

@@ -1,59 +1,10 @@
 """


setup.py is no longer required since setuptools 64.0.0 (https://setuptools.pypa.io/en/latest/userguide/development_mode.html)

Cool, it seems like setuptools v64.0.0 enabled non setup.py editable installs!

On the developer side, this seems ok to remove setup.py, but are there any potential side effects for users not having such a new setuptools version? Or to put it another way, do we want to remove setup.py for PyGMT v0.8.0, or wait another minor version (e.g. v0.9.0)?

are there any potential side effects for users not having such a new setuptools version?

I have setuptools 62.6.0 installed. Checking out this branch and running make install and make package work well for me.

When running make package, I see the following messages:

$ make package python -m build --sdist --wheel * Creating venv isolated environment... * Installing packages in isolated environment... (setuptools>=64, setuptools_scm[toml]>=6.2) * Getting dependencies for sdist... /tmp/build-env-ld441dbo/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py:108: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*. warnings.warn(msg, _BetaConfiguration) running egg_info writing pygmt.egg-info/PKG-INFO writing dependency_links to pygmt.egg-info/dependency_links.txt writing requirements to pygmt.egg-info/requires.txt writing top-level names to pygmt.egg-info/top_level.txt reading manifest template 'MANIFEST.in' adding license file 'LICENSE.txt' adding license file 'AUTHORS.md' adding license file 'AUTHORSHIP.md' writing manifest file 'pygmt.egg-info/SOURCES.txt' * Building sdist... /tmp/build-env-ld441dbo/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py:108: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*. warnings.warn(msg, _BetaConfiguration) running sdist running egg_info writing pygmt.egg-info/PKG-INFO writing dependency_links to pygmt.egg-info/dependency_links.txt writing requirements to pygmt.egg-info/requires.txt writing top-level names to pygmt.egg-info/top_level.txt reading manifest template 'MANIFEST.in' adding license file 'LICENSE.txt' adding license file 'AUTHORS.md' adding license file 'AUTHORSHIP.md' writing manifest file 'pygmt.egg-info/SOURCES.txt' running check ...

It seems build dependencies like setuptools>=64 are automatically installed in isolated environments and do not affect users' installations.

A downside here is that make install no longer works without internet access, even if setuptools>=64 is installed in the environment. I'll do some more searching for a workaround but if anyone knows a solution that would be helpful.

I tried disconnecting and used pip install --no-deps --no-build-isolation --editable=. which seems to work offline (presumably it manages to pick up a pre-downloaded setuptools>=64). However, it results in a editable version named pygmt-0.0.0. Maybe the offline feature can be raised upstream to PyPA?

A downside here is that make install no longer works without internet access, even if setuptools>=64 is installed in the environment.

It's unrelated to this PR. Offline make install worked for pygmt<=0.2.0 and started to fail since v0.3.0, in which we switched from versioneer to setuptools_scm (https://www.pygmt.org/dev/changes.html#id47). So it's likely a problem for setuptools_scm.

I tried disconnecting and used pip install --no-deps --no-build-isolation --editable=. which seems to work offline (presumably it manages to pick up a pre-downloaded setuptools>=64). However, it results in a editable version named pygmt-0.0.0.

This command gives the correct version for pygmt <= 0.6.0 but results in pygmt-0.0.0 for pygmt v0.7.0. Not sure if it's caused by changes in #1945.

@maxrjones Perhaps open an issue report for the offline pip issue?

I've opened a PR in #2097.

Thanks for opening the issue @seisman. Apologies for not responding to your question. I had originally interpreted your comment as opening an issue in the PyPA repo and wanted to do more debugging before taking that step.

doc/maintenance.md

seisman · 2022-08-14T07:20:09Z

There seems to be 166 in CI due to missing baseline images for some reason, is there something missing?

That's because make package doesn't include the baseline images.

weiji14

Great work, keeping PyGMT nice and modern 😃 Just some very minor suggestions, otherwise looks good to me.

pyproject.toml

weiji14 · 2022-08-18T12:37:35Z

pyproject.toml

+    "Programming Language :: Python :: 3.10",
+    "License :: OSI Approved :: BSD License",
+]
+dependencies = ["numpy>=1.20", "pandas", "xarray", "netCDF4", "packaging"]


Thinking of whether to declare optional dependencies (https://packaging.python.org/en/latest/specifications/declaring-project-metadata/#dependencies-optional-dependencies). Maybe do in a separate PR.

Motivation would be to enable something like pip install pygmt[all] that includes dependencies like geopandas.

Opened a PR #2069 for it.

Co-authored-by: Wei Ji <[email protected]>

…P621 (GenericMappingTools#1848) Delete setup.py and move all project metadata to pyproject.toml as per https://peps.python.org/pep-0621. * Migrate project metadata from setup.py to pyproject.toml following PEP621 * Add docstrings to setup.py * Use the new email address for pygmt team * Update the link for development mode * Set include-package-data to include baseline and data files * Remove setup.py completely. Requires setuptools>=64.0.0 * Fix the minimum required numpy version to 1.20 * Add package data * Update the command to check README syntax * Run tests in editable mode * Multi-line dependencies Co-authored-by: Wei Ji <[email protected]>

Migrate project metadata from setup.py to pyproject.toml following PE…

0c67c61

…P621

weiji14 reviewed Mar 26, 2022

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

setup.py Show resolved Hide resolved

setup.py Show resolved Hide resolved

setup.py Outdated Show resolved Hide resolved

Add docstrings to setup.py

7cff590

vercel bot temporarily deployed to Preview March 27, 2022 15:44 Inactive

seisman commented Mar 27, 2022

View reviewed changes

seisman commented Mar 29, 2022

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

pyproject.toml Outdated Show resolved Hide resolved

Use the new email address for pygmt team

cc9d8e0

vercel bot temporarily deployed to Preview March 29, 2022 01:42 Inactive

seisman added longterm Long standing issues that need to be resolved maintenance Boring but important stuff for the core devs labels Mar 29, 2022

weiji14 mentioned this pull request Mar 29, 2022

Distribute PyGMT wheels with libgmt included #1853

Open

seisman added 3 commits April 1, 2022 21:22

Update the link for development mode

033d369

Set include-package-data to include baseline and data files

f6bf6f6

Merge branch 'main' into pep621

796acf9

vercel bot temporarily deployed to Preview April 1, 2022 13:25 Inactive

Merge branch 'main' into pep621

f6a0514

vercel bot temporarily deployed to Preview April 29, 2022 03:26 Inactive

weiji14 reviewed Jun 7, 2022

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

seisman mentioned this pull request Jun 8, 2022

Set setuptools_scm fallback_version to follow PEP440 #1945

Merged

Merge branch 'main' into pep621

1155adc

vercel bot temporarily deployed to Preview June 11, 2022 07:19 Inactive

seisman added this to the 0.8.0 milestone Jul 2, 2022

Merge branch 'main' into pep621

f1015f5

vercel bot temporarily deployed to Preview July 7, 2022 06:07 Inactive

seisman commented Aug 12, 2022

View reviewed changes

seisman added 2 commits August 11, 2022 22:31

Merge branch 'main' into pep621

6aa4888

Remove setup.py completely. Requires setuptools>=64.0.0

cfc20a7

seisman force-pushed the pep621 branch from 7053785 to cfc20a7 Compare August 12, 2022 05:38

seisman commented Aug 13, 2022

View reviewed changes

doc/maintenance.md Show resolved Hide resolved

seisman added 4 commits August 13, 2022 23:03

Fix the minimum required numpy version to 1.20

f85015e

Add package data

f257a36

Update the command to check README syntax

7553f0b

Merge branch 'main' into pep621

8fbf603

seisman changed the title ~~WIP: Migrate project metadata from setup.py to pyproject.toml following PEP621~~ Migrate project metadata from setup.py to pyproject.toml following PEP621 Aug 13, 2022

seisman removed the longterm Long standing issues that need to be resolved label Aug 13, 2022

seisman marked this pull request as ready for review August 13, 2022 15:55

seisman marked this pull request as draft August 14, 2022 03:02

seisman added 2 commits August 18, 2022 13:08

Run tests in editable mode

9d9ce07

Merge branch 'main' into pep621

2764315

seisman marked this pull request as ready for review August 18, 2022 07:14

weiji14 approved these changes Aug 18, 2022

View reviewed changes

weiji14 added the final review call This PR requires final review and approval from a second reviewer label Aug 18, 2022

seisman and others added 2 commits August 19, 2022 12:37

Multi-line dependencies

5f54cc5

Co-authored-by: Wei Ji <[email protected]>

Merge branch 'main' into pep621

6ba0fe0

weiji14 removed the final review call This PR requires final review and approval from a second reviewer label Aug 19, 2022

weiji14 merged commit 06fca61 into main Aug 19, 2022

weiji14 deleted the pep621 branch August 19, 2022 15:53

seisman mentioned this pull request Aug 20, 2022

Add optional dependencies to pyproject.toml #2069

Merged

6 tasks

seisman mentioned this pull request Aug 30, 2022

Offline installation using pip no longer works #2097

Closed

weiji14 mentioned this pull request Jan 6, 2024

Exclude test files from source/binary distributions #2956

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate project metadata from setup.py to pyproject.toml following PEP621 #1848

Migrate project metadata from setup.py to pyproject.toml following PEP621 #1848

seisman commented Mar 26, 2022 •

edited

Loading

weiji14 left a comment

seisman Mar 27, 2022 •

edited

Loading

weiji14 Mar 27, 2022

leouieda commented Mar 28, 2022

seisman commented Mar 29, 2022

leouieda commented Mar 29, 2022

seisman Aug 12, 2022

weiji14 Aug 12, 2022

seisman Aug 13, 2022

maxrjones Aug 19, 2022

weiji14 Aug 19, 2022 •

edited

Loading

seisman Aug 20, 2022

seisman Aug 20, 2022

seisman Aug 30, 2022

maxrjones Aug 30, 2022

seisman commented Aug 14, 2022

weiji14 left a comment

weiji14 Aug 18, 2022

seisman Aug 20, 2022

Migrate project metadata from setup.py to pyproject.toml following PEP621 #1848

Migrate project metadata from setup.py to pyproject.toml following PEP621 #1848

Conversation

seisman commented Mar 26, 2022 • edited Loading

weiji14 left a comment

Choose a reason for hiding this comment

seisman Mar 27, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

leouieda commented Mar 28, 2022

seisman commented Mar 29, 2022

leouieda commented Mar 29, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

weiji14 Aug 19, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

seisman commented Aug 14, 2022

weiji14 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

seisman commented Mar 26, 2022 •

edited

Loading

seisman Mar 27, 2022 •

edited

Loading

weiji14 Aug 19, 2022 •

edited

Loading