Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add PyPy macOS arm64 #1372

Merged
merged 6 commits into from
Jan 14, 2023
Merged

feat: add PyPy macOS arm64 #1372

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
e6012fa
feat: add PyPy macOS arm64
mayeut Dec 11, 2022
5dab79d
fix: use PyPy 7.3.11
henryiii Jan 2, 2023
dd4ae8d
Update README.md
mayeut Jan 8, 2023
110c748
fix: filter-out unsupported PyPy cross-compilation configs
mayeut Jan 8, 2023
8f65097
Add quiet notice about the auto-skipping of arm64 pp builds on x86_64
joerick Jan 10, 2023
8dbabd8
Copy tweak
joerick Jan 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ jobs:
output-dir: wheelhouse
env:
CIBW_ARCHS_MACOS: x86_64 universal2 arm64
CIBW_SKIP: pp*-macosx_arm64
Copy link
Contributor

@joerick joerick Jan 6, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm. on further consideration this does seem a little like a footgun - many people will have cross-compilation written into their configs, which will fail when this PR is added. Can we perhaps provide a softer failure on macOS when pp*_arm64 is erroneously selected on x86_64?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

e.g. when we attempt to build pp-arm64 on x86_64, we print-

warning: pp38-macosx_arm64 was selected but can't be built on this architecture (x86_64). Deselect these builds using CIBW_SKIP=pp*-macosx_arm64 to remove this warning.

I realise this is a special case, but I expect many people to hit a failed build otherwise.

On this other hand, we could say that we'd rather error out for consistency. But since our example configs have included this since arm64 support was added, it's gonna hit a lot of people.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could even downgrade that from a warning to a notice, since there's no real problem with not having skipped pp*_arm64.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, since we don't support pypy cross compilation, I don't even know if we need a notice - it should simply not be a selectable option on a non-native platfrom, IMO, unless we add cross compilation.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went with @henryiii proposal to filter out this configuration. Tests were updated to check this, let's see CI results.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd still be in favour of a notice, since it's the only build identifier that's not selectable in this way.


- uses: actions/upload-artifact@v3
with:
Expand Down
5 changes: 3 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,13 @@ What does it do?
| CPython 3.10 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ |
| CPython 3.11 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ |
| PyPy 3.7 v7.3 | ✅ | N/A | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A |
| PyPy 3.8 v7.3 | ✅ | N/A | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A |
| PyPy 3.9 v7.3 | ✅ | N/A | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A |
| PyPy 3.8 v7.3 | ✅ | ✅⁴ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A |
| PyPy 3.9 v7.3 | ✅ | ✅⁴ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A |

<sup>¹ PyPy is only supported for manylinux wheels.</sup><br>
<sup>² Windows arm64 support is experimental.</sup><br>
<sup>³ Alpine 3.14 and very briefly 3.15's default python3 [was not able to load](https://github.com/pypa/cibuildwheel/issues/934) musllinux wheels. This has been fixed; please upgrade the python package if using Alpine from before the fix.</sup><br>
<sup>⁴ Cross-compilation not supported with PyPy.</sup><br>
mayeut marked this conversation as resolved.
Show resolved Hide resolved

- Builds manylinux, musllinux, macOS 10.9+, and Windows wheels for CPython and PyPy
- Works on GitHub Actions, Azure Pipelines, Travis CI, AppVeyor, CircleCI, GitLab CI, and Cirrus CI
Expand Down
18 changes: 12 additions & 6 deletions bin/update_pythons.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,8 +159,8 @@ def update_version_windows(self, spec: Specifier) -> ConfigWinCP:
)

def update_version_macos(self, spec: Specifier) -> ConfigMacOS:
if self.arch != "64":
msg = "Other archs not supported yet on macOS"
if self.arch not in {"64", "ARM64"}:
msg = f"'{self.arch}' arch not supported yet on macOS"
raise RuntimeError(msg)

releases = [r for r in self.releases if spec.contains(r["python_version"])]
Expand All @@ -172,12 +172,14 @@ def update_version_macos(self, spec: Specifier) -> ConfigMacOS:

release = releases[-1]
version = release["python_version"]
identifier = f"pp{version.major}{version.minor}-macosx_x86_64"
arch = "x86_64" if self.arch == "64" else self.arch.lower()
identifier = f"pp{version.major}{version.minor}-macosx_{arch}"

arch = "x64" if self.arch == "64" else self.arch.lower()
(url,) = (
rf["download_url"]
for rf in release["files"]
if "" in rf["platform"] == "darwin" and rf["arch"] == "x64"
if "" in rf["platform"] == "darwin" and rf["arch"] == arch
)

return ConfigMacOS(
Expand Down Expand Up @@ -251,6 +253,7 @@ def __init__(self) -> None:

self.macos_cpython = CPythonVersions()
self.macos_pypy = PyPyVersions("64")
self.macos_pypy_arm64 = PyPyVersions("ARM64")

def update_config(self, config: dict[str, str]) -> None:
identifier = config["identifier"]
Expand All @@ -261,11 +264,14 @@ def update_config(self, config: dict[str, str]) -> None:
config_update: AnyConfig | None = None

# We need to use ** in update due to MyPy (probably a bug)
if "macos" in identifier:
if "macosx" in identifier:
if identifier.startswith("cp"):
config_update = self.macos_cpython.update_version_macos(identifier, version, spec)
elif identifier.startswith("pp"):
config_update = self.macos_pypy.update_version_macos(spec)
if "macosx_x86_64" in identifier:
config_update = self.macos_pypy.update_version_macos(spec)
elif "macosx_arm64" in identifier:
config_update = self.macos_pypy_arm64.update_version_macos(spec)
elif "win32" in identifier:
if identifier.startswith("cp"):
config_update = self.windows_32.update_version_windows(spec)
Expand Down
2 changes: 2 additions & 0 deletions cibuildwheel/resources/build-platforms.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,9 @@ python_configurations = [
{ identifier = "cp311-macosx_universal2", version = "3.11", url = "https://www.python.org/ftp/python/3.11.1/python-3.11.1-macos11.pkg" },
{ identifier = "pp37-macosx_x86_64", version = "3.7", url = "https://downloads.python.org/pypy/pypy3.7-v7.3.9-osx64.tar.bz2" },
{ identifier = "pp38-macosx_x86_64", version = "3.8", url = "https://downloads.python.org/pypy/pypy3.8-v7.3.11-macos_x86_64.tar.bz2" },
{ identifier = "pp38-macosx_arm64", version = "3.8", url = "https://downloads.python.org/pypy/pypy3.8-v7.3.11-macos_arm64.tar.bz2" },
{ identifier = "pp39-macosx_x86_64", version = "3.9", url = "https://downloads.python.org/pypy/pypy3.9-v7.3.11-macos_x86_64.tar.bz2" },
{ identifier = "pp39-macosx_arm64", version = "3.9", url = "https://downloads.python.org/pypy/pypy3.9-v7.3.11-macos_arm64.tar.bz2" },
]

[windows]
Expand Down
4 changes: 2 additions & 2 deletions docs/options.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -227,8 +227,8 @@ When setting the options, you can use shell-style globbing syntax, as per [fnmat
| Python 3.10 | cp310-macosx_x86_64<br/>cp310-macosx_universal2<br/>cp310-macosx_arm64 | cp310-win_amd64<br/>cp310-win32<br/>cp310-win_arm64 | cp310-manylinux_x86_64<br/>cp310-manylinux_i686<br/>cp310-musllinux_x86_64<br/>cp310-musllinux_i686 | cp310-manylinux_aarch64<br/>cp310-manylinux_ppc64le<br/>cp310-manylinux_s390x<br/>cp310-musllinux_aarch64<br/>cp310-musllinux_ppc64le<br/>cp310-musllinux_s390x |
| Python 3.11 | cp311-macosx_x86_64<br/>cp311-macosx_universal2<br/>cp311-macosx_arm64 | cp311-win_amd64<br/>cp311-win32<br/>cp311-win_arm64 | cp311-manylinux_x86_64<br/>cp311-manylinux_i686<br/>cp311-musllinux_x86_64<br/>cp311-musllinux_i686 | cp311-manylinux_aarch64<br/>cp311-manylinux_ppc64le<br/>cp311-manylinux_s390x<br/>cp311-musllinux_aarch64<br/>cp311-musllinux_ppc64le<br/>cp311-musllinux_s390x |
| PyPy3.7 v7.3 | pp37-macosx_x86_64 | pp37-win_amd64 | pp37-manylinux_x86_64<br/>pp37-manylinux_i686 | pp37-manylinux_aarch64 |
| PyPy3.8 v7.3 | pp38-macosx_x86_64 | pp38-win_amd64 | pp38-manylinux_x86_64<br/>pp38-manylinux_i686 | pp38-manylinux_aarch64 |
| PyPy3.9 v7.3 | pp39-macosx_x86_64 | pp39-win_amd64 | pp39-manylinux_x86_64<br/>pp39-manylinux_i686 | pp39-manylinux_aarch64 |
| PyPy3.8 v7.3 | pp38-macosx_x86_64<br/>pp38-macosx_arm64 | pp38-win_amd64 | pp38-manylinux_x86_64<br/>pp38-manylinux_i686 | pp38-manylinux_aarch64 |
| PyPy3.9 v7.3 | pp39-macosx_x86_64<br/>pp39-macosx_arm64 | pp39-win_amd64 | pp39-manylinux_x86_64<br/>pp39-manylinux_i686 | pp39-manylinux_aarch64 |

The list of supported and currently selected build identifiers can also be retrieved by passing the `--print-build-identifiers` flag to cibuildwheel.
The format is `python_tag-platform_tag`, with tags similar to those in [PEP 425](https://www.python.org/dev/peps/pep-0425/#details).
Expand Down
9 changes: 8 additions & 1 deletion test/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,14 @@ def expected_wheels(

if platform == "macos" and machine_arch == "arm64":
# arm64 macs are only supported by cp38+
python_abi_tags = ["cp38-cp38", "cp39-cp39", "cp310-cp310", "cp311-cp311"]
python_abi_tags = [
"cp38-cp38",
"cp39-cp39",
"cp310-cp310",
"cp311-cp311",
"pp38-pypy38_pp73",
"pp39-pypy39_pp73",
]

wheels = []

Expand Down