add cibuildwheel GHA

[ocefpaf]

This is just a proof of concept to see if we can build wheels here. We probably want to build out own hdf5 and netcdf-c b/c the ones from CentOS are quite old probably the source of the failures we are seeing in the tests here. See https://github.com/MacPython/netcdf4-python-wheels/blob/master/config.sh#L13-L26 for the versions used in the muiltbuild.

xref.: #1204

[ocefpaf]

Ubuntu wheels are using netcdf 4.3.3.1 and hdf5 1.8.12 is failing with:

 netCDF4 v1.6.5.
  .......E..........F.........s........................s...........................................
  ======================================================================
  ERROR: test_get (tst_fancyslicing.VariablesTestCase)
  testing 'fancy indexing'
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/project/test/tst_fancyslicing.py", line 147, in test_get
      data = v[ibx,iby,ibz]
    File "src/netCDF4/_netCDF4.pyx", line 4948, in netCDF4._netCDF4.Variable.__getitem__
    File "/tmp/tmp.QW23YtRUJ7/venv/lib/python3.8/site-packages/netCDF4/utils.py", line 429, in _StartCountStride
      start[...,i] = np.apply_along_axis(lambda x: e*x, i, np.ones(sdim[:-1]))
    File "<__array_function__ internals>", line 200, in apply_along_axis
    File "/tmp/tmp.QW23YtRUJ7/venv/lib/python3.8/site-packages/numpy/lib/shape_base.py", line 376, in apply_along_axis
      raise ValueError(
  ValueError: Cannot apply_along_axis when any iteration dimensions are 0
  
  ======================================================================
  FAIL: runTest (tst_masked.PrimitiveTypesTestCase)
  testing auto-conversion of masked arrays and packed integers
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/project/test/tst_masked.py", line 164, in runTest
      assert var2[:].mask.any() == False
  AssertionError

macOS is passing with the libs hdf5 1.14.2 and netcdf 4.9.2_1

[richli]

One option, I'm not sure if it's the best:

What about setting the cibuildwheel image to use manylinux_2_28? manylinux2014 is based on CentOS 7, which has less than a year left before its end-of-life (2024-06-30). manylinux_2_28 is based on Alma Linux 8, which uses netCDF 4.7.0.

But, in the existing wheel generation process (https://github.com/MacPython/netcdf4-python-wheels), netCDF and its dependencies are compiled from source, so currently netCDF 4.9.2 is being bundled for the wheels on PyPI.

So maybe a better option is to follow suit and build the dependencies from source in here as well instead of using the ancient versions in the EPEL 7 repos.

EDIT: I didn't read in the original description that you already suggested building the dependencies from source

[ZedThree]

I've done something similar for my nc-complex package: CI workflow, cibuildwheel config in the pyproject.toml, example workflow run

I used manylinux_2_28 in order to get the more recent HDF5, and then only had to build netcdf-C from source.

Note that I use my fork of netcdf-C because I'm waiting for PR #2762 to fix finding HDF5 correctly from CMake. Once that's in, using the lastest main of netcdf-C would be good because it will have compatibility with h5netcdf.

[ocefpaf]

I'll try to focus on building hdf5 and netcdf-c over the next week There are a few options I want to try:

1. Use a more modern manylinux image and try the system libraries again. Advantage: easy. Disadvantage: little to no control over the hdf5 and netcdf-c;
2. Build using conda-forge packages for hdf5 and netcdf-c. This GitHub gave me this idea. Advantage: looks easier than the alternative 3 but harder than 1, uses community packages* that are curated by many maintainers who are also active here. Disadvantage: not 100% imediate control over the c-libs b/c someone needs to send a PR to conda-forge first;
3. Build both hdf5 and netcdf-c from source here. Advantage: 100% control over the builds. Disadvantage: hard to implement and maintain (IMO).

As a conda-forge founder, core member, and maintainer of many geo/met/ocean packages, I'm heavily biased towards 2. What do you think @ZedThree, @jswhit, and @dopplershift?

* we already do something similar for the Windows packages and I want to move the current Windows builds to this, regardless of the Linux and macOS choice, to keep things in a single repository.

[ZedThree]

I suggest using manylinux_2_28 with the system HDF5 (which is 1.10) and netcdf-C from source (because the system version is only 4.7.0).

I would definitely try avoiding building HDF5 from scratch if possible, as you might find you end up building a lot more of the stack!

[ocefpaf]

I suggest using manylinux_2_28 with the system HDF5 (which is 1.10) and netcdf-C from source (because the system version is only 4.7.0).

Looks like their hdf5 has libaec, that is a good start. I recall some issues with openssl in the past and that required folks to compiling their own. I'm not sure if that is still true. Let's call this suggestion would be "easy" to implement b/c I can copy your setup but would give 100% control only over the netcdf-c library. Somewhere between options 1 and 3.

[jswhit]

What about the approach of using conda built libs? (ocefpaf/netcdf4-win-wheels#6)

If you want to enable all the features of hdf5/netcdf-c it requires a lot of dependencies to be built - don't think we want to do all that here.

[ocefpaf]

What about the approach of using conda built libs? (ocefpaf/netcdf4-win-wheels#6)

If you want to enable all the features of hdf5/netcdf-c it requires a lot of dependencies to be built - don't think we want to do all that here.

Option 2 is a "modern" version of that where the main advantage is that it uses cibuildwheel+conda-forge clibs, and keep things in this repo, no need for third party ones. I'll try that and see if it works.

[ocefpaf]

The {project} directory is on Drive D:, not C: and for some odd reason the PATH variables change if you switch drives on Windows. It is easier just to copy them to C:.

[ocefpaf]

We can probably drop numpy, not sure why it wasn't installed with the wheel. Cython and packaging are test dependencies and we can figure out a more elegant way to handle this later.

[ZedThree]

Move them to pyproject.toml?

[project.optional-dependencies]
tests = [
  "Cython",
  "packaging",
]

[ocefpaf]

Windows wheels using conda package is working 🎉

I'll try the same for Linux and macOS next week. Time to clock out a bit.

[ocefpaf]

@jswhit everything is passing but the devil is in the details as usual. I'd love for more eyes here. We could also try to consolidate some of the many environment variables and options, netcdf-c version, etc but I prefer to leave that for another PR.

[ocefpaf]

Looks like cibuildwheel still needs this regardless of it being declared in the pyporoject.toml. The levels of abstraction and isolation when building wheels is quite confusing. I guess we need to to a pip [test] install here. It has been to long that I used pip and I forget.

[jswhit]

This looks great! Thank you @ocefpaf and @ZedThree. I'd like to get this merged soon so we can get a 1.6.5 release out with 3.12 wheels. The old MacPython wheel building repo is broken, and it looks like the multibuild package it relies on is no longer actively maintained.

[matthew-brett]

The old MacPython wheel building repo is broken, and it looks like the multibuild package it relies on is no longer actively maintained.

Just to say - for the record - we (the multibuild maintainers) think the build is broken because your multibuild is not up to date (see MacPython/netcdf4-python-wheels#23) - and we are maintaining multibuild - but no complaints from us for switching away.

[ocefpaf]

These are a bit high, most folks set a lower bar with MACOSX_DEPLOYMENT_TARGET=10.9. If we aim to set 10.9 that we would need to build our own HDF5. I'm not sure how many people we are leaving behind by setting a higher deployment target, probably the same amount as brew. With that said, I'd like to try the HDF5 in another PR. This one is getting unwieldy already.

[jswhit2]

Looks like 13.0 was released in Oct 24, 2022 - this seems problematic to me. I think we would be leaving at least half our users behind with this restriction.

[ocefpaf]

I have some guidelines on adding hdf5 from another project. I'll try that but I don't know if I can make it before the next release. Indeed, brew is too new.

[jswhit2]

OK I guess we could release 1.7.0 with the 13.0 restriction, and add new wheels to the pypi release later one the hdf5 build is working?

[jswhit2]

just a thought - macports supports older OSes than brew. Have no idea how hard that would be to get working.

[ocefpaf]

Let's keep that as a plan B then. I'll send if I can get a better alternative working first. Pytable's infrastructure seems pretty close to what we need here.

[ocefpaf]

Would MACOSX_DEPLOYMENT_TARGET 11 work for you? That is the oldest image available and brew is working fine to install both hdf5 and netcdf on it compiling from source. I still want to pursue the approach used in pytables at a later time. just want to be sure I'll have all the proper dependencies used in the current multibuild.

[jswhit2]

a target of macos 11 would be fine, yes.

[ocefpaf]

a target of macos 11 would be fine, yes.

Done!

agree that 24M is pretty bloated, but I'd rather have that than restrict to macos 13.0

BTW, the main reason for that size was the inclusion of icu. We are discussing an alternative in conda-forge to remove that so we can make smaller wheels.

[jswhit2]

thanks a million @ocefpaf! Merging now...

[ocefpaf]

@ocefpaf I'm fine with the current approach for hdf5. Hope you enjoyed the long weekend!

@jswhit I'm done here! I guess that, the only future change I'd make is to build everything, hdf5, curl, openssl, etc from ourselves. But I'll leave that for the future when we have more time.