BUILD: Pandas 1.5.3 is unusable due to incompatibility with Numpy 2.0.0

[rsayn]

Installation check

• I have read the installation guide.

Platform

macOS-14.5-arm64-arm-64bit

Installation Method

pip install

pandas Version

1.5.3

Python Version

3.10.13

Installation Logs

Installing pandas==1.5.3 with either Pip or Poetry on Python >= 3.9 leads to an error upon the first import of Pandas.

This is fixed by later versions, but some of our workflows are still using version 1.5.3.

Pinning Numpy to be below version 2.0.0 should fix the issue, I'll be happy to open a PR myself if needed 😄

Python 3.10.13 (main, Sep  4 2023, 17:19:45) [Clang 16.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<project_folder>/pip_project/.venv_310/lib/python3.10/site-packages/pandas/__init__.py", line 22, in <module>
    from pandas.compat import is_numpy_dev as _is_numpy_dev  # pyright: ignore # noqa:F401
  File "<project_folder>/pip_project/.venv_310/lib/python3.10/site-packages/pandas/compat/__init__.py", line 18, in <module>
    from pandas.compat.numpy import (
  File "<project_folder>/pip_project/.venv_310/lib/python3.10/site-packages/pandas/compat/numpy/__init__.py", line 4, in <module>
    from pandas.util.version import Version
  File "<project_folder>/pip_project/.venv_310/lib/python3.10/site-packages/pandas/util/__init__.py", line 2, in <module>
    from pandas.util._decorators import (  # noqa:F401
  File "<project_folder>/pip_project/.venv_310/lib/python3.10/site-packages/pandas/util/_decorators.py", line 14, in <module>
    from pandas._libs.properties import cache_readonly
  File "<project_folder>/pip_project/.venv_310/lib/python3.10/site-packages/pandas/_libs/__init__.py", line 13, in <module>
    from pandas._libs.interval import Interval
  File "pandas/_libs/interval.pyx", line 1, in init pandas._libs.interval
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

[lithomas1]

You need pandas 2.2.2 to use numpy 2.0.

Unfortunately, I don't think the release infrastructure still exists for pandas 1.5.3, so we aren't going to release another pandas in the 1.5.x series that fixes this.
(the build system is completely different)

We will try to pin numpy in future pandas , but figuring out which version of numpy to pin too requires some thought.

[Ark-kun]

Unfortunately, I don't think the release infrastructure still exists for pandas 1.5.3, so we aren't going to release another pandas in the 1.5.x series that fixes this.

Is it possible to just take the last 1.x wheel, unpack it, change the requirement, bump the version and upload the fixed wheel to PyPI?

[Ark-kun]

We will try to pin numpy in future pandas , but figuring out which version of numpy to pin too requires some thought.

I'm not sure exact pinning is required. The best practice is to at least set the upper bound to prevent usage of next major version (e.g. numpy<2.0.0 or numpy~=1.4). Otherwise, it's only a matter of time until all your released packages break forever (something that has just happened).