Python 3.12 has fatal error #2579
Comments
|
Could you please provide an uwsgi invocation to reproduce it? thanks |
It's hard because uwsgi in deep abstraction level. But this string of execution: I'll try to reproduce it without all abstractions. |
|
@xrmx simple fastapi app. fastapi_app.py from fastapi import FastAPI
application = FastAPI()
@application.get('/test')
def test():
return "Hello World!"Command: Output: |
|
Do you have the python module built into the main binary? @xrmx There's a second call to |
|
We are seeing the same error when trying to upgrade to Python 3.12. We are using uWSGI provided via the pyuwsgi package: https://pypi.org/project/pyuwsgi/. |
|
Would you be able to test from a branch instead of a normal source
tarball?
|
|
Yes, if that can help get this solved :) |
|
After staring at this for a while, and figuring out how to build I did:
And uwsgi starts up cleanly. I then get an error when actually requesting the The only possible explanation I have is that for some reason the wheel that was built in the github pipeline was somehow not built against 3.12, or a very early version of 3.12, that didn't trigger the version check in uwsgi, and took the pre-3.12 code path? |
|
@Lalufu I maintain the wheels and have been trusting
I upgraded |
|
Appreciate the test.
This should be good, though, we test for `(PY_VERSION_HEX >=
0x030c0000)` and 3.12.0rc1 is `0x030C00C1`, which is clearly greater.
|
|
I can confirm that I am only able to reproduce the issue with the A reproduction case for this can be found here: https://gist.github.com/Tenzer/bf8b018d823d0f5c4f291b464444c4cd. uWSGI fails to start with both amd64 and arm64. If line 3 in |
2.0.23 was affected by a packaging error for Python 3.12 see unbit/uwsgi#2579
|
I'm testing some different ways to build, but this should come pretty close to mirroring what the GHA is doing: You should end up with Python 3.12 wheel in the |
|
I'm stumped on this. I was able to build the wheel successfully locally using |
|
Try applying this to the uWSGI tree pyuwsgi before building?
diff --git a/plugins/python/python_plugin.c b/plugins/python/python_plugin.c
index c69abc18..fcfa66b5 100644
--- a/plugins/python/python_plugin.c
+++ b/plugins/python/python_plugin.c
@@ -222,6 +222,18 @@ int uwsgi_python_init() {
uwsgi_log_initial("Python version: %.*s %s\n", pyversion-Py_GetVersion(), Py_GetVersion(), Py_GetCompiler()+1);
}
+ uwsgi_log_initial("Built against 0x%08x\n", PY_VERSION_HEX);
+#ifdef UWSGI_PY311
+ uwsgi_log_initial("UWSGI_PY11 defined\n");
+#else
+ uwsgi_log_initial("UWSGI_PY11 NOT defined\n");
+#endif
+#ifdef UWSGI_PY312
+ uwsgi_log_initial("UWSGI_PY12 defined\n");
+#else
+ uwsgi_log_initial("UWSGI_PY12 NOT defined\n");
+#endif
+
if (Py_IsInitialized()) {
uwsgi_log("--- Python VM already initialized ---\n");
PyGILState_Ensure();
|
|
uwsgi-version-debug.txt |
|
I have wheels built here with the patch applied. I tried running them and am not seeing anything new in the output. Maybe you could take a look @Lalufu ? |
|
You should see output when using the wheel, not during the build. See
the `fastapi_app.py` example a bit further up.
Where can I find the artifacts (binary wheels) from the last pipeline
run?
|
|
Yes, I ran the example with the wheels from the build and didn't see (or missed) any additional output. You can find the wheels with the debugging patch applied here, https://github.com/lincolnloop/pyuwsgi-wheels/actions/runs/7144198124 |
|
Found them, thanks.
I installed
pyuwsgi-2.0.23.post0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
from that into a new venv, and got the following:
```
[...]
Python version: 3.11.6 (main, Oct 3 2023, 00:00:00) [GCC 13.2.1
20230728 (Red Hat 13.2.1-1)]
Built against 0x030711f0
UWSGI_PY11 NOT defined
UWSGI_PY12 NOT defined
--- Python VM already initialized ---
[...]
```
The runtime python is a 3.11.6 (as expected), but the binary module was
built against a 3.7.17. Which certainly isn't a 3.11.
I'm not sure what's going on in that build system, but it's not doing
what it's supposed to do. I'm somewhat surprised this isn't blowing up
more violently.
|
|
What I think is happening is that the build pipeline contains python
binaries of the proper version (like 3.12.0rc1) and is using these to
run the `pip wheel` process, but only contains development C headers for
3.7.11, and that's what's getting picked up by uWSGI.
I'm not familiar enough with this to know how to fix this, though.
|
|
I opened an issue upstream to try to get some more details on how to debug. |
|
@Lalufu based on the feedback here pypa/cibuildwheel#1693 (comment) Is there a way I can output the uWSGI Python version in a command that doesn't start a server that runs forever? Currently I'm using |
|
The include paths should be a good hint on which headers uwsgi is building against. See CFLAGS in plugins/pyuwsgi/uwsgiplugin.py. |
|
What might work is running
`~/venv/pyuwsgi/bin/uwsgi --module foo --need-app --http :8080`
(or whereever you installed the wheel), making sure that `foo` doesn't
actually exist in your path. This will start uwsgi, initialize the
python interpreter (showing the log messages about the version it was
built against if you used the patch), and then die because it can't find
`foo`.
|
|
Could I get confirmation that the wheels in the If so, I can clean things up and get this merged and published. |
@ipmb I've had a shot at that just now and here's my findings as I'm also hitting the I see this may be related to the upstream pypa/cibuildwheel#1693 but posting here to help add detail. Additional Context: Steps followed: Reproducing the issue: With Switching to Python 3.10, I'm able to successfully install & mostly* run
|
|
I believe the pyuwsgi issue has been resolved, there's a `2.0.23.post0`
release that should not have this issue.
|
That is still not published on PyPi, no (https://pypi.org/project/pyuwsgi/#history) ? Are there any plans to do so? |
|
That would be a question for @ipmb and the
https://github.com/lincolnloop/pyuwsgi-wheels/ project.
|
|
I want to avoid pushing another potentially bad version to PyPI. I believe the issue is resolved, but have not gotten confirmation from anyone that the wheels I linked to here are actually working. If I could get a second set of eyes on them, I will publish them asap. |
@ipmb thank u for the good work 👍🏼 I checked it on my laptop and everything works as expected. [uWSGI] getting INI configuration from /home/grey/Projects/vst-utils/vstutils/web.ini
Keep-Alive mode enabled
uWSGI metrics enabled
Run quick with lib `quick`
*** Starting uWSGI 2.0.23 (64bit) on [Tue Jan 9 22:48:07 2024] ***
compiled with version: 10.2.1 20210130 (Red Hat 10.2.1-11) on 13 December 2023 22:08:06
os: Linux-6.5.0-10013-tuxedo #16 SMP PREEMPT_DYNAMIC Mon Dec 18 11:55:02 UTC 2023
nodename: grey-notebook
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 12
current working directory: /home/grey/Projects/vst/quick
writing pidfile to /tmp/quick_web.pid
detected binary path: /home/grey/.pyenv/versions/3.12.1/bin/python3.12
your processes number limit is 255561
your memory page size is 4096 bytes
detected max file descriptor number: 1024
building mime-types dictionary from file /etc/mime.types...1517 entry found
building mime-types dictionary from file /etc/mime.types...1517 entry found
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address :8080 fd 3
uwsgi socket 1 bound to TCP address :8081 fd 4
Python version: 3.12.1 (main, Dec 20 2023, 15:39:09) [GCC 11.4.0]
--- Python VM already initialized ---
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x7f3a3e8c3708
your server socket listen backlog is limited to 128 connections
your mercy for graceful operations on workers is 60 seconds
mapped 145808 bytes (142 KB) for 1 cores
*** Operational MODE: single process ***
initialized 39 metrics
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 46320)
spawned uWSGI worker 1 (pid: 46321, cores: 1)
metrics collector thread started
[uwsgi-daemons] spawning "celery --app=quick.wapp:app worker --loglevel=WARNING --pidfile=/tmp/quick_worker.pid --autoscale=4,1 --hostname=grey@%h --beat --logfile=/dev/null --queues=celery" (uid: 1000 gid: 1000)
start uvicorn worker 1
INFO: Started server process [46321]
INFO: Waiting for application startup.
INFO: Application startup complete.
Stale pidfile exists - Removing it.But I use custom worker. Simple check |
|
pyuwsgi 2.0.23.post0 is on PyPI now https://pypi.org/project/pyuwsgi/2.0.23.post0/ |
|
I can confirm it works on my side. Thanks @ipmb 🙏🏽 🙏🏽 |
Fatal Python error: PyImport_AppendInittab: PyImport_AppendInittab() may not be called after Py_Initialize()I hoped that it was problem in old version but it still fails.
lincolnloop/pyuwsgi-wheels#20
Looks like that in this part of code
PyImport_AppendInittabcalled beforePy_Initialize.The text was updated successfully, but these errors were encountered: