Scheduled tasks will not close the connection, causing losing connection to MySQL server during query

[LogicJake]

According to my observation, each scheduled task will have its own database connection, but it will not close the connection by itself after the task ends. When this number of connections reaches a certain number, the program can no longer create new connections, resulting in the inability to perform subsequent tasks.

[LogicJake]

Task stuck！！

[ryleysevier]

I have just run into this issue as well with psql jobstore for django.

[ERROR] Mysql closed the connection. Perform reconnect...

I was able to replicate it on wsgi server by letting the backgroundscheduler start, confirming it had a connection entry in the psql server, then rebooting the psql server. The next job that runs throws the error above. All errors afterwards are thrown from jobstores.py trying to use the psql query backend.

Traceback (most recent call last): File "/home/site/wwwroot/antenv/lib/python3.7/site-packages/django_apscheduler/jobstores.py", line 193, in __call__ self._process_execution_event(event) File "/home/site/wwwroot/antenv/lib/python3.7/site-packages/django_apscheduler/jobstores.py", line 212, in _process_execution_event job = DjangoJob.objects.get(name=event.job_id) File "/home/site/wwwroot/antenv/lib/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/home/site/wwwroot/antenv/lib/python3.7/site-packages/django/db/models/query.py", line 393, in get num = len(clone) File "/home/site/wwwroot/antenv/lib/python3.7/site-packages/django/db/models/query.py", line 250, in __len__ self._fetch_all() File "/home/site/wwwroot/antenv/lib/python3.7/site-packages/django/db/models/query.py", line 1183, in _fetch_all self._result_cache = list(self._iterable_class(self)) File "/home/site/wwwroot/antenv/lib/python3.7/site-packages/django/db/models/query.py", line 54, in __iter__ results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size) File "/home/site/wwwroot/antenv/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1059, in execute_sql cursor = self.connection.cursor() File "/home/site/wwwroot/antenv/lib/python3.7/site-packages/django/db/backends/base/base.py", line 255, in cursor return self._cursor() File "/home/site/wwwroot/antenv/lib/python3.7/site-packages/django/db/backends/base/base.py", line 234, in _cursor return self._prepare_cursor(self.create_cursor(name)) File "/home/site/wwwroot/antenv/lib/python3.7/site-packages/django/db/utils.py", line 89, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/home/site/wwwroot/antenv/lib/python3.7/site-packages/django/db/backends/base/base.py", line 234, in _cursor return self._prepare_cursor(self.create_cursor(name)) File "/home/site/wwwroot/antenv/lib/python3.7/site-packages/django/db/backends/postgresql/base.py", line 218, in create_cursor cursor = self.connection.cursor() django.db.utils.InterfaceError: connection already closed
dj_aps_bug.txt

[jmpark1115]

I met this problems, too !
After meet this error, MySQL connections lost. Please refer to the following!
It really hard problems to solve it for me.
I wonder why the scheduler works bad with the time range above 300 sec.

https://stackoverflow.com/questions/61967471/operationalerror-pymysql-connections-in-read-bytes-error2013-lost-connection

[ryleysevier]

From what I can tell on, if you hit the timeout on the backend, the connection isn't reestablished on the the __reconnect()

But I'm not sure if django is supposed to be re-establishing that connection after it's closed and set to None

[ryleysevier]

I removed the line that sets the connection to None and added connection.reconnect() in __reconnect()

It now fails on the job the connection was broken in between and works great afterwards. I think there's a cursor that needs some reconnect logic somewhere that will fix it entirely.