[Question]: Multiple concurrent chats cause database connection error and can't update token usage error

[hoop22]

Describe your problem

I am encountering a database connection error (peewee.InterfaceError: (0, '')) when multiple users chat with the same chatbot instance concurrently.

I have set up RagFlow using Docker, and everything works fine for single-user interactions. However, when running a stress test with multiple users (e.g., 50-100 concurrent sessions), the system frequently crashes with the following error:

025-02-20 21:34:37,340 INFO     14 172.18.0.6 - - [20/Feb/2025 21:34:37] "POST /api/v1/chats/ac3396acef3311efa6510242ac120006/completions HTTP/1.1" 200 -
2025-02-20 21:34:40,062 INFO     14 172.18.0.6 - - [20/Feb/2025 21:34:40] "POST /v1/conversation/set HTTP/1.1" 200 -
2025-02-20 21:34:40,084 INFO     14 172.18.0.6 - - [20/Feb/2025 21:34:40] "GET /v1/conversation/list?dialog_id=ac3396acef3311efa6510242ac120006 HTTP/1.1" 200 -
2025-02-20 21:34:40,191 ERROR    14 LLMBundle.encode_queries can't update token usage for 670c4e1eef3011efbe580242ac120006/EMBEDDING used_tokens: 2
2025-02-20 21:34:40,272 INFO     14 POST http://es01:9200/ragflow_670c4e1eef3011efbe580242ac120006/_search [status:200 duration:0.006s]
2025-02-20 21:34:40,326 INFO     14 POST http://es01:9200/ragflow_670c4e1eef3011efbe580242ac120006/_search [status:200 duration:0.006s]
2025-02-20 21:34:50,616 INFO     14 HTTP Request: POST http://192.168.1.55:11434/api/chat "HTTP/1.1 200 OK"
2025-02-20 21:34:50,617 ERROR    14 (0, '')
Traceback (most recent call last):
  File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3291, in execute_sql
    cursor.execute(sql, params or ())
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/cursors.py", line 153, in execute
    result = self._query(query)
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/cursors.py", line 322, in _query
    conn.query(q)
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 562, in query
    self._execute_command(COMMAND.COM_QUERY, sql)
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 843, in _execute_command
    raise err.InterfaceError(0, "")
pymysql.err.InterfaceError: (0, '')

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/ragflow/.venv/lib/python3.10/site-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
  File "/ragflow/.venv/lib/python3.10/site-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
  File "/ragflow/api/utils/api_utils.py", line 303, in decorated_function
    return func(*args, **kwargs)
  File "/ragflow/api/apps/sdk/session.py", line 156, in chat_completion
    for ans in rag_completion(tenant_id, chat_id, **req):
  File "/ragflow/api/db/services/conversation_service.py", line 156, in completion
    ConversationService.update_by_id(conv.id, conv.to_dict())
  File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3128, in inner
    return fn(*args, **kwargs)
  File "/ragflow/api/db/services/common_service.py", line 109, in update_by_id
    num = cls.model.update(data).where(cls.model.id == pid).execute()
  File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 2011, in inner
    return method(self, database, *args, **kwargs)
  File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 2082, in execute
    return self._execute(database)
  File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 2600, in _execute
    cursor = database.execute(self)
  File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3299, in execute
    return self.execute_sql(sql, params)
  File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3289, in execute_sql
    with __exception_wrapper__:
  File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3059, in __exit__
    reraise(new_type, new_type(exc_value, *exc_args), traceback)
  File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 192, in reraise
    raise value.with_traceback(tb)
  File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3291, in execute_sql
    cursor.execute(sql, params or ())
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/cursors.py", line 153, in execute
    result = self._query(query)
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/cursors.py", line 322, in _query
    conn.query(q)
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 562, in query
    self._execute_command(COMMAND.COM_QUERY, sql)
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 843, in _execute_command
    raise err.InterfaceError(0, "")
peewee.InterfaceError: (0, '')

Here is my test script

import requests
import concurrent.futures
import time
import random

CHAT_ID = "XXXXXXX"  # All dummy users share the same assistant

headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {API_KEY}"
}

def create_session():
    """ Create a new session for each user """
    session_url = f"{BASE_URL}/chats/{CHAT_ID}/sessions"
    session_data = {"name": f"test_session_{random.randint(1000, 9999)}"}
    response = requests.post(session_url, json=session_data, headers=headers)
    return response.json()["data"]["id"] if response.status_code == 200 else None

def chat_with_bot(session_id):
    """ Simulate user chat """
    chat_url = f"{BASE_URL}/chats/{CHAT_ID}/completions"
    chat_data = {
        "question": "What are the hospital's operating hours?",
        "session_id": session_id,
        "stream": False
    }
    response = requests.post(chat_url, json=chat_data, headers=headers)
    return response.json()

# Simulating 100 users chatting at the same time
with concurrent.futures.ThreadPoolExecutor(max_workers=50) as executor:
    futures = [executor.submit(chat_with_bot, create_session()) for _ in range(100)]
    concurrent.futures.wait(futures)

I'm just wonderring if I should create a separate assistant (CHAT_ID) per user to avoid conflicts?
Any other suggestions?

Thx a lot!

[Mr-greenplus]

我也有这样的问题，我是在两个用户或以上同时使用同一个chatid时，就不输出了

[Mr-greenplus]

我也有这样的问题，我是在两个用户或以上同时使用同一个chatid时，就不输出了

我是报了这个错误：
2025-02-21 10:42:21,959 ERROR 26 LLMBundle.similarity can't update token usage for cdec426eac9711ef8fd30242ac130006/RERANK used_tokens: 60004

[Mr-greenplus]

我也有这样的问题，我是在两个用户或以上同时使用同一个chatid时，就不输出了

我是报了这个错误： 2025-02-21 10：42：21,959 错误 26 LLMBundle.similarity 无法更新 cdec426eac9711ef8fd30242ac130006/RERANK used_tokens的令牌用法：60004

我把RERANK模型取消掉就可以正常使用了，有没有办法可以让rerank支持并发呢？

[KevinHuSh]

can't update token usage could be ignored.

[hoop22]

Probably there's something wrong with the database. I just found when the error below occurs,

File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 843, in _execute_command
 raise err.InterfaceError(0, "")

restart docker ragflow-mysql , and it works fine again.