QueuePool limit of size 150 overflow 30 reached, connection timed out, timeout 30.00 #21437
Description
Self Checks
- This is only for bug report, if you would like to ask a question, please head to Discussions.
- I have searched for existing issues search for existing issues, including closed ones.
- I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
- [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
- Please do not modify this template :) and fill in all the required fields.
Dify version
1.4.2
Cloud or Self Hosted
Self Hosted (Docker)
Steps to reproduce
2025-06-24 10:55:59,187.187 ERROR [Thread-595 (worker_with_context)] [app_generator.py:500] - Unknown Error when generating
Traceback (most recent call last):
File "/app/api/core/app/apps/advanced_chat/app_generator.py", line 471, in _generate_worker
conversation = self._get_conversation(conversation_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/core/app/apps/message_based_app_generator.py", line 260, in _get_conversation
conversation = db.session.query(Conversation).filter([Conversation.id](http://conversation.id/) == conversation_id).first()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2728, in first
return self.limit(1)._iter().first() # type: ignore
^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2827, in _iter
result: Union[ScalarResult[_T], Result[_T]] = self.session.execute(
^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2362, in execute
return self._execute_internal(
^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2237, in _execute_internal
conn = self._connection_for_bind(bind)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2106, in _connection_for_bind
return trans._connection_for_bind(engine, execution_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<string>", line 2, in _connection_for_bind
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/state_changes.py", line 139, in _go
ret_value = fn(self, *arg, **kw)
^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
conn = bind.connect()
^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 3278, in connect
return self._connection_cls(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 146, in __init__
self._dbapi_connection = engine.raw_connection()
^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 3302, in raw_connection
return self.pool.connect()
^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 449, in connect
return _ConnectionFairy._checkout(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 1263, in _checkout
fairy = _ConnectionRecord.checkout(pool)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 712, in checkout
rec = pool._do_get()
^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/pool/impl.py", line 168, in _do_get
raise exc.TimeoutError(
sqlalchemy.exc.TimeoutError: QueuePool limit of size 150 overflow 30 reached, connection timed out, timeout 30.00 (Background on this error at: https://sqlalche.me/e/20/3o7r)
2025-06-24 10:55:59,396.396 INFO [ThreadPoolExecutor-138_0] [_client.py:1038] - HTTP Request: POST http://sandbox-ap-southeast-1-production/v1/sandbox/run "HTTP/1.1 200 OK"
2025-06-24 10:56:01,421.421 INFO [ThreadPoolExecutor-169_0] [_client.py:1038] - HTTP Request: POST http://sandbox-ap-southeast-1-production/v1/sandbox/run "HTTP/1.1 200 OK"
2025-06-24 10:56:01,677.677 INFO [ThreadPoolExecutor-167_0] [_client.py:1038] - HTTP Request: POST http://sandbox-ap-southeast-1-production/v1/sandbox/run "HTTP/1.1 200 OK"
2025-06-24 10:56:02,957.957 ERROR [Thread-285 (worker_with_context)] [node.py:285] - error while executing llm node
Traceback (most recent call last):
File "/app/api/core/workflow/nodes/llm/node.py", line 212, in _run
prompt_messages, stop = self._fetch_prompt_messages(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/core/workflow/nodes/llm/node.py", line 704, in _fetch_prompt_messages
model = ModelManager().get_model_instance(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/core/model_manager.py", line 430, in get_model_instance
provider_model_bundle = self._provider_manager.get_provider_model_bundle(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/core/provider_manager.py", line 253, in get_provider_model_bundle
provider_configurations = self.get_configurations(tenant_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/core/provider_manager.py", line 98, in get_configurations
provider_name_to_provider_records_dict = self._get_all_providers(tenant_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/core/provider_manager.py", line 401, in _get_all_providers
providers = session.scalars(stmt)
^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2470, in scalars
return self._execute_internal(
^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2237, in _execute_internal
conn = self._connection_for_bind(bind)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2106, in _connection_for_bind
return trans._connection_for_bind(engine, execution_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<string>", line 2, in _connection_for_bind
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/state_changes.py", line 139, in _go
ret_value = fn(self, *arg, **kw)
^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
conn = bind.connect()
^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 3278, in connect
return self._connection_cls(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 146, in __init__
self._dbapi_connection = engine.raw_connection()
^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 3302, in raw_connection
return self.pool.connect()
^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 449, in connect
return _ConnectionFairy._checkout(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 1263, in _checkout
fairy = _ConnectionRecord.checkout(pool)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 712, in checkout
rec = pool._do_get()
^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/pool/impl.py", line 168, in _do_get
raise exc.TimeoutError(
sqlalchemy.exc.TimeoutError: QueuePool limit of size 150 overflow 30 reached, connection timed out, timeout 30.00 (Background on this error at: https://sqlalche.me/e/20/3o7r)
✔️ Expected Behavior
No response
❌ Actual Behavior
No response