feat(backend): Improve pyro reliability by adding connection timeout, retry, cleanup, and dynamic connection thread size

[majdyz]

Background

To avoid the same issue happening with database we need to do some prevention on managing the lifecycle of the pyro connection.

Changes 🏗️

Applying the recommendation here https://pyro5.readthedocs.io/en/latest/tipstricks.html#release-proxies-when-no-longer-used-avoids-after-x-simultaneous-proxy-connections-pyro-seems-to-freeze, e.g:

• Add pyro connection release timeout
• Add pyro connection retry/auto-reconnect
• Adjust the connection thread pool cap to match the maximum possible number of execution threads.
• Add manual cleanup for pyro connection on thread/process termination on the block executor.

Testing 🔍

Note

Only for the new autogpt platform, currently in autogpt_platform/

• Create from scratch and execute an agent with at least 3 blocks
• Import an agent from file upload, and confirm it executes correctly
• Upload agent to marketplace
• Import an agent from marketplace and confirm it executes correctly
• Edit an agent from monitor, and confirm it executes correctly

Configuration Changes 📝

Note

Only for the new autogpt platform, currently in autogpt_platform/

If you're making configuration or infrastructure changes, please remember to check you've updated the related infrastructure code in the autogpt_platform/infra folder.

Examples of such changes might include:

• Changing ports
• Adding new services that need to communicate with each other
• Secrets or environment variable changes
• New or infrastructure changes such as databases

[codiumai-pr-agent-pro]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Potential Resource Leak The close_service_client function only handles PyroClient instances. There might be a need to handle cleanup for other types of clients to prevent resource leaks. Configuration Validation The new Pyro configuration settings are directly assigned without validation. Consider adding checks to ensure the values are within acceptable ranges. Error Handling The calculation of maximum_connection_thread_count doesn't include error handling for potential division by zero or negative values.