-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Memory leak in REP socket handling #2567
Comments
Solution: roll back the pipe if writing messages other than the first fails in router::xsend. Also add test case that reproduces the memory leak when ran with valgrind. Fixes zeromq#2567
This is a very good analysis, thank you. I've managed to create a simple and self contained test case that reproduces the problem both locally and on Travis. I think the solution is to call rollback () on current_out. Once the CI is green I'll open a PR. |
@bluca Thanks for preparing the patch, I'll merge it to my company's libzmq fork, and let it run for the next weekend. I'll keep you posted here. BTW. any plans to release 4.2.3 in near future? |
Yes it's almost time, but there's a couple of things that need fixing first (cmake and ipv6 related) |
The patch seems working for me. You may close the issue once PR #2572 is merged. Thanks for help! |
Great, thanks for confirming |
Solution: roll back the pipe if writing messages other than the first fails in router::xsend. Roll it back also when the pipe is terminating. Also add test case that reproduces the memory leak when ran with valgrind. Fixes zeromq#2567
Problem happens in a REP server, when a REQ client disconnects before the response is sent. According to valgrind, about 500B leaks per response on my test setup. I've discovered the problem in ZMQ 4.2.1, then confirmed also in master and 4.1.6.
The app I'm working on is a simple REQ/REP server, working on Linux, using tcp/ipc protocols. It occasionally requires significant amount of time to process the request, due to external hardware problems, during which our client timeouts and leaves the server with small, but growing leak.
I've traced the problem to a partial message being left in a pipe:
I suppose the session_base_t::clean_pipes() should remove them in step "3", however it does not happen for some reason... Unfortunately, I don't have enough time to get familiar with libzmq code, so I cannot came with a patch at hand... But I would gratefully test one.
Please see attached valgrind output, and a minimal server + client test apps for reproducing the issue (my real server/client uses czmq 4 and pyzmq 14.x, however that should not matter much). The sample server is hardcoded with "processing" time (just a sleep) set to 1s, and the sample client has RCVTIMEO set to 0.5s. With such values I get the memleak every request. When receive timeout is raised enough that client reads the response, the problem stops occurring (obviously).
libzmq_valgrind.txt
testapps.zip
The text was updated successfully, but these errors were encountered: