[client-v2] Connection Disruption Issues (Timeout, Reset by Peer, No Response) #1994
Assignees
Labels
Comments
chernser
changed the title
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
There is set of issues while different stages of request. This issue may be not covers them all, but aimed to investigate and understand these issues.
Unknown
No response tests causes broken pipe exception when run locally:
Connection Request
Connection pool may be source of timeouts, too. There is connection request timeout that limits for how long client would wait for connection from a pool. There is connection establish timeout that is for time spent to create TCP connection after connection is leased from the connection pool.
Socket timeout
There is a socket timeout that affects write/read operations. It seems not set by default or set implicitly. This should be investigated. Socket timeout may be affected by request timeout if Apache HTTP Client is used.
Socket timeout is measured by the wall clock so it is affected by GC pause when application is under significant load. We need to document this to make users aware of the problem. Another question - can we detect it? May be if we have two timers for wall clock and ticker counter.
Fixed:
Example stack trace:
Pay attention that test elapsed time is
3.772 s. This means that somewhere very short timeout was set like around1sFor tests it may be a common problem when run against the cloud.
Connection Reset by Peer
Most probably timeout happens on a server side. Client may see it as a connection reset because server close socket without sending any error.
There is one of examples: #2070
We need to test long running queries.
Here is native code that causes exception https://github.com/openjdk/jdk17/blob/master/src/java.base/unix/native/libnio/ch/SocketDispatcher.c
It can be:
Reset exceptions are reaction to a TCP reset package, but read and write operation has different execution branches so exceptions may vary.
Example stack trace:
The text was updated successfully, but these errors were encountered: