High latency due to dns refresh on https every x minutes

[waszak]

Elastic.Clients.Elasticsearch version: 8.15.8

Elasticsearch version: 8.14.0

.NET runtime version: 8.x

Operating system version: Linux

Description of the problem including expected versus actual behavior:
Depending on parameter DnsRefreshTimeout we see longer requests every X minutes.
I attached example with settings 10 minutes and 5 minutes so there is strong correlation

The issue doesn't occure on HTTP connection its only on HTTPS.
Also this is new index/cluster so there is not many request to it yet.

I would provide more diagnostic details if you tell me what to check. All I know its caused by Https and DNS refresh
but i just can't find any proof its due to client or some network configuration issue. ever
We had no issue like that with older elastic versions like 6

Steps to reproduce:
0. Its hard to reproduce

1. There should be enabled https
2. DnsRefreshTimeout set to some value like 5 minutes
3. .Run the same request in loop for idk 1h and you will see huge spikes every 5 minutes

Expected behavior
No timeouts/long requests during queries to elatic

Provide ConnectionSettings (if relevant):

Provide DebugInformation (if relevant):

[flobernd]

Hi @waszak, I'll transfer this issue over to the https://github.com/elastic/elastic-transport-net repository.

[flobernd]

When the DNS refresh happens, the underlaying HttpMessageHandler is recreated (to make sure any subsequent request is correctly routed to the new server after a potential DNS update). This involves establishing a new connection and a higher latency is definitely expected.

HTTPS introduces massive overhead over plain HTTP as it has to perform a TLS handshake. This again involves cryptographic operations which are not cheap.

Another factor could be a proxy that intercepts SSL traffic for deep packet inspection or any firewall in general.

Is this latency spike problematic for your concrete usecase? If I read your (second) graph correctly, the duration for a complete request is still below 350ms.