Port is not properly set in requests, causing No alive nodes found in your cluster

[larsnystrom]

Summary of problem or feature request

The elasticsearch-php library does not set the correct Host header when using a custom port.

I was getting "No alive nodes found in your cluster" when performing a query using elasticsearch-php, but everything was working fine when doing the same request using curl on the command line.

After setting CURLOPT_VERBOSE => 1 in the script and using the -v flag in the CLI, I noticed that the HTTP Host header was what differed between the elasticserach-php request and the CLI request. The CLI request set the Host header to elastic.example.com:9243 but the elasticsearch-php client set the Host header to elastic.example.com, even though the request was still made to the correct port on the TCP layer.

The missing port number in the Host header led to the server sending an empty response (cURL error 52: Empty reply from server).

The elasticsearch cluster is running HTTPS, on port 9243, with HTTP basic auth. This bug applies both when configuring the host (ClientBuilder::setHosts()) using a string such as https://user:[email protected]:9243 as well as when configuring the host with an array containing the same information in separate fields (scheme, host, port, etc).

Edit: After looking a bit closer at the other tickets (and having lunch...), I realized that this is closely related to #925, which also explains that this bug was introduced in #782. The author of #782 says that "having the port in the host header is optional", but as @polyfractal says a bit earlier in the same thread this is not really the case if you read the RFC. Elasticsearch uses the HTTP protocol which means the implied port will be 80, or in the case of HTTPS 443. You cannot omit it and still stay compliant with HTTP/S, and if you're not compliant with HTTP/S then all bets are off when it comes to configuration of proxies, web servers, etc, which invalidates the reason for introducing the PR in the first place.

Code snippet of problem

The problem is fixed by changing line 208 in Elasticsearch\Connections\Connection (in the performRequest method) from

'Host' => [$this->host]

to

'Host' => ["{$this->host}:{$this->port}"]

System details

• Operating System: Ubuntu 16.04
• PHP Version: 7.4.2
• ES-PHP client version: 7.4.1
• Elasticsearch version: 7.5.0

[ezimuel]

@larsnystrom I'm trying to reproduce the issue but I don't see any issue. Did you try the following configuration?

$client = ClientBuilder::create()
    ->setBasicAuthentication('user', 'pass')
    ->setHosts(['https://elastic.example.com:9243'])
    ->build();

or

$client = ClientBuilder::create()
    ->setBasicAuthentication('user', 'pass')
    ->setHosts([
        [
            'host' => 'elastic.example.com',
            'port' => 9243,
            'scheme' => 'https'
        ]
    ])
    ->build();

Let me know, thanks!

[larsnystrom]

It does not matter which of those configurations you use. Here is a test script as well as some actual output when I ran it. The only thing I have changed here is the domain name (example should be something else) and the IP and subject line concerning the SSL certificate in the output.

I have added both of the configuration variations to the test script so that it is easy to switch between them, but as you can see I'm only using one of them at a time.

<?php

use Elasticsearch\ClientBuilder;

require __DIR__ . '/vendor/autoload.php';

$host = [
    'scheme' => 'https',
    'host' => 'something.elastic.example.com',
    'port' => 9243,
    'user' => 'me',
    'pass' => 'pass',
];
$altHost = "{$host['scheme']}://{$host['user']}:{$host['pass']}@{$host['domain']}:{$host['port']}/";

$client = ClientBuilder::create()
    ->setHosts([
        $host,
    ])
    ->setConnectionParams([
        'client' => [
            'curl' => [
                \CURLOPT_VERBOSE => 1,
            ]
        ]
    ])
    ->build();

$client->get([
    'index' => 'myindex',
    'id' => 'some-id',
]);

Save it as test.php and run it like php test.php. Here's my output:

* Connected to something.elastic.example.com (12.34.56.78) port 9243 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: .....
*  start date: Nov 13 00:00:00 2019 GMT
*  expire date: Nov 17 12:00:00 2021 GMT
*  subjectAltName: host "something.elastic.example.com" matched cert's "*.elastic.example.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
* Server auth using Basic with user 'me'
> GET /myindex/_doc/some-id HTTP/1.1
Host: something.elastic.example.com
Authorization: Basic bWU6cGFzcw==
Content-Type: application/json
Accept: application/json
User-Agent: elasticsearch-php/7.4.1 (Linux 4.15.0-76-generic; PHP 7.4.2)

* Empty reply from server
* Connection #0 to host something.elastic.example.com left intact

The issue here is Host: something.elastic.example.com which should be Host: something.elastic.example.com:9243.

[ezimuel]

@larsnystrom ok, this is basically the opposite issue of what @afrozenpeach said here regarding the port specified in Host header with HAProxy.
Are you using a load balancer/proxy before the Elasticsearch server? I tested locally using a different port (9500) for Elasticsearch and everything works fine.
I'm thinking that maybe we can add an option to ClientBuilder to enable the port in the Host header. WDYT?

[larsnystrom]

There is a load balancer/proxy in front of the Elasticsearch server, but I don't know its name. That's where the request is dropped due to the missing port.

An option seems reasonable. Personally I think it should default to true, but since that would break backwards compatibility such a change should probably wait until the next major version. An includePortInHostHeader option which defaults to false would probably be the best course of action right now.

And sorry if I came off a bit harsh at the beginning of this issue.

[ezimuel]

@larsnystrom thanks for your feedback! I'll add a ClientBuilder::setPortHostHeader(bool $enable) that is false by default for backwards compatibility. Btw, after reading RFC 2616 - Section 14.23 I realized that the port is optional only if it's the default one (80 for HTTP and 443 for HTTPS). In the other case, it should be added as reported also in this discussion. I'll definitely use this approach for 8.0 release, adding the port as default option.

I'm preparing a PR for this and I'll ask you to review, thanks again!

[ezimuel]

@larsnystrom I just created PR #997, if you have time can you review it? Thanks!