While Elastic recommends always using SSL/HTTPS, not all clients support HTTPS. A notable example of this is Logstash. Also, some clients have poor support, because they do not actually verify the certificate, for example.
To work around this, you can use a proxy or a tunnel. The application without HTTPS support can connect to a local endpoint using HTTP, which in turn connects to a Elastic Cloud cluster using HTTPS.
stunnel is tool that can be used to provide secure encrypted connections for clients or servers that do not speak TLS or SSL natively.
We can use stunnel to bind to a port on localhost (e.g., 19200), which in turn will connect to Elastic Cloud.
Here is a sample configuration to achieve this:
; Actually verify the certificate verify = 2 ; Works for Ubuntu. Adapt to your system. CApath = /etc/ssl/certs pid = /var/run/stunnel4/found-us-east-1.pid ; Log level. WARN=4, DEBUG=7 debug = 4 [foundcluster] ; Service that tunnels traffic to a single region's endpoint. This configuration is not cluster specific. accept = 19200 client = yes ; Don't cache DNS. IPs of Elastic Cloud's load balancers may change. delay = yes ; Replace us-east-1 with your region. Valid hosts: ; - proxy-v1-us-east-1.foundcluster.com ; - proxy-v1-us-west-1.foundcluster.com ; - proxy-v1-eu-west-1.foundcluster.com ; - proxy-v1-sa-east-1.foundcluster.com ; - proxy-v1-ap-northeast-1.foundcluster.com ; - proxy-v1-ap-southeast-1.foundcluster.com connect = proxy-v1-us-east-1.foundcluster.com:9243
To use this with Ubuntu:
-
Install the
stunnel4-package, e.g.apt-get install stunnel4. -
Put the above file in
/etc/stunnel/found-us-east-1.conf -
Make sure
/etc/default/stunnel4containsENABLED=1 -
Run
service stunnel4 start
Then you will have a service that listens to port 19200 and forwards traffic to proxy-v1-us-east-1.foundcluster.com:9243.
Now we have a service that listens to port 19200 and tunnels traffic to Elastic Cloud, while making sure the traffic is encrypted and that the certificate is valid.
However, if you try to send HTTP requests to http://localhost:19200, no information is forwarded about what cluster requests must be routed to.
Typically, endpoints are on the form [cluster id]-[region].foundcluster.com. The hostname is what specifies what cluster requests will be routed to.
A compatible hostname must be sent to Elastic Cloud, but we still want to connect to 127.0.0.1.
To do this, we can use xip.io. By connecting to http://[your-cluster-id]-[region].127.0.0.1.xip.io:19200, e.g. http://7893883873a705aec69e2942901f20d7b1e28dec-us-east-1.127.0.0.1.xip.io, the hostname will resolve to 127.0.0.1 and the hostname with the cluster id will be sent to Elastic Cloud.
Elastic Cloud then has sufficient information to route requests to your cluster.
While this hostname is not matching that of the SSL certificate (which matches *.foundcluster.com), it is stunnel that is terminating the SSL-connection. stunnel knows nothing about HTTP, and the client connecting to 19200 is completely unaware that traffic is being tunneled through SSL.
Note that stunnel will establish a new SSL-connection for every client that connects to it. It is important to use persistent connections, even though you are connecting to localhost. If not, you will be spending a lot of time establishing SSL-connections!
A common use case for this is when running Logstash outside AWS infrastructure, while wanting to use a cluster managed by Elastic Cloud.
Assuming you have configured stunnel appropriately, you can then use the elasticsearch_http-output of Logstash, and point it to localhost. (While you can specify protocol => http with the elasticsearch-output, the latter does not support basic authentication at the moment.)
Here is an example:
elasticsearch_http {
hosts => "7893883873a705aec69e2942901f20d7b1e28dec-us-east-1.127.0.0.1.xip.io"
user => "admin"
password => "yourpassword"
manage_template => false
port => 19200
}