Connecting to nats server with tls enabled with certificate containig SAN as domain name, how to connect internally without tls or SAN verification.

[ekafe]

I have nats server running in k8 with tls enabled and tls certificate is created by lets-encrypt with our domain name, for communication from outside the server, providing domain name is nats connection is working given below.
nats://username:[email protected]:4224

In my use case, i have multiple pod inside the same k8 node, wants to connect to nats server, for now i am giving the same url above to connect, but the problem is everytime i have to provide the host aliasing to pod specifying the domain name and my server ip address.

In kubernetes u can connect to the pods using it service name, and i was trying to do that, but the problem, tls certificate it is checking the dns name, suppose if we give the connection url as nats://username:password@nats-service:4224.

i am getting the error tls: failed to verify certificate: x509: certificate is valid for *.app.local not nats-service

the above mentioned scenario sounds like a frequent scenario, because for internal k8 communication, i dont want the communication to go outside, (if we provide the domain name)

Is there any configuration in nats i am missing, or this is not possible for current nats server?

[amitamit281]

Hi

you need to add your k8s service url in certificate SAN list.

[ekafe]

yes, i have done this in local dev server, with my own ca certificate, problem is when i am deploying to outside with domain name, certificate is provided by ca, like lets-encrypt. it will be provided for that particular domain, i can't add extra url is SAN list. because ca doesnt allow.

If any other way is there for internet facing domain, help is appreciated.

[wallyqs]

What you can also do is to use two certs, one for public traffic and one for internal traffic then setup both in the tls config block:

tls {
      certs = [
        {
          cert_file: # public cert
          key_file:  # ...
        },
        {
          cert_file: # internal cert
          key_file:  # ...
        }
      ]
}

if you want to use the Let's Encrypt certs for internal traffic then you could set an dns entry to resolves to an static cluster ip, for example with a nats deployment like this:

$ kubectl -n examples get svc

NAME                        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                              
nats-tls-example            ClusterIP   10.245.177.2   <none>        4222/TCP,443/TCP
nats-tls-example-headless   ClusterIP   None           <none>        4222/TCP,443/TCP,6222/TCP,8222/TCP

then you could create a dns entry for that internal domain only:

dig  k8s.internal.nats.mydomain.dev

;; ANSWER SECTION:
k8s.internal.nats.mydomain.dev.	294	IN	A	10.245.177.2

your clients could use that domain to connect within the cluster network instead.

[ekafe]

ok, 2 certificate solution is good, I tried and its working, I don't want to do static cluster IP, if I am using multiple nodes, it will cause problem, but adding DNS entry with subdomain of the domain and assigning to cluster IP, that's clever.

I was thinking, how mosquito broker has a configuration where we can assign different port with TLS enabled and disabled, likewise some configuration I am missing. but given solutions is good.

Thank you guys for your help