Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scrape fail when target cert contains multiple 'Subject Alternative Name' #9403

Open
hatharom opened this issue Sep 27, 2021 · 3 comments · May be fixed by prometheus/common#350
Open

Scrape fail when target cert contains multiple 'Subject Alternative Name' #9403

hatharom opened this issue Sep 27, 2021 · 3 comments · May be fixed by prometheus/common#350

Comments

@hatharom
Copy link

hatharom commented Sep 27, 2021

What did you do?

I tried to scrape a TLS protected target which has multiple 'Subject Alternative Name' in its cert

What did you expect to see?
Scrape works without any error

What did you see instead? Under which circumstances?
An error on the Prometheus target page:
2021-09-27T09:23:44,312+00:00 WARN [SparkUI-43] org.sparkproject.jetty.server.SecureRequestCustomizer: Host 172.17.0.8 does not match SNI X509@47daa1b2(my-cert,h=[domain1, domain2],w=[])

Environment

  • Prometheus version: 2.15.2

  • Prometheus configuration file:

    - job_name: tls-target
      scheme: https
      tls_config:
        ca_file: /run/secrets/cacert/ca.pem
        cert_file: /run/secrets/clicert/cert.pem
        key_file: /run/secrets/clicert/key.pem
        server_name: domain1
      kubernetes_sd_configs:
        - role: pods
    
  • Server cert relevant part

            X509v3 Subject Alternative Name:
                DNS:domain1, DNS:domain2
  • Logs:
    - No logs on Prometheus side
    - Log message on target side :2021-09-27T09:23:44,312+00:00 WARN [SparkUI-43] org.sparkproject.jetty.server.SecureRequestCustomizer: Host 172.17.0.8 does not match SNI X509@47daa1b2(my-cert,h=[domain1, domain2],w=[])

What works

  1. Either curling the target by hand https://domain1:4440/metrics/prometheus. It gives back the metrics successfully
  2. Or removing one SAN entry from the server cert(In this case Prometheus can scrape the metrics successfully):
            X509v3 Subject Alternative Name:
                DNS:domain1
@hatharom hatharom changed the title Scrap fail when target cert contains multiple 'Subject Alternative Name' Scrape fail when target cert contains multiple 'Subject Alternative Name' Sep 27, 2021
@roidelapluie
Copy link
Member

I would need to double check the code, but it seems that we would be setting SNI without changing the Host: header. I will investigate.

@fpetkovski
Copy link
Contributor

fpetkovski commented Nov 29, 2021

@roidelapluie this seems to be related to the fact that net/http uses the request's Host field to set the Host header[1].

I am trying to get familiar with the Prometheus codebase and I have time this week to look into this one if that's okay.

[1] golang/go#29865

@bboreham
Copy link
Member

Hello from the bug-scrub!

It seems that some work was done; can you let us know whether it should go forward?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants