Override "Bad" SSL Certificate Rejections

[Lonniebiz]

deno 1.0.0
v8 8.4.300
typescript 3.9.2

In Deno, is there already a way to override the pedantic scrutiny that often causes self-signed SSL certificates to be deemed "Bad Certificates"?

It seems that Deno security is so high that it blocks self-signed SSL certificates for vague reasons.

Not all SSL certificates should be subjected to international banking standards. There are valid reasons for creating self-signed SSL certificates with (practically) never expiring expiration dates. Too many security zealots automatically frown upon self-signed certificates, but hear me out.

Let's say you have two servers on a LAN, that serve each other with separate APIs. And, the only security you want, is to have encrypted data transfer between those two servers and nothing else. And, let's say these services (that these two servers provide) do NOT require internet to function.

Let me ask you this. Why the hell would you want to authenticate the SSL certificate with a 3rd party internet resource? The internet itself now becomes an unneeded external dependency that could potentially bring your service to a halt if the internet goes down (for a service that otherwise needs zero internet).

Well that's exactly what can happen, if deno has no way to override errors like this:

The above script and certificates are zipped here: testDeno.zip

The ssl certificates were generated with a command like this:
openssl req -nodes -new -x509 -keyout private.key -out server.cert -days 1825000 -subj "/C=US/ST=Texas/L=Houston/O=ChickenPeck Technologies/OU=Org/CN=localhost"

I suspect the certificate is bad due to the extremely far off date to which it expires. It would be nice if the error went into more detail about the exact reason the certificate is a "bad certificate".

However, even if that is the reason, I'd like the ability to override and ignore such pedantic scrutiny. I'm not going to use something that inconveniences me with a mandatory security level that is beyond what I want it to be.

Also, why would you want to schedule, on your calendar, a date (one year from now) when your service will crash due to an expired certificate? Instead, I want the service I created to live on without interruption, without any certificate update maintenance, until I decide I want to update it. As the developer of these services, I should be able to prioritize "uninterruptible" over "unnecessarily secure".

Is there already a way to override these type of certificate rejections? If not, I'm requesting it.

[Caesar2011]

Continue to talk in issue #5260 . Consider to close this issue because of being a duplicate. :)

[kitsonk]

Yeah it is duplicate of #5260.

[Lonniebiz]

Isn't the focus of this issue slightly different than #5260?

My focus here is not to merely catch the exception or make the error message more specific about the exact reason for the rejected certificate. To me, it seems that catching an error like this (alone) doesn't necessarily mean you'd be able to override it to the point that Deno would consume the "bad certificate".

My main aim here, is to allow the developer to decide if Deno's reason for the certificate rejection is valid for the developer's use-case, and if it isn't, give the developer the ability to force Deno to use what Deno considers to be a "Bad Certificate".

For example, if the certificate is a "Bad Certificate" because it has an expiration date too far in the future, or because it is self-signed, or because it cannot be authenticated by a 3rd party internet resource, or because it has expired: Those are reasons that the developer should be able to preemptively make deno ignore, so that no exception even occurs. That's different from #5260.

To me, even though this issue is related to #5260, it seems to focus on priorities that would remain an issue even after #5260 is fixed.

[kitsonk]

A "Bad Certificate" is not a subjective thing that is left up to some code running on the server. It is something is part of the security toolchain of the local host. I agree it is worth investigating how to get self signed certificates working, but it isn't something that should be decided in runtime JavaScript code.

[Lonniebiz]

I disagree. A "Bad Certificate" is indeed a very subjective thing when it comes to runtime-javascript. Especially when you consider some of the reasons that a certificate is deemed bad:

subjective #1: A certificate is bad if it has an expiration date greater than one year.

subjective #2: A certificate is bad if it has expired (and this should halt a running service).

subjective #3: A certificate is bad if it is self-signed.

subjective #4: A certificate is bad if it cannot be authenticated by a 3rd party internet resource.

As a developer, who doesn't want any of those reasons to halt my HTTPS services, I find those reasons to be very subjective and I do indeed want to override them with server-side javascript at runtime.

And by the way, in Node.js, I'm able to run the very same certificates, that are being rejected by deno, to achieve uninterrupted services.

I'm also able to ignore bad certificates during fetch operations using an agent in node-fetch:
const httpsAgent = new https.Agent({ keepAlive: true, rejectUnauthorized: false });

The 1.0 Deno Release is just a toy to me, until I can do these same things in deno.

[bertvanbrakel]

I agree. I'm using self signed certs in local/dev/test, on a kubernetes cluster where the certs are generated at runtime, specific to the developers machine. I have my own self signed certs I want to use for that purpose.

My issue is on the consuming client side for now. Surely a callback handler where one can return a true/false whether to accept a cert would do? And maybe log a warning to the logs about it (so it's very obvious)

[Cretezy]

Facing this same issue while trying to write tests for a server. I'm using a self-signed certificate. It would be nice to have some sort of flag along side certFile to allow self-signed certs for these purposes (maybe named insecureAllowBadCertificates or something that conveys that this can be insecure)

[callionica]

I'm trying to write a client that consumes the Philips Hue API from Signify (a home automation API for controlling lighting). The API is accessed over https to hardware (the "Hue bridge") running on the local network. The Hue bridge uses a self-signed certificate. I've got no control over the certificate the bridge uses. Deno already can't handle sending requests to IP addresses, but Deno also doesn't like the self-signed cert.

I get this error when making a request to a Hue bridge using its host name:

Sending fatal alert BadCertificate
error: Uncaught (in promise) Error: error sending request for url (https://.../): error trying to connect: invalid certificate: UnknownIssuer 

I can get a web browser to accept the self-signed certificate (with some guidance to the user), but apparently I can't get Deno to accept it, so this is another vote to get fetch requests to servers that use self-signed certificates working in Deno.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Lonniebiz]

Auto-closing "stale" issues seems like bad github policy configuration to me. This is still an open issue and should not be auto-closed due to neglect.

[lucacasonato]

@Lonniebiz Thanks for the heads up. We have a lot of issues, so it is hard to keep track of old, actually stale issues. I have labeled the issue correctly now, so it won't be marked as stale again.

[bartlomieju]

Discussed with @lucacasonato and @ry and we're going to implement this request; I think we will go with --no-check-certificate flag.

To implement this feature we need to use two APIs:

• https://docs.rs/reqwest/0.11.0/reqwest/struct.ClientBuilder.html#method.danger_accept_invalid_certs - for HTTP client
• https://docs.rs/rustls/0.19.0/rustls/struct.DangerousClientConfig.html#method.set_certificate_verifier - for server

[lucacasonato]

I think the flag should really contain the word unsafe. It should be very clear to users that this is a last resort, and should generally not be used. Also not clear that this flag means tls certificates.

[callionica]

When you say "flag" do you mean that each process gets to choose only whether all certificates are validated or none? If so, this would be less secure than ideal because a single process will typically need to access both fully validated certs from the web as well as self-signed certs that need custom validation. Ideally, it would be something that could be controlled for each request.

Can I refer you to #8238, where I describe the features needed for public key pinning and also include the names of all the relevant CURL flags, in case they are useful to inform the naming.

[Lonniebiz]

@callionica I'm not requesting a global override for all bad certificates in a process. More granularly, I'm requesting the ability to override on a per https service basis.

If you look at my example, you'll see that I wasn't even able to run an https web service with deno using a self-signed certificate; Deno considered my self-signed certificate "bad", and it crashed the service immediately and permanently.

As the developer of my service, it is me who decides if a certificate is "bad" or not. And any framework, that doesn't give me the low-level discretion to decide that, is not one I'll be using. Not all web services are public facing, and if I want to use self-signed certificates (without any outside 3rd party validation), I need a framework that will allow that.

That's what this issue is about. However, let's say I make a webservice that makes screenshots of web pages when the user provides the URL. That's the other end of the spectrum. But, here too, I don't care if the web server I'm connecting to has a valid certificate or not. I'll accept any certificate that will transfer the page over https, as long as it allows me to grab a screenshot of that web page. Here again is an example where I'd want the ability to override any exception deno would have with a "bad" certificate. In this case, I'd be overriding bad certificates in my fetch options object.

So, whether dealing with requests or responses, the underlying framework should always provide the developer some options for overriding "bad" certificate exceptions.

[callionica]

Sorry I was unclear, @Lonniebiz: my comment was intended for the Deno devs who are suggesting a flag named --no-check-certificate, which looks like a command line flag to me. My comment was an attempt to say that a command line flag might not be sufficient: it wouldn't work very well if all certificates in a process stop being checked, and it would also be inconvenient for IOT uses if the command line had to include a list of hosts to stop checking certificates for, since we don't know what the list of hosts is at installation time (although I guess that could be worked around with a multi-process architecture).

Anyway, you and I are in agreement that "the underlying framework should provide the developer some options for overriding "bad" certificate exceptions".

[remdragon]

I also need this functionality. I'd like to propose that a 2-pronged approach is probably needed here:

1. a flag like --allow-unsafe-certs which enables the ability to accept unsafe certificates.
2. a fetch() parameter to allow an unsafe cert on that connection.

This way both the user and the code have to opt-in to self-signed certs before they start working. You don't lose safety on the connections where you want it and the code can't blindly accept unsafe certs without the user enabling it.

[zinthose]

Would this also address the issue presented in #6427
Being unable to consume certain 3rd party APIs affected by this is a deal-breaker for me.

[roseMix]

Are there any updates on this, or a date/version it will be included in @bartlomieju ?

Sorry for asking, just there have been quite a few releases since you said it will be implemented and no mention of this, just wondering if its still on the roadmap.

[mbrevoort]

+1 I'm blocked by this issue on using deno in a somewhat complex dev environment. Perhaps --no-check-certificate could take an optional list of hostnames like --allow-net. For example, --no-check-certificate=dev1966.foo.com. I prefer this approach over an option to fetch() or some other in-code configuration in the case where you need to run untrusted code.

[Lonniebiz]

@mbrevoort I like in-code options, because they allow you to override on a case by case granularity. I also like what you're proposing because it would allow you to override for every instance of a hostname, generally. Also, lets not forget about ipv4 and ipv6 addresses where a hostname is not even used; I need the ability to override "bad" SSL Certificates for ip addresses too. Perhaps some people consider an ip address a hostname, but I wanted to mention it separately to make sure that the scenario gets covered.

[pseudosavant]

I want to second overriding on a case-by-case basis in code whether to accept untrusted connections. Deno's fetch implementation is unusable for fetching untrustable endpoints (e.g. scrapers, user defined URLs, etc) without this ability. Implementing accepting untrusted connections using a runtime flag (--no-check-certificate) is not a substitute for the in code capabiltiy.

[bartlomieju]

I want to second overriding on a case-by-case basis in code whether to accept untrusted connections. Deno's fetch implementation is unusable for fetching untrustable endpoints (e.g. scrapers, user defined URLs, etc) without this ability. Implementing accepting untrusted connections using a runtime flag (--no-check-certificate) is not a substitute for the in code capabiltiy.

We first need a capability to disable SSL verification via CLI flag before we can tackle this request.

[pseudosavant]

What changes are needed to do a CLI flag? Is it mostly in Rust or TS/JS? I don't know Rust but I can contribute TS/JS code.

[roseMix]

@pseudosavant looks like this is being worked on (see: #11324), also look at the end of issue #1371 where @bartlomieju added it to the roadmap for v13 set to be released Aug 10

[mbrevoort]

🎉 Nice!

[Lonniebiz]

@bartlomieju Does that one flag address both the server-side and the client-side (see 4th paragraph for client-side issue) of the problem?

[bartlomieju]

@bartlomieju Does that one flag address both the server-side and the client-side (see 4th paragraph for client-side issue) of the problem?

That change works for internal Deno stuff (like fetching imported files) as well as runtime APIs like fetch, Deno.connectTls, etc. So yes, it should work for the use case from 4th paragraph.

[Lonniebiz]

@bartlomieju What about this https service issue discussed at the top of this ticket? I see that someone has closed this ticket, yet nothing I've read explicitly indicates that the flag also addresses the issue described at the top of this ticket. In addition to fetching, does this same flag override "Bad" SSL certificates rejections of an https SERVICE that uses self-signed certificates (where the deno app is the server - NOT the client)?

[bartlomieju]

@bartlomieju What about this https service issue discussed at the top of this ticket? I see that someone has closed this ticket, yet nothing I've read explicitly indicates that the flag also addresses the issue described at the top of this ticket. In addition to fetching, does this same flag override "Bad" SSL certificates rejections of an https SERVICE that uses self-signed certificates (where the deno app is the server - NOT the client)?

Yes - when you use TLS it will cause to use one of the runtime Deno APIs that will be affected by this flag.

[bartlomieju]

Additionally there's a new env variable DENO_TLS_CA_STORE that you can use to configure TLS to use system CA store when veryifying certificates.