Skip to content

OCPBUGS-6661: Manage mTLS CRLs #447

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rfredette wants to merge 1 commit into from
Closed

OCPBUGS-6661: Manage mTLS CRLs #447

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions hack/generate-placeholder-crl.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/bin/bash
# Generate a placeholder CRL, like the one included in pkg/router/crl/crl.go as dummyCRL. In order to create an already
# expired CRL, this script requires the utility faketime, which is provided by libfaketime on fedora.
set -e
cwd=$(dirname $0)
tmpdir=$(mktemp -d)
sed -e "s@%%tmpdir%%@${tmpdir}@" ${cwd}/placeholder-ca.cnf.template > ${tmpdir}/placeholder-ca.cnf
openssl genrsa -out ${tmpdir}/placeholder-ca.key 2048
openssl req -new -key ${tmpdir}/placeholder-ca.key -out ${tmpdir}/placeholder-ca.csr -subj "/C=US/ST=NC/L=Raleigh/O=OS4/OU=Eng/CN=Placeholder CA"
openssl x509 -req -in ${tmpdir}/placeholder-ca.csr -out ${tmpdir}/placeholder-ca.crt -days 3650 -signkey ${tmpdir}/placeholder-ca.key -extfile ${tmpdir}/placeholder-ca.cnf
cat /dev/null > ${tmpdir}/placeholder-crl-index.txt
faketime 'Jan 1, 2000 12:00AM GMT' openssl ca -gencrl -crlhours 1 -out ${tmpdir}/placeholder-ca.crl -config ${tmpdir}/placeholder-ca.cnf

echo "new placeholder crl at ${tmpdir}/placeholder-ca.crl" >&2
cat ${tmpdir}/placeholder-ca.crl
18 changes: 18 additions & 0 deletions hack/placeholder-ca.cnf.template
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid, issuer
basicConstraints=CA:TRUE

[ca]
default_ca=placeholder_ca

[placeholder_ca]
authorityKeyIdentifier=keyid,issuer
certificate=%%tmpdir%%/placeholder-ca.crt
database=%%tmpdir%%/placeholder-crl-index.txt
default_crl_hours=1
default_md=sha256
private_key=%%tmpdir%%/placeholder-ca.key
crl_extensions=crl_exts

[crl_exts]
authorityKeyIdentifier=keyid, issuer
2 changes: 1 addition & 1 deletion images/router/haproxy/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ RUN INSTALL_PKGS="haproxy22 rsyslog sysvinit-tools" && \
rpm -V $INSTALL_PKGS && \
yum clean all && \
mkdir -p /var/lib/haproxy/router/{certs,cacerts,whitelists} && \
mkdir -p /var/lib/haproxy/{conf/.tmp,run,bin,log} && \
mkdir -p /var/lib/haproxy/{conf/.tmp,run,bin,log,mtls} && \
touch /var/lib/haproxy/conf/{{os_http_be,os_edge_reencrypt_be,os_tcp_be,os_sni_passthrough,os_route_http_redirect,cert_config,os_wildcard_domain}.map,haproxy.config} && \
setcap 'cap_net_bind_service=ep' /usr/sbin/haproxy && \
chown -R :0 /var/lib/haproxy && \
Expand Down
2 changes: 1 addition & 1 deletion images/router/haproxy/Dockerfile.rhel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ RUN INSTALL_PKGS="haproxy22 rsyslog sysvinit-tools" && \
rpm -V $INSTALL_PKGS && \
yum clean all && \
mkdir -p /var/lib/haproxy/router/{certs,cacerts,whitelists} && \
mkdir -p /var/lib/haproxy/{conf/.tmp,run,bin,log} && \
mkdir -p /var/lib/haproxy/{conf/.tmp,run,bin,log,mtls} && \
touch /var/lib/haproxy/conf/{{os_http_be,os_edge_reencrypt_be,os_tcp_be,os_sni_passthrough,os_route_http_redirect,cert_config,os_wildcard_domain}.map,haproxy.config} && \
setcap 'cap_net_bind_service=ep' /usr/sbin/haproxy && \
chown -R :0 /var/lib/haproxy && \
Expand Down
2 changes: 1 addition & 1 deletion images/router/haproxy/Dockerfile.rhel8
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ RUN INSTALL_PKGS="haproxy22 rsyslog procps-ng util-linux" && \
rpm -V $INSTALL_PKGS && \
yum clean all && \
mkdir -p /var/lib/haproxy/router/{certs,cacerts,whitelists} && \
mkdir -p /var/lib/haproxy/{conf/.tmp,run,bin,log} && \
mkdir -p /var/lib/haproxy/{conf/.tmp,run,bin,log,mtls} && \
Comment thread
rfredette marked this conversation as resolved.
touch /var/lib/haproxy/conf/{{os_http_be,os_edge_reencrypt_be,os_tcp_be,os_sni_passthrough,os_route_http_redirect,cert_config,os_wildcard_domain}.map,haproxy.config} && \
setcap 'cap_net_bind_service=ep' /usr/sbin/haproxy && \
chown -R :0 /var/lib/haproxy && \
Expand Down
10 changes: 6 additions & 4 deletions images/router/haproxy/conf/haproxy-config.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@
{{- $dynamicConfigManager := .DynamicConfigManager }}
{{- $router_ip_v4_v6_mode := env "ROUTER_IP_V4_V6_MODE" "v4" }}
{{- $router_disable_http2 := env "ROUTER_DISABLE_HTTP2" "false" }}
{{- $haveClientCA := .HaveClientCA }}
{{- $haveCRLs := .HaveCRLs }}


{{- /* A bunch of regular expressions. Each should be wrapped in (?:) so that it is safe to include bare */}}
Expand Down Expand Up @@ -290,8 +292,8 @@ frontend fe_sni
{{- "" }} crt-list /var/lib/haproxy/conf/cert_config.map accept-proxy
{{- with (env "ROUTER_MUTUAL_TLS_AUTH") }}
{{- "" }} verify {{. }}
{{- with (env "ROUTER_MUTUAL_TLS_AUTH_CA") }} ca-file {{. }} {{ else }} ca-file /etc/ssl/certs/ca-bundle.trust.crt {{ end }}
{{- with (env "ROUTER_MUTUAL_TLS_AUTH_CRL") }} crl-file {{. }} {{ end }}
{{- if $haveClientCA }} ca-file /var/lib/haproxy/mtls/latest/ca-bundle.pem {{ else }} ca-file /etc/ssl/certs/ca-bundle.trust.crt {{ end }}

@gcs278 gcs278 Mar 21, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a thought, /var/lib/haproxy/mtls/latest/ca-bundle.pem could also be GO template variables, since they are already defined in crl.go

{{- if $haveCRLs }} crl-file /var/lib/haproxy/mtls/latest/crls.pem {{ end }}
Comment on lines +295 to +296

@Miciah Miciah May 16, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You replaced spaces with tabs. Please use spaces for consistency with the rest of the template.

I'm wary of breaking OKD users, or people using custom router deployments (unsupported, but better not to break them if we don't need to). If ROUTER_MUTUAL_TLS_AUTH_CA or ROUTER_MUTUAL_TLS_AUTH_CRL is set, we should still respect it. Likewise for ROUTER_MUTUAL_TLS_AUTH_CRL below.

Suggested change
{{- if $haveClientCA }} ca-file /var/lib/haproxy/mtls/latest/ca-bundle.pem {{ else }} ca-file /etc/ssl/certs/ca-bundle.trust.crt {{ end }}
{{- if $haveCRLs }} crl-file /var/lib/haproxy/mtls/latest/crls.pem {{ end }}
{{- with (env "ROUTER_MUTUAL_TLS_AUTH_CA") }} ca-file {{ . }} {{- else }}{{ if $haveClientCA }} ca-file /var/lib/haproxy/mtls/latest/ca-bundle.pem {{- else }} ca-file /etc/ssl/certs/ca-bundle.trust.crt {{- end }}{{ end }}
{{- with (env "ROUTER_MUTUAL_TLS_AUTH_CRL") }} crl-file {{ . }} {{- else }}{{ if $haveCRLs }} crl-file /var/lib/haproxy/mtls/latest/crls.pem {{- end }}{{ end }}

Likewise for ROUTER_MUTUAL_TLS_AUTH_CRL below.

@rfredette rfredette May 16, 2023

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My router changes still use ROUTER_MUTUAL_TLS_AUTH_CA since the operator still needs to supply the CA bundle, so using with (env "ROUTER_MUTUAL_TRLS_AUTH_CA") by itself to select the old behavior isn't specific enough.

What do you think of something like this?

Suggested change
{{- if $haveClientCA }} ca-file /var/lib/haproxy/mtls/latest/ca-bundle.pem {{ else }} ca-file /etc/ssl/certs/ca-bundle.trust.crt {{ end }}
{{- if $haveCRLs }} crl-file /var/lib/haproxy/mtls/latest/crls.pem {{ end }}
{{- with (env "ROUTER_MUTUAL_TLS_AUTH_CRL") }} crl-file {{ . }}
{{- with (env "ROUTER_MUTUAL_TLS_AUTH_CA") }} ca-file {{ . }} {{- else }} ca-file /etc/ssl/certs/ca-bundle.trust.crt {{ end }}
{{- else }}
{{- if $haveClientCA }} ca-file /var/lib/haproxy/mtls/latest/ca-bundle.pem {{- else }} ca-file /etc/ssl/certs/ca-bundle.trust.crt {{ end }}
{{- if $haveCRLs }} crl-file /var/lib/haproxy/mtls/latest/crls.pem {{ end }}
{{- end}}

@Miciah Miciah May 17, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That could work (though I think you also need to re-add {{- with (env "ROUTER_MUTUAL_TLS_AUTH_CRL") }} crl-file {{. }} {{ end }}).

@rfredette rfredette May 17, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix I proposed does include crl-file ... in that case, but maybe it's a bit confusing. I'll see if I can make it clearer.

@rfredette rfredette May 17, 2023

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've included a clearer version of this fix in #472.

{{- end }}
mode http

Expand Down Expand Up @@ -376,8 +378,8 @@ frontend fe_no_sni
bind unix@/var/lib/haproxy/run/haproxy-no-sni.sock ssl crt {{ firstMatch ".+" .DefaultCertificate "/var/lib/haproxy/conf/default_pub_keys.pem" }} accept-proxy
{{- with (env "ROUTER_MUTUAL_TLS_AUTH") }}
{{- "" }} verify {{. }}
{{- with (env "ROUTER_MUTUAL_TLS_AUTH_CA") }} ca-file {{. }} {{ else }} ca-file /etc/ssl/certs/ca-bundle.trust.crt {{ end }}
{{- with (env "ROUTER_MUTUAL_TLS_AUTH_CRL") }} crl-file {{. }} {{ end }}
{{- if $haveClientCA }} ca-file /var/lib/haproxy/mtls/latest/ca-bundle.pem {{ else }} ca-file /etc/ssl/certs/ca-bundle.trust.crt {{ end }}
{{- if $haveCRLs }} crl-file /var/lib/haproxy/mtls/latest/crls.pem {{ end }}
{{- end }}
mode http

Expand Down
Loading