Changing OpenStack endpoints from HTTP to HTTPS

After deploying OpenStack Keystone, Swift and Horizon I have a need to change the public endpoints for these services from HTTP to HTTPS.

Horizon endpoint

This deployment is a single server for Horizon. The TLS/SSL termination point is on the server (no loadbalancers or such).

To get Horizon using TLS/SSL all that needs to be done is adding the keys, cert, ca and updating the vhost. My vhost not looks like this:

WSGISocketPrefix run/wsgi
<VirtualHost *:80>
	ServerAdmin sys@test.mwclearning.com
	ServerName horizon-os.mwclearning.com.au
	ServerAlias api-cbr1-os.mwclearning.com.au
	ServerAlias api.cbr1.os.mwclearning.com.au
	ServerName portscan.assetowl.ninja
	Redirect permanent / https://horizon-os.mwclearning.com.au/dashboard
</VirtualHost>

<VirtualHost *:443>
	ServerAdmin sys@test.mwclearning.com
	ServerName horizon-os.mwclearning.com.au
	ServerAlias api-cbr1-os.mwclearning.com.au
	ServerAlias api.cbr1.os.mwclearning.com.au

	WSGIDaemonProcess dashboard
	WSGIProcessGroup dashboard
	WSGIScriptAlias /dashboard /usr/share/openstack-dashboard/openstack_dashboard/wsgi/django.wsgi
	Alias /dashboard/static /usr/share/openstack-dashboard/static

	<Directory /usr/share/openstack-dashboard/openstack_dashboard/wsgi>
		Options All
		AllowOverride All
		Require all granted
	</Directory>

	<Directory /usr/share/openstack-dashboard/static>
		Options All
		AllowOverride All
		Require all granted
	</Directory>
	SSLEngine on
	SSLCipherSuite ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS:!RC4
	SSLCertificateKeyFile /etc/pki/apache/wildcard.mwclearning.com.au-201710.key.pem
	SSLCertificateFile /etc/pki/apache/wildcard.mwclearning.com.au-201710.cert.pem
	SSLCACertificateFile /etc/pki/apache/wildcard.mwclearning.com.au-201710.ca.pem
</VirtualHost>

With a systemctl restart httpd this was working….

Logging into Horizon and checking the endpoints under Project -> Compute -> API Access I can see some more public HTTP endpoints:

Identity	http://api.cbr1.os.mwclearning.com:5000/v3/
Object Store	http://swift.cbr1.os.mwclearning.com:8080/v1/AUTH_---

These endpoints are defined in Keystone, to see them and edit them there I can ssh to the keystone server and run some mysql queries. Before I do this I need to make sure that the swift and keystone endpoints are configure to use TLS/SSL.

Keystone endpoint

Again the TLS/SSL termination point is apache… so some modification to /etc/httpd/conf.d/wsgi-keystone.conf is all that is required:

Listen 5000
Listen 35357

<VirtualHost *:5000>
    WSGIDaemonProcess keystone-public processes=5 threads=1 user=keystone group=keystone display-name=%{GROUP}
    WSGIProcessGroup keystone-public
    WSGIScriptAlias / /usr/bin/keystone-wsgi-public
    WSGIApplicationGroup %{GLOBAL}
    WSGIPassAuthorization On
    LimitRequestBody 114688
    <IfVersion >= 2.4>
      ErrorLogFormat "%{cu}t %M"
    </IfVersion>
    ErrorLog /var/log/httpd/keystone.log
    CustomLog /var/log/httpd/keystone_access.log combined

    <Directory /usr/bin>
        <IfVersion >= 2.4>
            Require all granted
        </IfVersion>
        <IfVersion < 2.4>
            Order allow,deny
            Allow from all
        </IfVersion>
    </Directory>
Alias /identity /usr/bin/keystone-wsgi-public
<Location /identity>
    SetHandler wsgi-script
    Options +ExecCGI

    WSGIProcessGroup keystone-public
    WSGIApplicationGroup %{GLOBAL}
    WSGIPassAuthorization On
</Location>
	SSLEngine on
	SSLCipherSuite ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS:!RC4
	SSLCertificateKeyFile /etc/pki/apache/wildcard.mwclearning.com.au-201710.key.pem
	SSLCertificateFile /etc/pki/apache/wildcard.mwclearning.com.au-201710.cert.pem
	SSLCACertificateFile /etc/pki/apache/wildcard.mwclearning.com.au-201710.ca.pem
</VirtualHost>

<VirtualHost *:35357>
    WSGIDaemonProcess keystone-admin processes=5 threads=1 user=keystone group=keystone display-name=%{GROUP}
    WSGIProcessGroup keystone-admin
    WSGIScriptAlias / /usr/bin/keystone-wsgi-admin
    WSGIApplicationGroup %{GLOBAL}
    WSGIPassAuthorization On
    LimitRequestBody 114688
    <IfVersion >= 2.4>
      ErrorLogFormat "%{cu}t %M"
    </IfVersion>
    ErrorLog /var/log/httpd/keystone.log
    CustomLog /var/log/httpd/keystone_access.log combined

    <Directory /usr/bin>
        <IfVersion >= 2.4>
            Require all granted
        </IfVersion>
        <IfVersion < 2.4>
            Order allow,deny
            Allow from all
        </IfVersion>
    </Directory>
Alias /identity_admin /usr/bin/keystone-wsgi-admin
<Location /identity_admin>
    SetHandler wsgi-script
    Options +ExecCGI

    WSGIProcessGroup keystone-admin
    WSGIApplicationGroup %{GLOBAL}
    WSGIPassAuthorization On
</Location>
</VirtualHost>

I left the internal interface as HTTP for now…

Swift endpoint

OK so swift one is a bit different… its actually recommended to have an SSL termination service in front of the swift proxy see: https://docs.openstack.org/security-guide/secure-communication/tls-proxies-and-http-services.html

With that recommendation from OpenStack and ease of creating an apache reverse proxy – I will do that.

# install packages
sudo yum install httpd mod_ssl

After install create a vhost  /etc/httpd/conf.d/swift-endpoint.conf contents:

<VirtualHost *:443>
  ProxyPreserveHost On
  ProxyPass / http://127.0.0.1:8080/
  ProxyPassReverse / http://127.0.0.1:8080/

  ErrorLog /var/log/httpd/swift-endpoint_ssl_error.log
  LogLevel warn
  CustomLog /var/log/httpd/swift-endpoint_ssl_access.log combined

  SSLEngine on
  SSLCipherSuite ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS:!RC4
  SSLCertificateKeyFile /etc/httpd/tls/wildcard.mwclearning.com.201709.key.pem
  SSLCertificateFile /etc/httpd/tls/wildcard.mwclearning.com.201709.cert.pem
  SSLCertificateChainFile /etc/httpd/tls/wildcard.mwclearning.com.201709.ca.pem
</VirtualHost>

#resart apache
systemctl restart httpd

So now we should have an endpoint that will decrypt and forward https request from port 443 to the swift listener on port 8080.

Updating internal auth

As keystones auth listener is the same for internal and external (vhost) I also updated the internal address to match the FQDN allowing for valid TLS.

Keystone service definitions

mysql -u keystone -h services01 -p
use keystone;
select * from endpoint;
# Updating these endpoints with 
update endpoint set url='https://swift-os.mwclearning.com:8080/v1/AUTH_%(tenant_id)s' where id='579569...';
update endpoint set url='https://api-cbr1-os.mwclearning,com:5000/v3/' where id='637e843b...';
update endpoint set url='http://controller01-int.mwclearning.com:5000/v3/' where id='ec1ad2e...';

Now after restarting the services all is well with TLS!