Sabharwal - Docker Hands on Deploy - Administer Docker Platform Docker server access over https https://shipyard-project.com/docs/quickstart/ SSL Certificate cert.pem SSL Key key.pem CA Certificate ca.pem http://middlewaresnippets.blogspot.com.by/2015/08/spring-boot-and-docker.html /opt /docker /certs (contains the certificates for client-server communication) /ca ca-key.pem # CA pvt signing key ca.pem ca.srl /client cert.pem key.pem /server server-cert.pem # server server-key.pem # server pvt rsa key If we need Docker to be reachable via the network in a safe manner, we can enable TLS by specifying the tlsverify flag and pointing Docker's tlscacert flag to a trusted CA certificate. In the daemon mode, it will only allow connections from clients authenticated by a certificate signed by that CA. In the client mode, it will only connect to servers with a certificate signed by that CA. By using OpenSSL we can generate the necessary keys and certificates. To generate the private and public signing keys, we can use # create the necessary directories [root@docker ~]# mkdir -p /opt/docker/certs/{ca,client,server} # generate private and public signing keys # genrsa - generate an RSA private key # -aes256: encrypt the private key with the defined cipher # -out: the output filename # numbits: the size of the private key to generate in bits. This must be the last option specified. [root@docker ~]# openssl genrsa -aes256 -out /opt/docker/certs/ca/ca-key.pem 2048 Generating RSA private key, 2048 bit long modulus ...........................+++ ...................+++ e is 65537 (0x10001) Enter pass phrase for /opt/docker/certs/ca/ca-key.pem: Verifying - Enter pass phrase for /opt/docker/certs/ca/ca-key.pem: # req - PKCS#10 certificate request and certificate generating utility. # -new: this option generates a new certificate request. It will prompt the user for the relevant field values. # -x509: this option outputs a self signed certificate instead of a certificate request. # -days n: when the -x509 option is being used this specifies the number of days to certify the certificate for. # -key: specifies the file to read the private key from. # -[digest]: this specifies the message digest to sign the request with (run 'openssl dgst -h output' to see a list of possible values). # -out filename: specifies the output filename to write to. # -subj arg: sets subject name for new request or supersedes the subject name when processing a request. The arg must be formatted as /type0=value0/type1=value1/type2=... [root@docker ~]# openssl req -new -x509 -days 365 -key /opt/docker/certs/ca/ca-key.pem -sha256 -out /opt/docker/certs/ca/ca.pem -subj '/C=NL/ST=Middleware/L=Snippets/O=Middleware Snippets/OU=Blogging/CN=*.machine.com' Enter pass phrase for /opt/docker/certs/ca/ca-key.pem: Note that we have used a wild-card in the common-name (CN=*.machine.com) such that we can use the certificates on all hosts. To generate the server key and a certificate signing request, we can use # generate the server key and certificate signing request # genrsa - generate an RSA private key # -out: the output filename # numbits: the size of the private key to generate in bits. This must be the last option specified. [root@docker ~]# openssl genrsa -out /opt/docker/certs/server/server-key.pem 2048 Generating RSA private key, 2048 bit long modulus ...........................................................................................................................................+++ ......................................................................................................................+++ e is 65537 (0x10001) # req - PKCS#10 certificate request and certificate generating utility. # -new: this option generates a new certificate request. It will prompt the user for the relevant field values. # -key: specifies the file to read the private key from. # -[digest]: this specifies the message digest to sign the request with (run 'openssl dgst -h output' to see a list of possible values). # -out filename: specifies the output filename to write to. # -subj arg: sets subject name for new request or supersedes the subject name when processing a request. The arg must be formatted as /type0=value0/type1=value1/type2=... [root@docker ~]# openssl req -new -key /opt/docker/certs/server/server-key.pem -sha256 -out /opt/docker/certs/server/server.csr -subj '/C=NL/ST=Middleware/L=Snippets/O=Middleware Snippets/OU=Blogging/CN=*.machine.com' Next, we need to sign the certificate signing request by our CA # we can also add additional subject identities if needed # x509 - Certificate display and signing utility # -req: by default a certificate is expected on input. With this option a certificate request is expected instead # -days arg: specifies the number of days to make a certificate valid for. # -[digest]: this specifies the message digest to sign the request with (run 'openssl dgst -h output' to see a list of possible values). # -in filename: specifies the input filename to read a certificate from . # -CA filename: specifies the CA certificate to be used for signing. When this option is present x509 behaves like a "mini CA". The input file is signed by this CA using this option: that is its issuer name is set to the subject name of the CA and it is digitally signed using the CAs private key. # -CAkey filename: sets the CA private key to sign a certificate with. If this option is not specified then it is assumed that the CA private key is present in the CA certificate file. # -CAcreateserial: with this option the CA serial number file is created if it does not exist: it will contain the serial number "02" and the certificate being signed will have the 1 as its serial number. Normally if the -CA option is specified and the serial number file does not exist it is an error. # -out filename: specifies the output filename to write to. # -extfile filename: file containing certificate extensions to use. If not specified then no extensions are added to the certificate. [root@docker ~]# echo subjectAltName = IP:192.168.101.210,IP:192.168.101.220,IP:192.168.101.230,IP:127.0.0.1 > /opt/docker/certs/server/extfile.cnf [root@docker ~]# openssl x509 -req -days 365 -sha256 -in /opt/docker/certs/server/server.csr -CA /opt/docker/certs/ca/ca.pem -CAkey /opt/docker/certs/ca/ca-key.pem -CAcreateserial -out /opt/docker/certs/server/server-cert.pem -extfile /opt/docker/certs/server/extfile.cnf Signature ok subject=/C=NL/ST=Middleware/L=Snippets/O=Middleware Snippets/OU=Blogging/CN=*.machine.com Getting CA Private Key Enter pass phrase for /opt/docker/certs/ca/ca-key.pem: # sign the public key without additional subject identities [root@docker ~]# openssl x509 -req -days 365 -sha256 -in /opt/docker/certs/server/server.csr -CA /opt/docker/certs/ca/ca.pem -CAkey /opt/docker/certs/ca/ca-key.pem -CAcreateserial -out /opt/docker/certs/server/server-cert.pem Signature ok subject=/C=NL/ST=Middleware/L=Snippets/O=Middleware Snippets/OU=Blogging/CN=*.machine.com Getting CA Private Key Enter pass phrase for /opt/docker/certs/ca/ca-key.pem: Finally, we generate the client key and certificate # generate client key and certificate signing request # genrsa - generate an RSA private key # -out: the output filename # numbits: the size of the private key to generate in bits. This must be the last option specified. [root@docker ~]# openssl genrsa -out /opt/docker/certs/client/key.pem 2048 Generating RSA private key, 2048 bit long modulus ..+++ .................+++ e is 65537 (0x10001) # req - PKCS#10 certificate request and certificate generating utility. # -new: this option generates a new certificate request. It will prompt the user for the relevant field values. # -key: specifies the file to read the private key from. # -out filename: specifies the output filename to write to. # -subj arg: sets subject name for new request or supersedes the subject name when processing a request. The arg must be formatted as /type0=value0/type1=value1/type2=.. [root@docker ~]# openssl req -new -key /opt/docker/certs/client/key.pem -out /opt/docker/certs/client/client.csr -subj '/CN=client' # sign the client key # x509 - Certificate display and signing utility # -req: by default a certificate is expected on input. With this option a certificate request is expected instead # -days arg: specifies the number of days to make a certificate valid for. # -[digest]: this specifies the message digest to sign the request with (run 'openssl dgst -h output' to see a list of possible values). # -in filename: specifies the input filename to read a certificate from . # -CA filename: specifies the CA certificate to be used for signing. When this option is present x509 behaves like a "mini CA". The input file is signed by this CA using this option: that is its issuer name is set to the subject name of the CA and it is digitally signed using the CAs private key. # -CAkey filename: sets the CA private key to sign a certificate with. If this option is not specified then it is assumed that the CA private key is present in the CA certificate file. # -CAcreateserial: with this option the CA serial number file is created if it does not exist: it will contain the serial number "02" and the certificate being signed will have the 1 as its serial number. Normally if the -CA option is specified and the serial number file does not exist it is an error. # -out filename: specifies the output filename to write to. # -extfile filename: file containing certificate extensions to use. If not specified then no extensions are added to the certificate. [root@docker ~]# echo extendedKeyUsage = clientAuth > /opt/docker/certs/client/extfile.cnf [root@docker ~]# openssl x509 -req -days 365 -sha256 -in /opt/docker/certs/client/client.csr -CA /opt/docker/certs/ca/ca.pem -CAkey /opt/docker/certs/ca/ca-key.pem -CAcreateserial -out /opt/docker/certs/client/cert.pem -extfile /opt/docker/certs/client/extfile.cnf Signature ok subject=/CN=client Getting CA Private Key Enter pass phrase for /opt/docker/certs/ca/ca-key.pem: # change the permissions of the private keys [root@docker ~]# chmod 0400 /opt/docker/certs/ca/ca-key.pem /opt/docker/certs/client/key.pem /opt/docker/certs/server/server-key.pem # change the permissions of the public keys [root@docker ~]# chmod 0444 /opt/docker/certs/ca/ca.pem /opt/docker/certs/client/cert.pem /opt/docker/certs/server/server-cert.pem # optionally the certificate signing requests and extention files can be removed [root@docker ~]# rm -f /opt/docker/certs/client/client.csr /opt/docker/certs/client/extfile.cnf /opt/docker/certs/server/server.csr /opt/docker/certs/server/extfile.cnf # copy the certificates to the other hosts [root@docker certs]# scp -rp * root@springboot1.machine.com:/opt/docker/certs/ root@springboot1.machine.com's password: ca-key.pem 100% 1766 1.7KB/s 00:00 ca.pem 100% 1383 1.4KB/s 00:00 ca.srl 100% 17 0.0KB/s 00:00 key.pem 100% 1679 1.6KB/s 00:00 cert.pem 100% 1155 1.1KB/s 00:00 server-key.pem 100% 1675 1.6KB/s 00:00 server-cert.pem 100% 1265 1.2KB/s 00:00 [root@docker certs]# scp -rp * root@springboot2.machine.com:/opt/docker/certs/ root@springboot2.machine.com's password: ca-key.pem 100% 1766 1.7KB/s 00:00 ca.pem 100% 1383 1.4KB/s 00:00 ca.srl 100% 17 0.0KB/s 00:00 key.pem 100% 1679 1.6KB/s 00:00 cert.pem 100% 1155 1.1KB/s 00:00 server-key.pem 100% 1675 1.6KB/s 00:00 server-cert.pem 100% 1265 1.2KB/s 00:00 To make the Docker daemon only accept connections from clients providing a certificate trusted by our CA, we edit the /usr/lib/systemd/system/docker.service file [root@springboot2 system]# cat docker.service [Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com After=network.target docker.socket Requires=docker.socket [Service] Type=notify #ExecStart=/usr/bin/dockerd -H fd:// ExecStart=/usr/bin/docker -d --tlsverify --tlscacert=/opt/docker/certs/ca/ca.pem --tlscert=/opt/docker/certs/server/server-cert.pem --tlskey=/opt/docker/certs/server/server-key.pem -H=0.0.0.0:2376 MountFlags=slave LimitNOFILE=1048576 LimitNPROC=1048576 LimitCORE=infinity [Install] WantedBy=multi-user.target # Restart docker [root@springboot2 system]# systemctl daemon-reload [root@springboot2 system]# systemctl start docker.service [root@springboot2 system]# systemctl status docker.service docker.service - Docker Application Container Engine Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled) Active: active (running) since Wed 2015-08-19 14:36:56 CEST; 7s ago Docs: https://docs.docker.com Main PID: 11021 (docker) CGroup: /system.slice/docker.service -- 11021 /usr/bin/docker -d --tlsverify --tlscacert=/opt/docker/certs/ca/ca.pem --tlscert=/opt/docker/certs/server/server-cert.pem --tlskey=/opt/docker/certs/server/server-key.pem ... To communicate with the Docker daemon we now have to use [root@docker ~]# docker --tlsverify --tlscacert=/opt/docker/certs/ca/ca.pem --tlscert=/opt/docker/certs/client/cert.pem --tlskey=/opt/docker/certs/client/key.pem -H=springboot2.machine.com:2376 pull docker.machine.com:5000/springbootexample:latest latest: Pulling from springbootexample f1b10cd84249: Pull complete c852f6d61e65: Pull complete 7322fbe74aa5: Pull complete 4cfba0adf483: Pull complete 6a5b64d0925e: Pull complete e136fb7fd983: Pull complete f027b1b29898: Pull complete e8261eb40ec4: Pull complete 20ee6480f62e: Pull complete 369ff136c2b5: Pull complete c200531e0027: Pull complete 012ae7d5f8b5: Pull complete c8a563965cf0: Pull complete Digest: sha256:0b50559aa6d94d3a6919c5efd77b2ce262760c1bc4a9db7cb0b4f539ee854e0e Status: Downloaded newer image for docker.machine.com:5000/springbootexample:latest [root@docker ~]# docker --tlsverify --tlscacert=/opt/docker/certs/ca/ca.pem --tlscert=/opt/docker/certs/client/cert.pem --tlskey=/opt/docker/certs/client/key.pem -H=springboot2.machine.com:2376 images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE docker.machine.com:5000/springbootexample latest c8a563965cf0 3 hours ago 528.4 MB [root@docker ~]# docker --tlsverify --tlscacert=/opt/docker/certs/ca/ca.pem --tlscert=/opt/docker/certs/client/cert.pem --tlskey=/opt/docker/certs/client/key.pem -H=springboot2.machine.com:2376 run -d --name="springbootexample" --net="host" -v /home/temp:/home/temp -p 8080:8080 -p 9090:9090 docker.machine.com:5000/springbootexample 38a2ff01166a7c5c540c4fc6117946a535af9cf915a0c5bb7ae29b573429392e [root@docker ~]# docker --tlsverify --tlscacert=/opt/docker/certs/ca/ca.pem --tlscert=/opt/docker/certs/client/cert.pem --tlskey=/opt/docker/certs/client/key.pem -H=springboot2.machine.com:2376 ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 38a2ff01166a docker.machine.com:5000/springbootexample "java -XX:+UnlockComm" 41 seconds ago Up 39 seconds springbootexample [root@docker ~]# docker --tlsverify --tlscacert=/opt/docker/certs/ca/ca.pem --tlscert=/opt/docker/certs/client/cert.pem --tlskey=/opt/docker/certs/client/key.pem -H=springboot1.machine.com:2376 start springbootexample springbootexample [root@docker ~]# docker --tlsverify --tlscacert=/opt/docker/certs/ca/ca.pem --tlscert=/opt/docker/certs/client/cert.pem --tlskey=/opt/docker/certs/client/key.pem -H=springboot1.machine.com:2376 ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 7193fa0ef4c8 docker.machine.com:5000/springbootexample "java -XX:+UnlockComm" 3 hours ago Up 19 seconds springbootexample To secure Docker client connections by default, we can move the client certificate files to the .docker directory in the home directory, and set the DOCKER_HOST and DOCKER_TLS_VERIFY environment variables. # create the .docker directory [root@docker ~]# mkdir -p ~/.docker # copy the certificates [root@docker ~]# cp /opt/docker/certs/ca/ca.pem ~/.docker [root@docker ~]# cp /opt/docker/certs/client/cert.pem ~/.docker [root@docker ~]# cp /opt/docker/certs/client/key.pem ~/.docker # check [root@docker .docker]# ll -r--r--r-- 1 root root 1383 Aug 19 15:02 ca.pem -r--r--r-- 1 root root 1155 Aug 19 15:02 cert.pem -r-------- 1 root root 1679 Aug 19 15:07 key.pem # edit .bashrc [root@docker ~]# cat .bashrc # .bashrc # User specific aliases and functions alias rm='rm -i' alias cp='cp -i' alias mv='mv -i' # Source global definitions if [ -f /etc/bashrc ]; then . /etc/bashrc fi export DOCKER_HOST=tcp://springboot1.machine.com:2376 export DOCKER_TLS_VERIFY=1 # check [root@docker ~]# echo $DOCKER_HOST tcp://springboot1.machine.com:2376 [root@docker ~]# echo $DOCKER_TLS_VERIFY 1 # this communicates only with springboot1.machine.com [root@docker ~]# docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE docker.machine.com:5000/springbootexample latest c8a563965cf0 4 hours ago 528.4 MB # note that it is better to leave out the $DOCKER_HOST and $DOCKER_TLS_VERIFY environment variables and use [root@docker ~]# docker --tlsverify -H=springboot1.machine.com:2376 ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 7193fa0ef4c8 docker.machine.com:5000/springbootexample "java -XX:+UnlockComm" 3 hours ago Up 15 minutes springbootexample [root@docker ~]# docker --tlsverify -H=springboot2.machine.com:2376 ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 38a2ff01166a docker.machine.com:5000/springbootexample "java -XX:+UnlockComm" 23 minutes ago Up 23 minutes springbootexample