server: SSL Support

[gabe-l-hart]

Description

This PR adds SSL support to the examples/server using the built-in SSL support in the vendored httlib.h header library.

Changes

• Add the CMake build option LLAMA_SERVER_SSL (default OFF) to toggle building with/without SSL support
  • NOTE: Building with SSL requires linkage against libssl/libcrypto, so this adds a dependency on openssl and requires version >= 3 (see note here on the cpp-httplib repo)
• Hide all SSL support behind #define CPPHTTPLIB_OPENSSL_SUPPORT following the pattern in httplib
• When enabled, add ssl_key_file and ssl_cert_file to the server_params struct
• When enabled, add parsing logic to the command line args for --ssl-key-file and --ssl-cert-file
  • NOTE: httplib does not support inline PEM strings, so these must point to files
• Move the svr instance variable to be a unique_ptr<httplib::Server> and instantiate it as either a httplib::Server or httplib::SslServer depending on the values of server_params.ssl_key_file and server_params.ssl_cert_file.
  • NOTE: This caused all uses of svr to move from . to ->, so touched a lot of lines!

Testing

I noticed the tests in examples/server/tests, but these all appear to be testing against the python bindings. Since building with SSL support requires additional dependencies, I did not want to add these as dependencies in the python bindings and did not attempt to make the default build for python include SSL support, therefore I did not update these tests.

I performed manual testing using locally-generated SSL files (see my utility script for generating them below). With the generated server.key.pem and server.cert.pem, I ran the following:

cd build

# Build with SSL disabled
cmake .. -DLLAMA_SERVER_SSL=OFF
make server

# Boot with SSL: verify invalid arguments
server -m my_model --ssl-key-file server.key.pem --ssl-cert-file serever.cert.pem
# -> Display help, exit code 1

# Build with SSL enabled
cmake .. -DLLAMA_SERVER_SSL=ON
make server

# Boot with SSL: verify log message
server -m my_model --ssl-key-file server.key.pem --ssl-cert-file serever.cert.pem
# {"cert":"server.cert.pem","function":"main","key":"server.key.pem","level":"INFO","line":2644,"msg":"Running with SSL","tid":"0x1d8b71000","timestamp":1709837564}

# Run http call: verify no valid response
curl http://localhost:8080

# Run insecure https call: verify valid response
curl -k https://localhost:8080

# Run https call with signing CA: verify valid response
curl --cacert ca.cert.pem https://localhost:8080

# Boot without SSL: verify log message
server -m my_model
# {"function":"main","level":"INFO","line":2649,"msg":"Running without SSL","tid":"0x1d8b71000","timestamp":1709837582}

# Run http call: verify valid response
curl http://localhost:8080

# Run insecure https call: verify invalid response
curl -k https://localhost:8080

gen_mtls_test_files.sh

#!/usr/bin/env bash

## Config ######################################################################

# Optional additional SANs can be set with SANS
SANS=${SANS:-""}

# CN can be overloaded
CN=${CN:-"foo.bar.com"}

set -euo pipefail

# Set up additional SANs block
IFS=' ' read -r -a sans_arr <<< "$SANS"
extra_sans=""
counter="1"
for san in "${sans_arr[@]}"
do
    counter=$(expr "$counter" "+" "1")
    echo "Adding SAN DNS.$counter [$san]"
    extra_sans="$extra_sans\nDNS.$counter = $san"
done

root_name="ca"
root_key="$root_name.key.pem"
root_crt="$root_name.cert.pem"

server_name="server"
server_key="$server_name.key.pem"
server_crt="$server_name.cert.pem"

client_name="client"
client_key="$client_name.key.pem"
client_crt="$client_name.cert.pem"

common_config='
[req]
default_bits       = 4096
default_keyfile    = server.key.pem
distinguished_name = req_distinguished_name
x509_extensions    = x509_ext
string_mask        = utf8only

[req_distinguished_name]
countryName                 = Country Name (2 letter code)
countryName_default         = US
stateOrProvinceName         = State or Province Name (full name)
stateOrProvinceName_default = Denver
localityName                = Locality Name (eg, city)
localityName_default        = Denver
organizationName            = Organization Name (eg, company)
organizationName_default    = IBM Watson
commonName                  = Common Name (eg, YOUR name)
commonName_max              = 64
'

ca_config="
$common_config

[x509_ext]
subjectKeyIdentifier   = hash
authorityKeyIdentifier = keyid:always, issuer
basicConstraints       = critical, CA:TRUE, pathlen:1
keyUsage               = keyCertSign, cRLSign
"

derived_config="
$common_config

[x509_ext]
subjectKeyIdentifier   = hash
authorityKeyIdentifier = keyid,issuer
basicConstraints       = CA:FALSE
keyUsage               = digitalSignature, keyEncipherment
subjectAltName         = @alt_names

[alt_names]
DNS.1 = localhost
$extra_sans
IP.1  = '127.0.0.1'
"

# use wild card in subject, not all clients accept that, but Java grpc client does
SUBJ="/C=US/ST=Denver/L=Denver/O=ME/OU=YOU/CN=$CN"

# Set the expiration for 10 years
expiration_days=3650

## Root ########################################################################


# Create the root key
echo "[Creating root key]"
openssl genrsa -out $root_key 2048

# create root key and cert
echo "[Creating root cert]"
openssl req \
    -config <(echo -e "$ca_config") \
    -x509 \
    -new \
    -nodes \
    -key $root_key \
    -sha256 \
    -subj "$SUBJ" \
    -out $root_crt

## Server ######################################################################

# create a new server key and certificate signing request
echo "[Creating server key and signing request]"
openssl req -config <(echo -e "$derived_config") -new -nodes -sha256 -keyout $server_key -out $server_name.csr -newkey rsa:2048 -subj "$SUBJ"

# sign the request with our root cert key
echo "[Sign server cert]"
openssl x509 -req -sha256 -extfile <(echo -e "$derived_config") -extensions x509_ext -in $server_name.csr -CA $root_crt -CAkey $root_key -CAcreateserial -out $server_crt -days $expiration_days

# write out server key in pkcs8 format, required by grpc
echo "[Convert server key to pkcs8]"
cp $server_key $server_key.tmp
openssl pkcs8 -topk8 -nocrypt -in $server_key.tmp -out $server_key

## Client ######################################################################

# create a new server key and certificate signing request
echo "[Creating client key and signing request]"
openssl req -config <(echo -e "$derived_config") -new -nodes -sha256 -keyout $client_key -out $client_name.csr -newkey rsa:2048 -subj "$SUBJ"

# sign the request with our root cert key
echo "[Sign client cert]"
openssl x509 -req -sha256 -extfile <(echo -e "$derived_config") -extensions x509_ext -in $client_name.csr -CA $root_crt -CAkey $root_key -CAcreateserial -out $client_crt -days $expiration_days

# write out client key in pkcs8 format, required by grpc
echo "[Convert client key to pkcs8]"
cp $client_key $client_key.tmp
openssl pkcs8 -topk8 -nocrypt -in $client_key.tmp -out $client_key

# Clean up
rm *.tmp
rm *.csr
rm *.srl

Open Questions

• As discussed in this issue, for full security support, an nginx proxy is a more robust way to go. That said, having basic SSL support in the native server is a much simpler deployment topology, so may be worth the maintenance effort.
• The --ssl-key-file and --ssl-cert-file arguments only work if both are specified. Currently, if either is not specified, they are both ignored. Should they instead cause an error if one is specified without the other?
• Since cpp-httlib does not support mutual TLS, this change also does not support it. Is it worth having non-mutual support without going all the way to mTLS?

[gabe-l-hart]

I kind of hate that this breaks the brace formatting of the rest of the if/else if chain, but I figured it was better than duplicating the else if (arg == "--timeout"....

[phymbert]

Note that the tests are written in Python, but there is no binding, just plain old http calls to the server. Anyway, these are mainly functional tests, and I don't expect to add an ssl handcheck against the server as it will require a trusted root certificate. If it compiles with the good libopenssl version, it will work.

[phymbert]

Please document the README.md with the proposed changes

[gabe-l-hart]

Thanks for the review! The README changes are pushed as well as a first pass at top-level Makefile support. I'm honestly not that familiar with Makefile best-practices, so feel free to suggest a better way to improve there. In particular, I'm not familiar with the make equivalent of CMake's find_package to detect the available OpenSSL installation.

[cebtenzzre]

In particular, I'm not familiar with the make equivalent of CMake's find_package to detect the available OpenSSL installation.

For a few libraries we are using pkg-config. It's probably not necessary here though.

[phymbert]

Really useful, thanks

[volfyd]

If anyone is struggling to get this to work, please note that there does not appear to be error handliing for --ssl-key-file and --ssl-cert-file. So if you fat-finger the filename, you won't get an error that the filename doesn't exist. You'll get

main: couldn't bind HTTP server socket, hostname: ..., port: ...