Monthly Archives: December 2016

Trusted Python client

With a trusted root CA and signed server certificate we can now look at developing trusted client-server communication using the CA to sign client certificates and using them in server requests.

I have prepared some Python code that creates a client CSR and sends it to the server for signing before picking up a copy of the certificate and using it to upload form data. Because we’re using Nginx as a reverse proxy for the Python Flask server (which doesn’t seem to work too well over https) we need to be able to connect the components together.

Python Flask, Requests and PyOpenSSL

The general process for using PyOpenSSL to create a CSR boils down to the following

key = OpenSSL.crypto.PKey()
 key.generate_key(OpenSSL.crypto.TYPE_RSA, 2048)
req = OpenSSL.crypto.X509Req()
... set req properties: name, country, location, etc.
san_list = ["DNS:*." + common_name, "DNS:" + common_name]
     req.add_extensions([
     OpenSSL.crypto.X509Extension( 'subjectAltName'.encode(), False, ", ".join(san_list).encode() ) ] )
req.set_pubkey(key)
 req.sign(key, 'sha256')
private_key = OpenSSL.crypto.dump_privatekey(OpenSSL.crypto.FILETYPE_PEM, key)
csr = OpenSSL.crypto.dump_certificate_request(OpenSSL.crypto.FILETYPE_PEM, req)

See [2] and [3] for more detail on how to create a CSR with PyOpenSSL.

The CSR is sent to the server with,

resp = requests.post( "https://inventory-master.localdomain/package-inventory/client-cert/new",
 data=csr,
 headers={'Content-Type': 'application/pkcs10'},
 verify=cacert,
 allow_redirects=True)

The cacert refers to the CA chain certificate, so, when testing this with a remote client, the chain cert needs to be downloaded from the server. allow_redirects is needed because the server will redirect to another resource to retrieve the signed cert.

When it comes to Python code for signing the request on the server side, I took the easy way out; using PyOpenSSL seems like hard work when the openssl command is all you need.

I added a Flask route like,

@app.route('/package-inventory/client-cert/new', methods=["POST"])
def post_client_csr():

Which saves the POST data t a file and uses it (with the obvious security risks; it’s hard enough getting it to work in the first place, never mind making it secure) as the CSR input to a statement like,

sign_cert = ("openssl ca -config %s -in %s -passin file:%s -batch -out %s -extensions v3_req -extfile %s -days 375 -notext -md sha256 " % (csrconf, csrfile, passfile, outcert, csrconf) )
output = subprocess.getoutput(sign_cert)

The client is sent a 301 redirect to another (GET) route that provides the signed certificate for download, after which it is deleted by the server.

Create a wsgi wrapper script

from PackageInventoryServer import app as application

if __name__ == "__main__":
 application.run()

We also need an initialization file, say, wsgi.ini, to describe the operating parameters for wsgi,

[uwsgi]
module = wsgi

master = true
processes = 5

socket = PackageInventoryServer.sock
chmod-socket = 666
vacuum = true

die-on-term = true

Then we need a systemd (cough, spit – [1]) to start the wsgi application at boot if the server is to start automatically. Otherwise, it is started with a command like,

uwsgi --ini wsgi.ini

And while the documentation indicates it might be possible to support https at this level I couldn’t get it working and Nginx likely does it better in any case.

And finally, we connect the application to Nginx with the following snippet in the ssl server block in /etc/nginx/nginx.conf,

server {
 listen 443 ssl;
...
  location / {
    root html;
    index index.html index.htm;
    include uwsgi_params;
    uwsgi_pass unix:/var/www/PackageInventoryServer/PackageInventoryServ
er.sock;
  }
...}

where uwsgi_params are loaded from /etc/nginx/uwsgi_params.

This is not quite as elegant a method as using PhusionPssenger with Ruby on Rails (and, yes, it does support Python, but I haven’t tried it in that way; wsgi seems to be the de-facto method)

Client requests

The Python Requests library can be used to send a server request with something like,

resp = requests.post( “https://inventory-master.localdomain/package-inventory/packages/new”, data=json.JSONEncoder().encode( jdata ), headers={‘Content-Type’: ‘application/json’}, cert=(cert,key), verify=chain)

where,

  • jdata is a data structure (array, dict, whatever) containing the data to be passed to the server,
  • cert is the contents of the CA-signed client certificate,
  • key is the private key for the certificate; passphrse protected keys don’t work,
  • chain is the CA keychain (which must be downloaded by the client and stored locally)

Testing and troubleshooting

 

References

The following resources were of assistance while working through the development of the client and server code.

[1] – I still have no idea what problem systemd is trying to solve other than a lack of complexity and obfuscation in init.d scripts. The init.d approach is beautifully simple to implement and troubleshoot and nothing is ever likely to come close; systemd doesn’t even try.

[2] http://stackoverflow.com/questions/3215780/generating-a-csr-in-python – Using PyOpenSSL to create a CSR

[3] http://stackoverflow.com/questions/24475768/is-it-possible-to-set-subjectaltname-using-pyopenssl – creating a CSR with Subject Alternate Names

[4] http://stackoverflow.com/questions/9496698/how-to-revoke-an-openssl-certificate-when-you-dont-have-the-certificate – how to get past certificate database errors when re-requesting a replacement

[5] https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-uwsgi-and-nginx-on-ubuntu-14-04 – A good guide to fronting Flask applications with uwsgi and Nginx.

[6] http://stackoverflow.com/questions/30405867/how-to-get-python-requests-to-trust-a-self-signed-ssl-certificate – How to get Python Requests calls to trust self-signed certificates.

[7] http://serverfault.com/questions/721572/nginx-verifying-client-certs-only-on-a-particular-location- Location-based authorisation so that CSRs and heartbeats can be processed without requiring a cert; everything else does; we’re not bothered if it
doesn’t work for one browser or another.

[8] http://stackoverflow.com/questions/27611193/curl-ssl-with-self-signed-certificate – simple answer that can help check that the web server is using the expected
certs. Getting this right makes life much easier.

[] – http://kracekumar.com/post/54437887454/ssl-for-flask-local-development – It might be possible to start flask with SSL options, but not if the private key requires a passphrase.

 

 

I really wanted OSMC to work

With  a couple of new 16GB SD cards for use with my Raspberry Pi collection, I thought I’d give Xbmc another try for a friendly media center over the festive break with some old vinyl I had ripped to disk.

But then I came cross OSMC as a distribution available on the Pi that fixed many of the xbmc shortcomings so I thought I’d give it a go.

One of the most frustrating things regarding Xbmc was the download of a 16Gb image only for there to be most of the space unallocated on the filesystem and no space to install any media files. OSMC certainly is a major improvement: download a small image and flash to the SD card, boot the Pi with and from there complete the installation of the rest of the system over the network.

Alas, on the Pi, it’s downhill from this point.

  • the default confluence skin is too big for the HDMI screen, left and right, top and bottom with text in the tool tips hidden,
  • the default interface is very clunky and hard to navigate with a mouse, frequently losing focus,
  • incoherent menus: trying to remember the difference between system and settings is no end of pain,
  • the OSMC skin is all but unusable: the scrolling text does all it can to avoid being selected under the pointer; the pointer itself at times getting replaced by a vertical bar of mis-render on the screen;
  • I had problems with no audio until a switch to the OSMC skin showed a ‘audio muted’ message onscreen; there was no equivalent on the confluence skin and I have no idea how the audio became muted,
  • the audio was often paused during playback even when nothing else was going on.
  • No obvious way to get music files on to the device: I could define an NFS mount for the library but no means of copying the files down: it’s a Pi so there’s no CD or SD card available.

While, as always, I appreciate the hard work that goes into putting these things together, I’m afraid that this is no way to showcase a Linux media center and I decided to bin it before letting any of my family see it.

Flask over https and local root CA

I recently had a week’s PTO and decided to spend the time getting to grips with setting up a private root CA so that I could try getting a python flask application to use client certificates (signed by the server) for authorization to request upload resources.

It was quite a struggle, more because of the root CA than anything else but since the aim was to gain an understanding of how this kind of set up worked, how to structure the requests and do the work in python (with a ruby version to follow no doubt) I am happy with the progress so far.

I will post a couple of pages describing the work I did with references and how I got past some of the major sticking points.