Author Archives: julianrawcliffe

Git: stop tracking a tracked file

There are times when a project needs to include the default version of a file in the git repository that will subsequently change to support development (e.g., secret application key or sqlite3 database).

After committing the safe  default copy, any subsequent changes to the files will appear as a modified file in ‘git status’ reports and will also prevent any git flow feature finish operations if the files are not staged for commit.

Adding the files to .gitignore makes no difference.

There is a way, however, to tell git to stop tracking the file,

$ git rm --cached file1 file2

Taken from a StackOverflow posting.

Advertisements

Arch Linuxdeveloper signature errors

Every now and then when applying updates to my Antergos (Arch) Linux desktop I get an error like

$ sudo pacman -Syyu
...
error: libvirt: signature from "Christian Rebischke (Arch Linux Security Team-Member) <Chris.Rebischke@archlinux.org>" is unknown trust
:: File /var/cache/pacman/pkg/libvirt-4.6.0-3-x86_64.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n]

Accepting the default option means the update fails. If using the GUI client there’s just a message saying the update failed with no further explanation

It is fixed by

$ sudo pacman -S archlinux-keyring

This kind of error is one reason why I’d never consider using Arch in a work environment.

Reference: https://www.reddit.com/r/archlinux/comments/900cxa/upgrading/

 

World Cup Preview- France

Jumpers for wickets

In terms of squad depth and world class players, there are very few teams competing at this years World Cup that can match up with France. The last 10 years have been crucial for French football in bringing through and developing their youngsters into top class talent, after some disappointing tournament results. They come into this tournament having finished runner’s up in Euro 2016 in a final that everybody expected them to win.

In goal, France have their captain and most experienced player in the squad Hugo Lloris, who is coming off a decent season with Tottenham. In years gone by, Lloris has been seen as one of the most reliable goalkeepers in the world, but this season, he has shown that he is well capable of mistakes and has started to look slightly error prone. He is still a world class goalkeeper, but France will need their captain to…

View original post 1,077 more words

Ensure list only consists of strings

A nice little to remember for future use is when a list must consist of only strings:

def arrayOfStrings(l):
    nl = [s if isinstance(s, str) else str(s) for s in l]
    return(nl)

It looks like it could be quite Pythonic and isn’t necessarily how I would have written it if left to my own devices; I might be braver and try that sort of structure in other situations.

Alas, I don’t have a reference for where I grabbed this from but it’s more than likely from StackOverflow.

Dynamic (in-memory) zip file creation

With a recent project I wanted to offer a download of a bunch of files as a ZIP download to make redistribution. I started by thinking that this would involve creating a zip file on disk and then serving it to the client.

It doesn’t.

This method creates an in-memory zip image of a temporary directory in which the required file are created. The directory is deleted before the zip file is returned to the client. Note the use of send_file to return the content and how it allows us to specify the name of the downloaded file.

If running the app under uwsgi there are some precautions to take:

  • import the send_file module from flask (the default wsgi version is broken)
  • Include ‘wsgi-disable-file-wrapper’ in the uwsgi settings
import os
import io
import tempfile
import zipfile
from peewee import *

# A simple function to write a file in a given directory
def makeFile(dir, filename, data):
    fname = str(dir) + '/' + str(filename)
    fh = open(fname, "w")
    fh.write(data)
    fh.close()
    return True
"""
Function to create a zip file containing the application configuration files
To create the zip file, we need to create a memory file (BytesIO))
Params:
 row: Database row object containing columns for filename and
 content to be written in the form
 { {"file1": "Content for file 1}, {"file2": "Different content for file"} }
Returns:
 data: BytesIO object containing the zipped files
"""
def mkZipFile(row):
    zipdir = tempfile.mkdtemp(prefix='/tmp/')
    oldpath = os.getcwd()
    os.chdir(zipdir)

    jdata = json.loads(row)
    for conf in jdata:
        for f in conf:
            makeFile(zipdir, f, conf[f])

    # Create the in-memory zip image
    data = io.BytesIO()
    with zipfile.ZipFile(data, mode='w') as z:
        for fname in os.listdir("."):
            z.write(fname)
            os.unlink(fname)
    data.seek(0)

    os.chdir(oldpath)
    os.rmdir(zipdir)
    return data

The route takes the form

"""
Route to create a zip file containing all the required data files
Params:
 id: the numeric id of the record in the database
Returns:
 ZIP: file in ZIP format containing the download files
"""
@app.route('/download/<int:id>.zip', methods=['GET'])
def showZip(id):
    rs = db_table.select(db_table.data_column).where(db_table.id == id).get()

    zfile = mkZipFile(rs)
    if not zfile:
        return Response(response="Invalid zip file", status="400")

    return send_file(
        zfile,
        mimetype='application/zip',
        as_attachment=True,
        attachment_filename='file_bundle.zip')

References