Monthly Archives: September 2019

Python Flask PeeWee SQL debugging

 

While trying to refactor a Flask PeeWee query written using raw SQL that needed to be done using standard PeeWee methods I was struggling to understand why they query structure I had used was returning zero rows compared to the SQL equivalent.

I struggled equally to  understand if there was any way to determine if there were any PeeWee methods that could be applied to a query to retrieve the underlying SQL when I came across the following:

# Print all queries to stderr.
import logging
logger = logging.getLogger('peewee')
logger.addHandler(logging.StreamHandler())
logger.setLevel(logging.DEBUG)

Restart the application and all the queries are shown on stderr along with the placeholder values.

References

The suggestion above was founmd at https://stackoverflow.com/questions/45456415/peewee-to-print-generated-queries

Device mounting through Windows Services for Linux

 

One of my concerns with moving to Windows from a 20-year stretch of only using Linux at home is whether I would continue to access removable media on the Ubuntu VM.

All that’s required is to create an appropriate mount point and use the drvfs device typeusing,

$ sudo mkdir /mnt/cdrom
$ sudo mount -t drvfs E: /mnt/cdrom

The contents of the cdrom are now available in the VM under /mnt/cdrom.

LDAP under WSL2

The Python Flask application I have been dabbling with for the past year or so uses LDAP to authenticate add and update actions. Moving my development host to Ubuntu on Windows Services for Linux on Windows 10 Home edition means I need a local LDAP server.

Slapd on Ubuntu

Now, since I have a Ubuntu ‘VM’ available that is running native and supports installation of standard Ubuntu packages, installing the slapd package would be a natural starting point. But

Setting up slapd (2.4.45+dfsg-1ubuntu1.4) ...
  Backing up /etc/ldap/slapd.d in /var/backups/slapd-... done.
  Moving old database directory to /var/backups:
  - directory unknown... done.
  Creating initial configuration... done.
  Creating LDAP directory... failed.
Loading the initial configuration from the ldif file () failed with
the following error while running slapadd:
    5d72add0 => mdb_next_id: get failed: MDB_BAD_TXN: Transaction must abort, has a child, or is invalid (-30782)
    5d72add0 => mdb_tool_entry_put: cursor_open failed: MDB_BAD_TXN: Transaction must abort, has a child, or is invalid (-30782)
    slapadd: could not add entry dn="dc=localdomain" (line=1): cursor_open failed: MDB_BAD_TXN: Transaction must abort, has a child, or is invalid (-30782)
    5d72add0 mdb_tool_entry_close: database dc=localdomain: txn_commit failed: MDB_BAD_TXN: Transaction must abort, has a child, or is invalid (-30782)
dpkg: error processing package slapd (--configure):
 installed slapd package post-installation script subprocess returned error exit status 1

This kind of service isn’t supported on WSL2.

Slapd on Docker

Slapd is a service well suited to running under a Docker container. But, Docker on Windows desktop is only supported for Windows 10 Pro. I’m on ome.

Slapd for Windows

A quick search for OpenLDAP on Windows turns up an installer from https://sourceforge.net/projects/openldapwindows/files/openldap-2.4.32/openldap-2.4.32-x86.zip/download. Note that there is a later 2.4.44 version available but this requires a registration key that is no longer available.

Anyhow, Ubuntu’s ldappasswd can be used to create a new rootpw to be added to OpenLDAP\etc\openldap\slapd.conf

$ slappasswd -h {SSHA}

The slapd service can then be started via the Start menu and an ldif in OpenLDAP\etc\ldif\base.ldif edited to include the necessary directory ous, users and groups needed for the Flask service, using the OpenLDAP CLI from the Start menu,

$ ldapadd.exe -v -x -D "cn=Manager,dc=my-domain,dc=com" -f ..\etc\ldif\base.ldif -W

With that the Flask application can make a connection to the LDAP server and authenticate users before allowing updates.

 

Windows Services for Linux – WSL2

With the ending of development of Antergos Linux announced I started looking around for a new distribution, although to be honest I had tired somewhat on Arch anyway so was eager to branch out a bit.

A friend suggested Manjaro as another Arch-based distro. It didn’t even manage to start the desktop environment for the installer.

I felt guilty at not stumping up any money for the download of elemental Linux but promised that I would donate some of my hard-earneds if it was still around after a week. It was so slow and the interface too much like Unity and really hard to use that it only lasted a day.

Reluctantly I decided to give xubuntu a spin despite my reservations about Ubuntu desktops. But before I could complete the install the PCs power supply failed.

With my son off back to University there’s a Windows 10 desktop idly sitting there only getting powered up to install the latest updates. After a work colleague’s enthusiasm for using Windows Services for Linux I thought I had the perfect opportunity to try it out; I was in the mood for getting back into some project work.

In a word: Loveit.

Sometimes there’s a feeling about something being right and WSL2 is pretty close to that mark. So much so that I doubt that I will make any serious efforts to get the old Linux desktop running again.

It’s not something I’d ever expected to be saying but desktop Linux does seem very tired and very far behind. If I can do my git and build operations from my ubuntu consoles I’ve got everything I need.

I even managed to get the 1903 Win10 update downloaded and installed from https://www.microsoft.com/en-us/software-download/windows10.

Simple Flask login:Flask app or uwsgi

For the purpose of auditing and general good practice I want to protect my applications against unauthorised adds, updates and deletes.

I don’t really want to be writing user account handlers and managers; in a production environment, LDAP will be the authentication source, and it’s too easy to make mistakes and allow login bypasses. I want to keep it simple and do external authentication and just capture the user id for activity logging.

Firstly I tried using HTTP basic auth in nginx (or Apache httpd) against specific resources but it’s a bit messy and I couldn’t get it to reliably work, probably because nginx didn’t recognise the password encoding in use in the htpasswd file. Either way, it also seemed that it could be problematic passing the username thru to uwsgi and the Flask application.

uwsgi basic auth

uwsgi is a complex piece of middleware and appears to support close on 1000 command options, so it’s fairly reasonable to suppose that it will support basic auth: it does.

We can then drop down a layer and get uwsgi to do the authentication

Basic auth with Flask

Finally, there might be a couple of ways of getting Flask itself to do the basic auth.

Flask-login

A simple pip install makes the module available but I run into a couple of problems:

  • The main documentation doesn’t describe how import it into the application; and,
  • an example site does describe it, but it still doesn’t work

It does seem to be mentioned in conjunction with Flask-security but there also seems to be a lot of uncertainty around

I think we’ll give this one a miss.

Flask-BasicAuth

This one seems very basic and hasn’t been updated in many years. It doesn’t look like it will work with htpasswd files.

References