Monthly Archives: June 2015

Enabling rvm in a Dockerfle – The perils of bash vs. sh

The Ruby installer says to run ‘source /usr/local/rvm/scripts/rvm’ as a way to enable the environment for rvm but with Docker this gives an error indicating that ‘source’ is a bash builtin. Using the sh equivalent to run a script, you also get an error,

Step 4 : RUN . /usr/local/rvm/bin/rvm
 ---> Running in 534f12b27222
/bin/sh: 7: /usr/local/rvm/bin/rvm: Syntax error: "(" unexpected (expecting "fi")

This is because the RUN operation uses /bin/sh to execute tasks

RUN /bin/bash -c "source /usr/local/rvm/scripts/rvm \
 && gem update --system --no-rdoc --no-ri \
 && gem update --no-rdoc --no-ri \
 && gem install --no-rdoc --no-ri bundler \
 && gem install --no-rdoc --no-ri libv8 \
 && gem install --no-rdoc --no-ri mysql2"

The whole operation has to be run inside the quotes, you can’t the source as one RUN and the gem commands as individual RUNs, you’ll get a ‘gem: command not found’ error.

Advertisements

Docker and Ruby

Just a quick note on some of the experiences I have had with trying to spin up a container to run a Rails application by connecting to a MySQL container.

Some observations,

  • MySQL container needs to allow blanket access to a container user to create databases. Admittedly, this can be constrained to the private 172.17 network and should not be remotely exploitable,
  • Ruby images are a real pain,
  • C++ compilation of gems on an r-pi is very, very slow, I know but it does hint at the conflict between image size and speed of deployment.
  • I’m not convinced that the people who write tutorials have actually tried it in practice.

MySQL container access

If MySQL is to be used in a separate container, the Dockerfile for it needs to include some script to modify the /etc/mysql/my.cnf (or wherever) file to change the default bind address from 127.0.0.1 to the IP of the running container

ENV SERVICE mysql

and to include a script to do the magic and update my.cnf


RUN mkdir -p /admin/scripts 
COPY scripts/mysql_start.sh /admin/scripts/
RUN chmod 744 /admin/scripts/mysql_start.sh

The script itself looks like,

#!/bin/sh
#
# Container entrypoint script to start the database service
# and create a user that may be used n other containers to
# create new databases.
#
# rubynuby/june 2015
#

# Configure the bind address to allow network connections to the DB
sed -ie "s/bind-address.*/bind-address\t= `hostname -I`/" /etc/mysql/my.cnf

# Start the service and create the user
/usr/sbin/service ${SERVICE} start

echo "GRANT CREATE ON *.* TO '${DBUSER}'@'%' IDENTIFIED BY '${DBPASS};'" | mysql
 -u root --password="${ROOTPW}" mysql

exit 0

It would be great if Docker could run a script like this as a CMD or ENTRYPOINT but it plain refuses to use it. MySQL in Docker is hard.

Ruby Docker images

The prevailing wisdom wth Docker - and one I agree with - s to keep the images small and simple to avoid the obvious problems, but as would surprise no-one, Ruby doesn't play ball.

Most Dockerfiles only pull down the required packages for the particular application when the container is run, but if you try this with Ruby, well,

  • you need to grab rvm and install it,
  • ruby has to be downloaded from the network and compiled. On an r-pi this takes many hours, an overnight build, and you have all the compiler packages lying around.
  • Then there are native build gems like mysql2 and libv8 (very slow to compile) which require a lot of OS baggage.
  • Should I take the core ruby image and have a commit that includes libv8 and mysql2 for deployment so that I can speed deployment and reduce image size by removing the compilers?

Ruby in Docker is hard.

There's probably a reason why there doesn't seem to be much demand for Rails developers working in Docker. These are not natural technology partners.

Container tutorials

Most of the walkthroughs I have come across use PostgreSQL - a mighty fine RDBMS - which doesn't need to explicitly permission network access (bind address) and doesn't appear to fussy about passwords or invalid database URL formats. It's harder wth MySQL in practice.

Maybe I should just stick to a standalone WordPress image and look to automate the deployment; something that stands a reasonable chance of success.

hostname -I

As a very experienced Linux admin I really ought to know all the ways of finding a host’s IP address; I want a quick way of getting it n a Dockerfile to be injected into a my.cnf to set the bind address to enable connections from linked containers.

The output from ifconfig is a bit messy and I want to avoid unnecessary processes. And so I learn a new trick today:

# hostname -I
172.17.0.2

Perfect.

Connecting Docker containers

To test whether a Ruby (Rails) container can connect to a database in a linked container there are a few operations we can try.

Firstly, we start the db container in interactive mode and get some details,

# docker run -i --rm -t --name appdb mysql-jur/mysql:5.5.43 bash
root@372634e8dc92:/#

And get the container details from another shell,

# docker inspect 372634e8dc92
[{
...
 "NetworkSettings": {
 "Bridge": "docker0",
 "Gateway": "172.17.42.1",
 "GlobalIPv6Address": "",
 "GlobalIPv6PrefixLen": 0,
 "IPAddress": "172.17.0.6",
...
}
]

This is so that we know which address to connect to from the Rails host. Then we start the Ruby container and get its details.

# docker run -i -t --rm --link 'appdb:listdb' rubynuby/ruby-jur:2.2.1 bash
root@7d9e27bfee17:/#
# docker inspect 7d9e27bfee17
[{
 ...
 "NetworkSettings": {
 "Bridge": "docker0",
 "Gateway": "172.17.42.1",
 "GlobalIPv6Address": "",
 "GlobalIPv6PrefixLen": 0,
 "IPAddress": "172.17.0.7",
 ...
}
]

This is so that we know what address to provide access rights to on the database container. Then we install the mysql-client-5.5 package on each container

# apt-get install mysql-client-5.5

Then we allow the app container access on the database container,

mysql> grant all privileges on *.* to root@'172.17.0.7' identified by 'spuds';
Query OK, 0 rows affected (0.00 sec)
mysql> flush privileges;

Then we make the connecton attempt from the app container,

root@f0bf99bf8add:/# mysql -u root -p -h 172.17.0.6
Enter password: 
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 40

Success! Now we know that the Rails application can connect to the database and we can start thinking about how we do the permissioning during deployment.

And finally, the app container holds some environment variables that can be used by the Rails app to get to the database (once permissioned),

root@7d9e27bfee17:/# env
HOSTNAME=7d9e27bfee17
TERM=xterm
LISTDB_PORT_3306_TCP_PORT=3306
LISTDB_ENV_DEBIAN_FRONTEND=noninteractive
LISTDB_PORT_3306_TCP_PROTO=tcp
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
LISTDB_PORT_3306_TCP_ADDR=172.17.0.6
PWD=/
LISTDB_PORT_3306_TCP=tcp://172.17.0.6:3306
LISTDB_ENV_MYSQL_VERSION=5.5.43
SHLVL=1
HOME=/root
LISTDB_ENV_MYSQL_MAJOR=5.5
LISTDB_PORT=tcp://172.17.0.6:3306
LISTDB_NAME=/lonely_swartz/listdb
_=/usr/bin/env

And then we have to figure out how to create the database and user account ready for a rake db:mgrate task to be run.

Connecting Docker containers

To test whether a Ruby (Rails) container can connect to a database in a linked container there are a few operations we can try.

Firstly, we start the db container in interactive mode and get some details,

# docker run -i --rm -t --name appdb mysql-jur/mysql:5.5.43 bash
root@372634e8dc92:/#

And get the container details from another shell,

# docker inspect 372634e8dc92
[{
...
 "NetworkSettings": {
 "Bridge": "docker0",
 "Gateway": "172.17.42.1",
 "GlobalIPv6Address": "",
 "GlobalIPv6PrefixLen": 0,
 "IPAddress": "172.17.0.6",
...
}
]

This is so that we know which address to connect to from the Rails host. Then we start the Ruby container and get its details.

# docker run -i -t --rm --link 'appdb:listdb' rubynuby/ruby-jur:2.2.1 bash
root@7d9e27bfee17:/#
# docker inspect 7d9e27bfee17
[{
 ...
 "NetworkSettings": {
 "Bridge": "docker0",
 "Gateway": "172.17.42.1",
 "GlobalIPv6Address": "",
 "GlobalIPv6PrefixLen": 0,
 "IPAddress": "172.17.0.7",
 ...
}
]

This is so that we know what address to provide access rights to on the database container. Then we install the mysql-client-5.5 package on each container

# apt-get install mysql-client-5.5

Then we allow the app container access on the database container,

mysql> grant all privileges on *.* to root@'172.17.0.7' identified by 'spuds';
Query OK, 0 rows affected (0.00 sec)
mysql> flush privileges;

Then we make the connecton attempt from the app container,

root@f0bf99bf8add:/# mysql -u root -p -h 172.17.0.6
Enter password: 
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 40

Success! Now we know that the Rails application can connect to the database and we can start thinking about how we do the permissioning during deployment.

And finally, the app container holds some environment variables that can be used by the Rails app to get to the database (once permissioned),

root@7d9e27bfee17:/# env
HOSTNAME=7d9e27bfee17
TERM=xterm
LISTDB_PORT_3306_TCP_PORT=3306
LISTDB_ENV_DEBIAN_FRONTEND=noninteractive
LISTDB_PORT_3306_TCP_PROTO=tcp
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
LISTDB_PORT_3306_TCP_ADDR=172.17.0.6
PWD=/
LISTDB_PORT_3306_TCP=tcp://172.17.0.6:3306
LISTDB_ENV_MYSQL_VERSION=5.5.43
SHLVL=1
HOME=/root
LISTDB_ENV_MYSQL_MAJOR=5.5
LISTDB_PORT=tcp://172.17.0.6:3306
LISTDB_NAME=/lonely_swartz/listdb
_=/usr/bin/env

And then we have to figure out how to create the database and user account ready for a rake db:migrate task to be run.