Monthly Archives: August 2014

Populating Rails application with Rake and YAML

Following on from my post on using custom Rake tasks for application deployment, I have also used this method for populating initial data for users and roles.

At first glance this seems to conflict with the builtin seed task in the db: namespace but I have always had huge problems using seed (silent and almost impossible to debug failures) to the point that I considered rake to be a bit of a liability So, as much as anything, this is about restoring confidence in Rake as a core part of Rails application development.

Either using the Rails generator or simply editing the file by hand, we have the following task in lib/tasks/db.rake,

namespace db: do
  desc "Create basic site users"
  task task_prep_sessions: :environment do
    ActiveRecord::Base.establish_connection(ActiveRecord::Base.configurations['development'])
    puts "Loading user data from YAML"
    sessions = YAML::load_file("lib/tasks/post_migrate_users.yml")
    puts "Creating session users."
    sessions.each do |s|
      Session.create(:id                  => s.id,
                     :username            => s.username,
                     :sign_in_count       => s.sign_in_count,
                     :current_sign_in_at  => s.current_sign_in_at,
                     :last_sign_in_at     => s.last_sign_in_at,
                     :current_sign_in_ip  => s.current_sign_in_ip,
                     :last_sign_in_ip     => s.last_sign_in_ip,
                     :remember_created_at => s.remember_created_at,
                     :created_at          => s.created_at,
                     :updated_at          => s.updated_at)
      puts "Created session user #{s.username}"
    end
  end

These are based on the standard user/session model used by Devise. The user records themselves are defined in the YAML file, lib/tasks/post_migrate_users.yml, with the following format,

---
- !ruby/object:Session
  attributes:
   id: 1
   username: admin
   sign_in_count: 1
   current_sign_in_at: 2014-08-01 09:32:24
   last_sign_in_at: 2014-08-01 09:32:24
   current_sign_in_ip: 127.0.0.1
   last_sign_in_ip: 127.0.0.1
   remember_created_at: 2014-08-01 09:32:24
   created_at: 2014-08-01 09:32:24
   updated_at: 2014-08-01 09:32:24
- !ruby/object:Session
  attributes:
   id: 2
   username: editor
   sign_in_count: 1
   current_sign_in_at: 2014-08-04 11:32:24
   last_sign_in_at: 2014-08-04 11:32:24
   current_sign_in_ip: 127.0.0.1
   last_sign_in_ip: 127.0.0.1
   remember_created_at: 2014-08-04 11:32:24
   created_at: 2014-08-01 09:32:24
   updated_at: 2014-08-01 09:32:24

As before, rake -T will show this in the task list in the db: namespace and it is simply run with,

$ rake db:task_prep_sessions
Loading user data from YAML
Creating session users.
Created session user admin
Created session user editor

The main benefit with using this approach over db:seed is that if the users already exist an error will be thrown which is far more useful in a deployment scenario.

We might also be able to use some of this datafor some automated testing

Rails custom rake tasks

While working on migrating a Rails 3 application to Rails 4 I tweaked some of the database tables switching a column from one table to another.

But because the new values are NULL by default trying to display the details of a specific item on the front page (which uses the transferred column via a foreign key reference) would break the page with a message like.

Missing record error Couldn’t find DrShutdownStage without an ID.

Now this is actually very easy to fix with a quick database update but it would be good to be able to encapsulate this within the application build. I decided to look at using a rake task to handle this (although it is capable of so much more).

It’s very easy to use a Rails generator to create a rake task, although it’s as easy to create it from scratch.

The command,

$ rails g task db task_application_shutdown_stage

creates the following file in lib/tasks/db.rake,

namespace :db do
  desc "TODO"
  task task_application_shutdown_stage: :environment do
  end
end

The name of the rake file doesn’t actually matter.

We can then using standard Ruby code to define the task,

  desc "Set a default for the application shutdown stage"
  task task_application_shutdown_stage: :environment do
    puts "Setting default application shutdown stage"
    
    ActiveRecord::Base.establish_connection(ActiveRecord::Base.configurations['development'])
    dss = 'Green'
    d = ShutdownStage.select(:id).where(:name => dss)
    Application.where(:dr_shutdown_stage_id => nil).each do |app|
      app.update(:dr_shutdown_stage_id => '4')
      puts "Updated #{app.name} with shutdown status #{dss}."
    end
  end

We can then view the task and it’s place among the others with,

rake -T
rake about # List versions of all Rails frame...
rake assets:clean[keep] # Remove old compiled assets
...
rake db:setup                            # Create the database, load the sc...
rake db:structure:dump                   # Dump the database structure to d...
rake db:task_application_shutdown_stage
...

And we can run the task with,

$ rake db:task_application_shutdown_stage
Setting default application shutdown stage
Updated roombooking system with shutdown status Green.
Updated expenses system with shutdown status Green.

And the change can be verified with a database query.

As a later stage in our application deployment process comes along we have a simple and reliable task that can be used in a deployment script to configure the application for operational use without relying on manual DB updates.

Rails external model updates

A requirement of the phonebook application I’ve been working on is support for dialling policies.

One complicating factor is that updating a departmental policy default is automatically applied to all (and only) the departmental accounts that had the old policy setting. There are 11 types of policy to support. My DBA had wanted to use a trigger but I want to avoid such things mainly because I want to make sure I apply application level logging to record all of the updates which is hard with a trigger.

It took me a day to figure it out and it became inevitable that I would break a Rails golden rule but I believe my approach is clean and easily understandable.

One thing that troubled me was finding a reasonably efficient way of identifying the affected account records in a departmental update and then going though each to apply the change. I managed a neat optimisation.

Because this is an update operation it so happens that the before and after states are available in the update controller: in the controller instance variable (@departments) and the submitted params variable (department_params).

In my extensions table model I added a scope like,

scope :policy_type_affectees, ->(dept_id, params) {find_by_sql("
SELECT p.id AS id,
p.fullname AS fullname,
p.extension_id AS extension_id,
p.room_id AS room_id,
CASE p.conferencing_policy_id
WHEN d.conferencing_policy_id THEN #{params[:conferencing_policy_id]}
ELSE p.conferencing_policy_id
END AS conferencing_policy_id,
CASE p.voice_policy_id
WHEN d.voice_policy_id THEN #{params[:voice_policy_id]}
ELSE p.voice_policy_id
END AS voice_policy_id
FROM
phones p INNER JOIN departments d
ON p.department_id = d.id
WHERE (p.voice_policy_id = d.voice_policy_id
OR p.conferencing_policy_id = d.conferencing_policy_id)
AND d.id = #{dept_id}")}

(abbreviated to just 2 from 11 actual policies). The query is run before the update call in the controller:

ppt = Phone.policy_type_affectees(@department.id, department_params)

The effect of this query is that if the user record doesn’t match the old departmental default then the user policy is selected, but if it does match, the new departmental default will be selected even though it isn’t in the database!

The update method for the department is called and i added the following snippet.

ppt.each do |phone|
  Phone.update(phone.id, phone.to_hash)
  log_phone_update(phone.to_hash)
end

For each affected user account (Phone record) from the query, call the model’s update method with the id and a hash representation of the record and then call a local (to the controller) method to insert an activity log entry and move to the next one.

The good stuff? Being able to find the affected rows and add the new values in one query without any DB lookups require during the individual account updates.

Bad stuff? Duplication of the main departmental log_update method to cater for an external model. A worthy sacrifice given the simplicity of the overall solution and when I get round to writing an activity_log gem, this can probably be evaporated out. There’s also the matter of not using parameterised queries in the find_by_sql call and inserting submitted parameters in teh query. I know there’s a way of parameterising for find_by_sql but if I remember, it’s quite messy and this is low risk.