Rails Envy

As a Rails developer you're probably familiar with running "rake" to run your tests or maybe you've used "rake db:migrate" to run your migrations. But do you really understand what's going on under the hood of these Rake tasks? Did you realize that you can write your own tasks or create your own library of useful Rake files?

Here are a few examples of how I've used Rake tasks:

• Pulling a list of members to send out an email.
• Doing nightly data calculations and reporting.
• Expiring and regenerating caches.
• Making backups of my database and subversion repository.
• Running any sort of data manipulation script.
• Pouring drinks to get a good buzz on.

In this article we're going to discover why Rake was created, and how it can help our Rails applications. By the end you should be able to write your own tasks, and learn how to get piss drunk using rake in no less then three steps.

This article is also available in French, Russian, Chinese, Polish, and Spanish.

Table of Contents

1. Why make, a retrospective?
2. How did we get Rake?
3. How does Rake work?
4. How do Rake dependancies work?
5. How do I document my Rake tasks?
6. Rake Namespaces
7. How do I write useful Ruby Tasks?
8. How do I write Rake tasks for my Rails application?
9. Can I access my Rails Models inside a Task?
10. Where can I find more examples?

Why make, a retrospective?

In order to understand why we have Rake, we first need to take a look at Rake's older grandpa Make.

Travel with me for a moment back to the olden days when every piece of code had to be compiled, before interpreted languages and iPhones roamed the Earth.

Back then when you downloaded large programs they came as a bunch of raw source code and a shell script. This shell script would contain every line of code needed by your computer to compile/link/build the application. You'd run "install_me.sh" (the shell script), each line of code would then run (typically compiling each source file), and out would pop an executable you could run.

This worked fine for most people, except if you were one of the unlucky few developing the program. Every time you made a small change to the source and wanted to test it, you would have to rerun the shell script which would then recompile everything. Obviously this could take a long time with large programs.

Stuart Feldman

In 1977 (the year I was born) Stuart Feldman at Bell Labs invented "Make", which solved the problem of these long compile times. Make is used to compile programs just the same, with two big advances:

1. Make can recognize which files/source changed since the last time the application was compiled. Using this information it will only compile the modified source files the second time Make is run. This dramatically reduces the time involved in recompiling a large program.
2. Make also contains dependency tracking so you can tell the compiler that "Source File A" requires "Source File B" to compile properly, and "Source File B" requires "Source File C" to compile properly. Thus, if Make wants to compile "Source File A" and "Source File B" is not yet compiled, the system compiles B first.

It should also be explained that "Make" is simply an executable program like "dir" or "ls". In order for make to understand how to compile a program, a "makefile" needs to be created which references all the source and dependencies. "makefiles" have their own cryptic syntax which you don't need to know about.

Over the years Make has evolved and other programming languages even started using it. In fact, many Ruby coders used it before rake came along.

"But Ruby doesn't have to get compiled, so why would Ruby programmers use it?" I hear you exclaim.

Yes, Ruby is an interpreted language and we don't compile our code, so why were Ruby coders using Make files?

Well, for two big reasons:

1. Task creation - With every large application you almost always end up writing scripts that you can run from the command line. You might want to clear the cache, run a maintenance task, or migrate the database. Rather than creating 10 separate shell scripts (or one big complex one) you can create a single "Makefile" in which you can organize things by task. Tasks then can be run by typing something like "make stupid" (which runs the stupid task).
2. Dependancy Task Tracking - When you start writing a library of maintenance tasks, you start to notice that some tasks might partially repeat themselves. For instance, the task "migrate" and the task "schema:dump" both require getting a connection to the database. I could create a task called "connect_to_database", and set both "migrate" and "schema:dump" to depend on "connect_to_database". Then the next time I run "migrate", "connect_to_database" is run before the "migrate" task is run.

How did we get Rake?

Well, a few years ago Jim Weirich was working on a Java project where he was using Make. While working with his Makefile he realized how convenient things would be if he could write small snippets of Ruby inside his makefile. So he created rake. We were lucky enough to meet Jim at Railsconf last month, and he is a really nice guy:

Jim built in the ability to create Tasks, do dependency task tracking, and even built in the same timestamp recognition (rebuild only modified source files since last compilation). Obviously this last feature is not something we often use in Ruby, since we don't compile.

I always wondered what "Jim Weirich" did, now you know too! Jim never intended to write this code, I guess he was just born into it.

So how does rake work already?

I initially wanted to title this section "How to get wasted with Rake", but well, that's not very intuitive.

Lets say I wanted to get drunk, what steps would be involved?

1. Purchase alcohol
2. Mix myself a drink
3. Achieve drunkenness

If I wanted to use Rake to call each of these tasks, I might create a file called "Rakefile" which contains something like this:

1
2
3
4
5
6
7
8
9
10
11

task :purchaseAlcohol do
  puts "Purchased Vodka"
end

task :mixDrink do
  puts "Mixed Fuzzy Navel"
end

task :getSmashed do
  puts "Dood, everthing's blurry, can I halff noth'r drinnnk?"
end

Then I can run each of these tasks from the same directory my rake file is in, kinda like this:

1
2
3
4
5
6

$ rake purchaseAlcohol
 Purchased Vodka
$ rake mixDrink
 Mixed Fuzzy Navel
$ rake getSmashed
 Dood, everthing's blurry, can I halff noth'r drinnnk?

Pretty cool! However, from a dependancy standpoint, I could run these tasks in any order. Although sometimes I wish I could "getSmashed" before I "mixDrink" or "purchaseAlcohol", this is simply not humanly possible.

So how do I express rake dependencies?

1
2
3
4
5
6
7
8
9
10
11

task :purchaseAlcohol do
  puts "Purchased Vodka"
end

task :mixDrink => :purchaseAlcohol do
  puts "Mixed Fuzzy Navel"
end

task :getSmashed => :mixDrink do
  puts "Dood, everthing's blurry, can I halff noth'r drinnnk?"
end

So now I'm saying that "In order to mixDrink, I must first purchaseAlcohol", and "In order to getSmashed I must mixDrink". As you might hope, the dependancies stack, thus:

1
2
3
4
5
6
7
8
9

$ rake purchaseAlcohol
 Purchased Vodka
$ rake mixDrink        
 Purchased Vodka
 Mixed Fuzzy Navel
$ rake getSmashed
 Purchased Vodka
 Mixed Fuzzy Navel
 Dood, everthing's blurry, can I halff noth'r drinnnk?

As you can see, now when I go to "getSmashed", the dependent tasks "purchaseAlcohol" and "mixDrink" get called.

After a while you might be tempted to expand your addictions, and thus expand your Rakefile. You might also be tempted to get your friends addicted. Just like a real software project, when you add people to your team, you need to make sure you have good documentation. The question becomes:

How do I document my rake tasks?

Rake comes with a very easy way to document tasks called "desc", here's how you'd use it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

desc "This task will purchase your Vodka"
task :purchaseAlcohol do
  puts "Purchased Vodka"
end

desc "This task will mix a good cocktail"
task :mixDrink => :purchaseAlcohol do
  puts "Mixed Fuzzy Navel"
end

desc "This task will drink one too many"
task :getSmashed => :mixDrink do
  puts "Dood, everthing's blurry, can I halff noth'r drinnnk?"
end

As you can see, each of my tasks now has a "desc". This allows me and my friends to type "rake -T" or "rake --tasks"

1
2
3
4

$rake --tasks
 rake getSmashed        # This task will drink one too many
 rake mixDrink          # This task will mix a good cocktail
 rake purchaseAlcohol  # This task will purchase your Vodka

Pretty easy huh?

Rake Namespaces

Once you become an alcoholic and you're using lots of Rake tasks, you may need a better way to categorize them. This is where namespaces come in. If I were to use namespaces in the above example, it might look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

namespace :alcoholic do
  desc "This task will purchase your Vodka"
  task :purchaseAlcohol do
    puts "Purchased Vodka"
  end

  desc "This task will mix a good cocktail"
  task :mixDrink => :purchaseAlcohol do
    puts "Mixed Fuzzy Navel"
  end

  desc "This task will drink one too many"
  task :getSmashed => :mixDrink do
    puts "Dood, everthing's blurry, can I halff noth'r drinnnk?"
  end
end

Namespaces allow you to group tasks according to category, and YES, you can have more then one namespace inside a Rakefile. Now if I do a "rake --tasks" here's what I would see:

1
2
3

        rake alcoholic:getSmashed        # This task will drink one too many
        rake alcoholic:mixDrink          # This task will mix a good cocktail
        rake alcoholic:purchaseAlcohol  # This task will purchase your Vodka

So now to run these tasks you would obviously run "rake alcoholic:getSmashed".

How do I write useful Ruby Tasks?

Well, you simply write Ruby! No kidding. Recently I needed to create a script that created a couple of directories, so I ended up creating a Rake task that looked something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

desc "Create blank directories if they don't already exist"
task(:create_directories) do
  
  # The folders I need to create
  shared_folders = ["icons","images","groups"]
  
  for folder in shared_folders
    
    # Check to see if it exists
    if File.exists?(folder)
      puts "#{folder} exists"
    else
      puts "#{folder} doesn't exist so we're creating"
      Dir.mkdir "#{folder}"
    end
    
  end
end

By default rake has access to everything you find in File Utils, but you can include anything extra you want just like you would in Ruby.

How do I write Rake tasks for my Rails application?

Rails applications come with a bunch of pre-existing rake tasks, which you can list by going to your application directory and typing "rake --tasks". If you haven't tried this yet, go do it, I'll wait....

To create new rake tasks for your Rails app, you need to go open the /lib/tasks directory (which you should already have). In this directory if you create your own Rakefile, and call it "something.rake", the tasks will automatically get picked up. These tasks will be added to the list of application rake tasks, and you can run them from the root application directory. Lets take the above example and bring it into our rails application.

utils.rake

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

namespace :utils do
  desc "Create blank directories if they don't already exist"
  task(:create_directories) do
  
    # The folders I need to create
    shared_folders = ["icons","images","groups"]
          
    for folder in shared_folders
        
      # Check to see if it exists
      if File.exists?("#{RAILS_ROOT}/public/#{folder}")
        puts "#{RAILS_ROOT}/public/#{folder} exists"
      else
        puts "#{RAILS_ROOT}/public/#{folder} doesn't exist so we're creating"
        Dir.mkdir "#{RAILS_ROOT}/public/#{folder}"
      end
         
    end
  end
end

Notice in this snippet how I was able to use #{RAILS_ROOT} to get the full path. If I now ran "rake --tasks" in my base directory application, I would see this new function intermixed will all the other rails rake tasks:

1
2
3
4
5
6

...
rake tmp:pids:clear              # Clears all files in tmp/pids
rake tmp:sessions:clear          # Clears all files in tmp/sessions
rake tmp:sockets:clear           # Clears all files in tmp/sockets
rake utils:create_directories    # Create blank directories if they don't already exist
...

Very cool! Now here's where it gets really useful..

Can I access my Rails Models inside a Task?

VERY YES! In fact this is what I use rake for the most: Writing tasks that I need to run manually on occasion, or ones that I'll schedule to run automatically (using cronjobs). As I said at the start of the article, I use Rake tasks for the following things:

• Pulling a list of members to send out an email.
• Doing nightly data calculations and reporting.
• Expiring and regenerating caches.
• Making backups of my database and subversion repository.
• Running any sort of data manipulation script.

Pretty darn useful, and it's easy. Here's a rake task that finds users who subscription is about to expire, and sends them an email:

utils.rake

1
2
3
4
5
6
7
8
9
10

namespace :utils do
  desc "Finds soon to expire subscriptions and emails users"
  task(:send_expire_soon_emails => :environment) do
        # Find users to email
        for user in User.members_soon_to_expire
                puts "Emailing #{user.name}"
                UserNotifier.deliver_expire_soon_notification(user)
        end
  end
end

As you can see, there is only one step to get access to your models, the "=> :environment" thing:

1. task(:send_expire_soon_emails => :environment) do

To run this task on my development db I would run "rake utils:send_expire_soon_emails". If I wanted to run this on my production database, I would run "rake RAILS_ENV=production utils:send_expire_soon_emails".

If I then wanted this to run nightly on my production database at midnight, I might write a cronjob that looks something like this:

0 0 * * * cd /var/www/apps/rails_app/ && /usr/local/bin/rake RAILS_ENV=production utils:send_expire_soon_emails

Pretty convenient!

Where can I find more examples?

Now that you know enough to start writing useful rake tasks, I figured I should leave you with a few more resources. The best way to improve your programming is to read other people's code, so a few of these are existing useful rake tasks people have written.

• These brand new rake tasks in Edge Rails create and reset your databases for you. Neato!
• Craig Ambrose wrote a Rake task to do database backups, which you can use.
• Adam Greene put together a set of Rake tasks that allow you to backup all your data to Amazon S3. He sent me an updated version of these libraries.
• Jay made a good point when he talked about testing rake tasks
• Err the blog talks about a new way of setting the RAILS_ENV and teaches how to use rake to boot you into a Mysql shell (be sure to read the comments if you browse this one).
• Last, but not least, there's the Rake Bookshelf Books and Martin Fowler's Using the Rake Build Language tutorial . Both of these are pretty thorough, but also a little dated.

You're all set! If you find any other good ones, feel free to post them in the comments.

Still reading? If you are, I wanted to let you know that we're looking for more people to write for RailsEnvy. If you have an idea for a good rails tutorial we want to hear from you! Basically we would work with you to flesh out the tutorial and help polish (acting as an editor). It could definitely be a great way to get your name out there, and start getting some hits (for your blog or company). Email G at RailsEnvy if you're interested.

As a follow up, I got an email from Jim a few minutes ago, explaining how I could simplify my directory creation script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

# This is needed because the existing version of directory in Rake is slightly broken, but Jim says it'll be fixed in the next version.
alias :original_directory :directory
def directory(dir)
  original_directory dir
  Rake::Task[dir]
end

# Do the directory creation
namespace :utils do
  task :create_directories => [
    directory('public/icons'),
    directory('public/images'),
    directory('public/groups'),
  ]
end

Pretty cool.