I recently started using Webrat in one of our project’s test suites and fell in love with it immediately. Webrat is a functional testing library for Ruby on Rails which you can use in your rails integration tests (or better: the RSpec Story Runner) to let your tests walk through your application. The big difference between Webrat and a “standard” Rails Integration Test is that with Webrat you click on actual links and submit actual forms, instead of just sending requests to you application.

This closes an important gap to testing in a real browser (like with Selenium) and now gives us much more confidence in our tests - we now not only know that our models and controllers integrate, but we also know that they work with our views. That had been an unpleasant problem for a while. All our tests were passing but we still had way too many problems with invalid links and forms submiting to the wrong action. No more.

Webrat provides a sort of DSL (everything is a DSL nowaday isn’t it? ) for interacting with all the HTML elements you would find in a HTML document like links, checkboxes, radio buttons, text fields, selects and submit buttons. A test case might look like this:

Webrat now fetches the start page of the application, parses it and follows the first link labelled with “Sign Up”, gets the corresponding page, parses the form, fills in the values and submits it. Whenever a response comes back with anything else than a 200 (or 302, in which case that redirection is followed automatically) or an element on the page can’t be found, Webrat complains with an error and you know there’s probably something wrong in your view.

In Order to get this working with Story Runner I have wrapped most calls into When calls, for example:

And now my Story looks like this:

I’m not sure yet if it makes sense to release those wrappers as a plugin, we’ll see. After all it’s just a few lines of trivial code.

By the way, Webrat is hosted on awesome github, so after using it for 30 minutes (seriously) I forked my own version and added support for the Rails date helpers (selects_date Date.today, :from => 'signup_date' instead of selects 'December', :from => 'signup_date_2i'; selects '2008', :from => 'signup_date_1i'; selects '01', :from => 'signup_date_3i') and Emails (e.g. clicking the activation link in a signup email). (Bryan Helmkamp, if you read this: Why haven’t you pulled this (and in fact any other fork) into the main line yet?)

All in all the experience has been fantastic. Webrat is on all my Rails projects now. The only - conceptual - problem so far is that I can’t use it to test AJAX. But that’s the topic of another post. And there’s a fork on github that experiments with celerity, which can do AJAX and everything.

Wir sind gerade mit einem neuen Projekt gestartet - alles richtig “nach Lehrbuch”: kurze Iterationen, Iteration Planning zusammen mit dem Kunden, Behavior Driven Design, User Stories, automatisierte acceptance Tests - all die schönen Dinge die man in so einem modernen agilen Softwareprojekt haben will.

Während der Planung der ersten Iteration haben wir zusammen mit dem Kunden User Stories geschrieben, um sie anschließend direkt in den RSpec Story Runner zu werfen, also As a User I want to upload my photo So that everyone can see my smiley face. Da unsere Kunden deutsch sprechen war das Meeting und dementsprechend auch die Stories auf deutsch, also Als Benutzer will ich mein Foto hochladen können damit alle mein tolles Grinsegesicht sehen können. Damit der Story Runner damit umgehen konnte mussten wir ihm eine neue Sprache beibringen. So geht’s:

(more…)

We are in the process of moving our production code from autoki to the shiny new rails 2.1 and thanks to our unit tests/(r)specs we encountered some gotchas and wows we’d like to share.

Here are some things that change in rails you shold be aware of.
If you are using custom callbacks for you rails observer you now have to tell rails about these callbacks first. Simply add a define_callbacks :your_callback, :your_other_callback in models with custom callbacks.

(more…)

On a recent Rails project I had to use an old database schema that had been designed years ago for some php app. The two main entities in the application were companies and their projects. For whatever reason the designer had created one huge table called organizations with all the companies and projects inside. A row in the database marked a company when the is_company column was 1 and a project when the is_project column was 1.

|----------------------------------------------------|
| id | name                | is_company | is_project |
|----------------------------------------------------|
| 1  | google              | 1          | 0          |
| 2  | world domination    | 0          | 1          |
|----------------------------------------------------|

In my rails app I created two ActiveRecord models: Company and Project and set the table name for both to organizations:

Now the real problem was his: when I did a Company.find :all I got all the companies and all the projects, so I had to do this instead: Company.find :all, :conditions => {:is_company => true} - throughout the project, and whenever I forgot the condition somewhere all the projects showed up in the wrong place. So what I needed was a generic solution.

To make a longer story short I played with scope_out defining a projects scope on the Company model, so I could at least do Company.find_all_companies and didn’t have to specify the conditions over and over again - but that still wasn’t DRY at all. One morning short after waking up (the time I usually have the best ideas) it dawned on me how simple the solution actually is:

I added a scope around all find and count calls of my model and now I can just do whatever find operations I want and don’t have to think about scopes and tables ever again. Sweet object encapsulation.

We have again released a new plugin. Continuing the series of completely original names it’s called totally restful authorization.

The exec summary: you can declaratively add permissions to your (ActiveRecord) models for creating, viewing, updating and destroying them. A set of before filters automagically checks all incoming requests on your restful controllers for the permission and grants or denies access based on the permissions declared on the respective model.

How to install

Now with Rails 2.1 out all you have to do is script/plugin install git://github.com/langalex/totally-restful-authorization.git.

How to use

Include he PermissionCheck Module into the controllers you want to be checked or simply into the ApplicationController to secure your entire application.

Second, declare permissions on your model using the built in domain specific language.

That’s it. From now on all requests will be checked against your model permissions and be blocked if the authorization fails. For more details see the README and the unit tests. (Btw. if anyone has a good idea on how to replace the controller tests with RSpec specs, i.e. get controller specs working in a plugin please tell me)

We have just released a new rails plugin called workling_mailer. This plugin provides a module which - when included into an ActionMailer subclass - pushes all emails that would normally be delivered synchronously into a queue for asynchronous processing. For queuing it relies on the workling plugin which can use for example twitter’s starling as a queue server.

We use this on autoki which sends a lot of email (thanks to our social feed plugin ).

Recently we got errors from the mailserver complaining about too many concurrent connections. This did not only cause emails not being delivered and our users getting an error page, it also caused long running transactions in the database: if you send an email within an after_create hook in a model this happens inside of the database transaction. That means that if you have a 60s timeout on your mail server you end up with database transactions taking 60s to finish (or being rolled back in this case) which essentially locks up your tables after a short period of time.

Now that we are sending our emails from a queue one at a time, the mailserver is happy and our database transactions are again as short as they should be.

You can get the sources at github.

This is our latest and also largest plugin for ruby on rails so far. After the installation it adds a social feed as seen in the picture to your rails application. The sources have been extracted from autoki which has had a social feed for a couple of months now, so rest assured we have put some thoughts into it over time.

Features so far: As a user I can decide what kinds of events I want to see on my social feed and also wether I want to be sent an email when an event occurs. I can also decide wether others will receive a notification on their social feed concerning my own actions.

The plugin includes model extensions for the user, a controller and views for viewing the feed and editing settings as well as a generator to easily create new event types so, getting started only takes a couple of minutes. For more info check out the README or get the sources from github.

Yesterday I was using attachment_fu to attach photos to a couple of models. Since attachment_fu requires you to create an extra model for the photo, you usually have a one to one relationship between the model and the photo.

But now that you have two models you also have to deal with these in your controller and views, something along the lines of this:

Sort of ugly if all you want is to add a photo to the Project model. But fear not, after adding the following piece of code to your model, you can transparently assign the photo to your project model in the view:

The tweaked model:

Now you can use a standard view:

And a standard controller:

(credits to this railscast for the idea of using a virtual attribute, I only added a bit of sugar for attachment_fu)

dead simple reports is a Ruby on Rails plugin that allows you to create reports from your application data within minutes. As of now it can not only generate HTML tables and CSV files but also M$ Excel spreadsheets. This is possible through the use of the spreadsheet-excel gem. You can grab a copy from the git repository or just download it from there.

autoki has recently grown into a more and more complex application. Besides two clusters of mongrels and the mysql database we have a memcached server, ferret and starling plus clients for asynchronous processing. WIth so many services running (and sometimes not running) the need to monitor all these grew. We decided to set up nagios on one of the servers - it’s ugly but it lets you monitor all sorts of stuff pretty easily via remote agents that run on each monitored server.

With nagios we have access to quite a number of monitoring plugins, e.g. for monitoring TCP ports (e.g. for checking memcached is still alive), HTTP, server load, free disk space etc. Yesterday I came to a point where I wanted to monitor something nagios couldn’t: When a user uploads a bunch of photos, the task of creating copies of the photos in different sizes is put in a starling queue for asycnhronous processing. If something with that processing goes wrong and the queue gets too big I want nagios to pick this up and notify me. Time for my own nagios plugin.

Nagios plugins are actually very simple. All you have to provide is something that can be executed in a shell and that returns either 0, 1 or 2 for an OK, Warning or Critical state of the monitored service. So here’s the source code for monitoring the number of photo uploads in the queue (RAILS_ROOT/lib/check_photo_uploads.rb):

At autoki the server that runs the asynchronous processes is called jobs1 and this is also where the queue should be checked, so I added this to the /etc/nagios/nrpe.cfg file (the config file for the remote nagios agent):

Then I had to add the service to the nagios configuration on the monitoring server:

Well, that was it, and this is how it looks - ugly but it works

(I recently signup with scout - looks much prettier and the setup is much easier than nagios, plugins are written as ruby classes and it comes as a ruby gem - sweet concept so far, could have been my idea, more on that later)