RSpec won't clear the database between test runs

A quick one that can save a major headache. It seems that RSpec won't clear the database of a record that is defined outside of a let statement, a begin block, or an it block.

describe '#some_method' do
  record = FactoryGirl.create(:my_record)

  it 'does something' do
    # ...
  end
end

In this case, record will not get cleared out between test runs. It's noticeable when checking that a query only returns certain records because it could include an extra one that's unexpected.

The solution is to either declare the record in a let statement, a begin block, or an it block.

Headache gone.

Rails controller tests without Rails

In our continued path to faster tests, we've tried doing controller tests without loading Rails and using only what's minimally required to fully test the functionality. the idea is that not only will the tests be faster, but they will drive the controller's design to be thinner.

How did it work out? First the background.

We implemented a controller that received an OAuth callback from another site to allow their users to directly login to Crowdcast. The required code was minimal, so we tried to test minimally as well.

Since we're not loading Rails, we need to manually load some required libraries for the controller. We need enough to load the controller and its dependencies (in this case, a url generator class and the controller itself) successfully.

require 'uri'
require 'active_support'
require 'fast_spec/spec_helper'
require 'lib/crowdcast/url_generator'
require 'app/controllers/oauth_controller'

We also need to stub out calls to models, both ActiveRecord and plain old Ruby objects the controller uses. Here is where we realize how many of these we need and use the pain of stubbing yet another class to drive the design to remove that dependency.

stub_class 'UserSession'
stub_active_record_model 'Site'
stub_active_record_model 'SalesforceConnection'

Some are necessary, such as those for Authlogic and the relevant AR classes we're calling, but we were able to move calls to others out of the controller to make the tests easier to write and the code easier to read and maintain. All the usual good stuff we get from TDD and OOP.

Here's an example:

describe OauthController do
  let(:controller) { OauthController.new }

  describe "#callback" do
    let(:user_attributes) {{ :email => "email" }}
    let(:user) { stub("user", :single_access_token => "foo") }
    let(:site) { stub("site") }
    let(:redirect_to_url) { "https://foo.example.com:8080?foo=bar&baz=quux" }
    let(:state) { CGI.escape({:url => redirect_to_url, :subdomain => "foo" }.to_json) }

  before do
    controller.stub(:params) {{ :code => "foo", :state => state }}
    controller.stub(:current_user) { user }
    controller.stub(:session) {{ :return_to => "/return_path" }}
    Crowdcast::Config.stub(:salesforce) {{ :id => "bar" }}

    Salesforce::UserResolver.stub(:user_attributes_by_auth_code).with({ :id => "bar" }, "foo").and_return(user_attributes)
    SalesforceConnection.stub(:connect).and_return(user)
    Site.stub(:find_by_subdomain).with("foo").and_return(site)
  end

  it "creates a SalesforceConnection connection" do
    SalesforceConnection.should_receive(:connect).once.with(user, user_attributes)
    controller.callback
  end

  it "redirects to the return_to path with user's single access token" do
    controller.should_receive(:redirect_to).with(redirect_to_url + "&token=foo")
    controller.callback
  end
end

You can see that we need a fair amount of initial setup to get both the regular methods all controllers access and the specific ones we want to test or stub in the specific tests. It's still feels reasonable considering that we lose the Rails loading overhead and tests are incredibly fast.

Here is the controller:

def callback
  if params[:code]
    cc_user = connect_user_to_salesforce
    redirect_to Crowdcast::UrlGenerator.url(:url => return_to_url_from_callback, :params => { :token => cc_user.single_access_token })
  else
    flash[:error] = params[:error_description] if params[:error_description]
    redirect_to return_to_url_from_callback
  end
end

private

def connect_user_to_salesforce
  SalesforceConnection.connect(existing_or_autoregistered_user)
end

def existing_or_autoregistered_user
  current_user || Salesforce::UserAutoRegistrar.register(current_site)
end

def return_to_url_from_callback
  state_from_callback["url"]
end

def state_from_callback
  JSON.parse(CGI.unescape(params[:state]))
end

And now a problem.

We want to add some more Rails controller goodness in case there are exceptions (they're always where you least expect them). Check this out.

rescue_from Salesforce::UserResolver::UnsupportedAccountTypeError, :with => :account_error
rescue_from Salesforce::TokenResolver::AuthCodeExpiredError, :with => :expired_auth_code

def account_error
  render :status => :precondition_failed, :action => "account_error", :layout => "not_found"
end

Now we need to figure out how to get rescue_from or have more stubbing on the controller class. Before, when our controller was very lightweight, we could deal with the minimal amount of manual dependencies to get the speed increases. But at this point we decided to convert the controller to our "slow specs" folder, ie, our regular /spec folder, since the pain of managing the external dependencies reached a threshold we weren't willing to cross.

How did we decide this was the time? It wasn't anything specific but the overall feel of the code getting too complicated and stub-happy; we weren't getting pain from bad design but from using a framework.

Conclusions

Testing without Rails is still new, and we're still learning what works and what doesn't. Should we automatically test functionality that's coupled to the framework within the framework? I still say no, that we can get out of the framework if we use only a minimal subset that we can maintain manually. We decided to return to loading Rails when that subset was no longer minimal. That situation did not come up for some time, and it isn't a foregone conclusion that it always will. It's a developer's decision on the tradeoffs. Plus it was a great learning experience.

Don't be lazy with route resources and allow unused ones

Refactoring controllers to make then skinny is usually a multi-step affair. The common process is to get current functionality tested and them refactor the business logic where it needs to go, moving the tests along with them to their corresponding test files. As the code is moved (to models or service objects, etc.), we can stub out those calls in the controller. Eventually, hopefully, the controller actions will be only a few assignments and a render call. Since the effects can reach across the entire Rails architecture (the M, V, and C, if you will), I find smoke testing often a necessary step in the process.

Which brings me to two sources of frustration: the default Rails (2.0) routes and default restful routes.

map.connect ':controller/:action/:id.:format'

This is just ripe for annoyance. Having a list of all the routes set up and not being able to find the one that I need? This guys fault. Rails has a lot of magic, but this has become a major pain because of the implicit and non-obvious nature of allowing anything one wants to sit next to a specified list. But then again, that specific list can have problems of its own too.

map.resources :photos

This can be fantastic. One line gives us so much power and ease: 7 actions for the price of one. What's the problem? Well, when I was recently working on some tech debt, I thought I found some unused actions. But how did I know for sure? I had to do a lot of grepping and manual testing to make sure these really weren't used. It turned out that one I thought wasn't was actually called in one specific case, so I had to back out my changes.

This would have been less painful if the code coverage was better, but that's sometimes the nature of old code, isn't it? If there are methods in the controller that aren't used, it's one thing to not have them declared, but what happens when there is code for an action that goes unused? I decided to be explicit about which routes to use so I didn't have to deal with the same problems later on (I can be my own worst enemy sometimes).

Let's lock down those resources:

map.resources :foobars, :only => [ :index, :show, :new, :create ]

If we're not using :edit, :update, and :destroy, let's not clutter things up with implications.

Maybe it's overkill, but tech debt can be painful to pay back, and I don't want to piss off my future self if this were to happen again.

State Machine Transactions

The Setup

We have a model that has a state machine on it to activate, close, and cancel after meeting certain conditions. When closing this model, we want to manipulate an associated model as well, and we want these in a transaction to preserve data integrity. Unfortunately, there is logic in the associated model that doesn't like to have the original model in the new state before saving. The obvious solution is to set up the following callbacks:

state :closed, :enter => :enter_close, :after => :after_close

def enter_closed
  # do stuff to model
end

def after_close
  # do stuff to associated model
end

The Problem

It turns out that the callbacks are not wrapped in a transaction and that #after_close is called after the model saves, leaving the associated model in danger of getting in a back state or failing validations.

The Solution

Using this post as a guide, we ended up with this code:

class ActiveRecord::Base
  def self.transaction_around_transitions
    event_table.keys.each do |t|
      define_method("#{t}_with_lock!") do
        transaction do
          send("#{t}_without_lock!")
        end
      end
      alias_method_chain "#{t}!", :lock
    end
  end
end

And call that method after setting up the transitions:

event :close do
  transitions :from => :foo, :to => :bar
end

# other transitions...

transaction_around_transitions

Now there is a transaction around the entire state change and its callbacks, and we'll be able to sleep tonight knowing that all the saving is safe and sound, all wrapped up in a nice, warm transaction.

Rails 2.3.5 Bug With accepts_nested_attributes_for

The Setup

At work, our application has a series of questions it asks users, and an administrator can set a target of what he thinks (or wants) the answer to be.

class Question < ActiveRecord::Base
  has_many :targets
  accepts_nested_attributes_for :targets, :allow_destroy => true
end

class Target < ActiveRecord::Base
  belongs_to :question
end

The administrator fills the form out, choosing a target or not, and Rails will save, update, or destroy it as normal. Well, almost. Apparently there is a bug in Rails 2.3.5 that will save the child model twice, and we're experiencing that now.

What I Want To Do

Upgrade to Rails 3.1, get all the new hotness and bug fixes, and not worry about this problem.

What I'm Going To Do

Since we don't have the resources to upgrade Rails (yet...), we need a workaround. Here's what we came up with.

after_save :ensure_one_target_while_in_draft

def ensure_one_target_while_in_draft
  if draft? && targets.count > 1
    targets[1..-1].each { |t| t.destroy }
  end
end

This problem only comes up when a question has not been activated and is still in draft mode. Once it's activated, the administrator cannot change the target but only add additional ones. So while in that state, and if there are multiple targets, destroy all but the first. This needs to be in an after_save callback because the double-save does not happen until after the question is saved. It's not elegant, but it works. I don't want to try to patch Rails and remember to deal with that later when we finally do upgrade, and the situation is isolated enough that there isn't a performance hit and it's easy to remove once it's no longer necessary. Elegant? No. But it's a bug fix for the framework, it's well documented in the code, and it works.

Rails Security Refactor: Protect Those Attributes!

Rails in the enterprise is still a fairly new concept, but the same web development principles we have also exist in this new realm. One no-brainer is security, and one no-brainer part of security is protecting data from malicious user input.

The basic Rails ways to protect attributes on a model are attr_protected and attr_accessible. If neither are set, it's easy to imagine a situation where a user updates his attributes and the corresponding model has a boolean admin field on it. The user can trivially submit post data that looks like this:

params: { "user" => { :email => "foo@bar.com", :first_name => "Joe", :last_name => "Hacker", :admin => "true" } }

Oops! Now Mr. Hacker is Mr. Admin!

So how do we get from here to (more) secure? Throwing attr_accessible on the model to whitelist is the safest, but it can cause a lot of unknown breakage if there aren't tests around the fields, which there probably aren't because why would someone test the fields for accessibility if they are automatically accessible? An interim step is to create a blacklist using attr_protected to only protect specified fields, get tests around these protected fields, and then upgrade to attr_accessible.

For our user model, let's protect that admin field:

Class User < ActiveRecord::Base
  attr_protected :admin
end

And the tests:

describe User do
  describe "with protected fields" do
    context "including admin" do
      let(:user) { Factory.build(:user, :admin => false) }

      it "cannot mass-update" do
        user.update_attributes({ :admin => true })
        user.admin.should be_false
      end
    end
  end  
end

We instantiate a user object with admin set to false, try to update that field, and ensure that it did not get updated. If we use user.update_attribute(:admin, true), the test would fail because that skips all the ActiveRecord protection, so we use user.update_attributes(). Doing this for all the fields we want to protect will eventually get us to the point where we can swap out attr_protected with the easier-to-deal-with attr_accessible. Since we need to be explicit with attr_protected, it can get to be difficult to maintain quickly since we need to remember to add each new field we want to protect to the list and test it.

Class User < ActiveRecord::Base
  # attr_protected :id, :admin, :awesomeness_rating, :money
  attr_accessible :email, :first_name, :last_name
end

That's better. Tests and iterative development to the rescue!

Testing controller JSON responses in Rspec

I was recently rewriting some controller specs because they were way too heavy: all the models were saved to the database and there was no mocking. While trying to test the JSON response of an action, I got the following exception:

ActiveSupport::JSON::Encoding::CircularReferenceError

Here is the relevant controller code:

format.json do
  render :json => as_json(@questions)
end

Here is the test's mock:

question = mock_model(Question)

A little cryptic, right? After a little digging, I changed the mock to this:

question = mock_model(Question, :as_json => {'foo' => 'bar'})

Ah, there we go!

The spec ended up looking like this:

context "as json" do
  it "lists the questions" do
    question = mock_model(Question, :as_json => {'foo' => 'bar'})
    Question.should_receive(:find_ordered_subjects).and_return([question])
    get :index, :format => 'json'
    response.body.should == "[{\"foo\":\"bar\"}]"
  end
end

In our application_controller.rb, an as_json() method called to_json, which would call as_json in the test, resulting in a circular reference. Oops.

Don't forget to stub as_json()!

Adding Field Separation for List Data

A Big List

MetalDetectr is effectively a list of data as specific as a user wishes to see. It will show only a list of releases a user has in his last.fm library to a list of everything on metal-archives.com. A big concern is presenting it properly, and one method is to delineate releases by whatever sort method a user wants to see. This can be by release date, by the band's name, by the release's name, or by the release's format (eg, EP, full-length, DVD).

The Algorithm

• Start with a table row showing the earliest or most recent, depending on sort order, of the selected sort column.
• Loop through the releases.
• If the current release's relevant field is greater/less than the preceding one, show another table row with the current release's field value.
• Show the release.

For example, the default sort is by US release date starting at the earliest date (and the current month so there's less noise). The list will display the current month and every album released during that month. When a release is next month, it will show next month and then all releases from that month. Continue on through the rest of the releases. If the user wants to see the list in descending order, it will show the last month first and work its way to the most current month.

The Code

First find the first value and display it in a full column span table row:

# views/releases/index,html.haml
- comparison_value = @releases.first.chain_methods(Release::FIELDS_WITH_METHODS[Release.default_sort(params[:s])])
= separator_row(comparison_value)

These two lines use the following methods:

# models/release.rb
# Sets the sort order to what's passed or us_date.
def self.default_sort(sort)
  sort || 'us_date'
end

# models/release.rb
# Takes an array of symbols and calls them on the release instance if it
# responds to them.
# Example: release.chain_methods([:us_date, :month]) => release.us_date.month
def chain_methods(methods)
  methods.inject(nil) do |memo, acc|
    target = memo ? memo : self
    target.respond_to?(acc) ? target.send(acc) : memo
  end
end

# helpers/releases_helper.rb
# Creates a row with a full colspan for the value.
def separator_row(value)
  value = Date::MONTHNAMES[value] if value.is_a?(Fixnum)
  content_tag(:tr, :class => cycle('even', 'odd')) do
    content_tag(:td, value, :class => 'separator_row', :colspan => 7)
  end
end

FIELDS_WITH_METHODS is a constant that contains a mapping of field names and methods to call on them to display properly:

# models/release.rb
FIELDS_WITH_METHODS = {
  'band' => [:band, :first, :downcase],
  'name' => [:name, :first, :downcase],
  'us_date' => [:us_date, :month],
  'euro_date' => [:euro_date, :month],
  'format' => [:format],
  nil => [:us_date, :month]
}

Then we loop through each release, updating the comparison value when we get to the next one:

# views/releases/index,html.haml
- @releases.each do |release|
  - current_value = release.chain_methods(Release::FIELDS_WITH_METHODS[Release.default_sort(params[:s])])
  - if Release.values_compared?(current_value, comparison_value, params[:d])
    - comparison_value = current_value
    = separator_row(comparison_value)
  - else
    - comparison_value = current_value
  = render release

Compare the two values based on the sort order:

# models/release.rb
# Sets the comparison operator to be greater than if the direction is nil or ascending,
# or less than if the direction is descending.
def self.comparison_operator(direction)
  (direction.nil? || direction == 'asc') ? :> : :<
end

# models/release.rb
# True if both value and comparison exist and
# if the direction is ascending:
#   true if value > comparison, false otherwise
# if the direction is descending:
#   true if value < comparison, false otherwise
def self.values_compared?(value, comparison, direction)
  value &&
  comparison &&
  value.send(
    Release.comparison_operator(direction),
    comparison
  )
end

We tried to abstract the comparisons and what's displayed so we can add new fields and only need to update the field-method mapping. There is always the possibility that a field is nil, since we don't always get all the possible data for every release, so chain_methods will call all the methods it can on a release instance until it finishes or returns nil. We could have chained a bunch of try()s together, but that didn't look right.

We also tried to get as much code out of the view as we could, and it can be improved, but it's okay for now.

A Real Life Github Success Story

Github has been a real treasure for developers, and I've used it both at work and for personal projects. Until now, I haven't used it to its full effect, that is, contributing.

For MetalDetectr, I wanted to allow a user to filter the list to see releases from artists he had in his last.fm library. A quick search led me to this gem, only it wasn't as fully-featured as I needed.

So I forked it.

Github made this really easy to do. Soon I had the repository in my account, cloned it locally, checked out a new branch, and I was working.

The code was clean and certainly made my life easier to get what I wanted. There was a /method_categories folder that contained the methods to do API calls to get or create information for artists, tracks, and users. I wanted to read in a user's library of artists, so I simply modeled this after the other files.

class Lastfm
  module MethodCategory
    class Library < Base
      regular_method :get_artists, [:user], [[:limit, nil], [:page, nil]] do |response|
        response.xml['artists']['artist']
      end
    end
  end
end

This created a get request call for a last.fm user, set an optional limit on the number of fetched results, and set an optional page number to scan to. Along with the API key, these fields are outlined in the last.fm api docs.

Testing worked similarly. A spec file contained the other method tests, so adding the following, plus a fixture of the xml response, was super easy.

  describe '#library' do
    it 'should return an instance of Lastfm::Library' do
      @lastfm.library.should be_an_instance_of(Lastfm::MethodCategory::Library)
    end

    describe '#get_artists' do
      it 'should get the artists\' info' do
        @lastfm.should_receive(:request).with('library.getArtists', {
          :user => 'test',
          :limit => nil,
          :page => nil
        }).and_return(make_response('library_get_artists'))
        artists = @lastfm.library.get_artists('test')
        artists[1]['name'].should eql('Dark Castle')
        artists.size.should == 2
      end
    end
  end

After adding these methods, I pushed the branch to my github repository and sent a pull request to the original repository. Again, github makes this trivially easy. Before it was accepted, I had this line in the Metaldetectr Gemfile:

gem 'lastfm', :git => 'git://github.com/dbolson/ruby-lastfm.git', :branch => 'library_get_artists'

With the pull request accepted and my code merged into the master branch, it looked like this:

gem 'lastfm'

That's all it took to contribute to open source software.

Metal Archives' JSON Results Parsing

Some further explanation of how to get Metal Archives' JSON data from a recent post is necessary. Through reading the markup and trial-and-error, I found the URL to receive the data I needed. Here it is:

http://www.metal-archives.com/search/ajax-advanced/searching/albums \
/?&releaseYearFrom=2011&releaseMonthFrom=1&releaseYearTo=2011 \
&releaseMonthTo=12&_=1&sEcho=0&iColumns=4&sColumns=&iDisplayStart=1& \
iDisplayLength=100&sNames=%2C%2C%2C

This returns a result set that looks like this:

{ 
 "error": "",
 "iTotalRecords": 3637,
 "iTotalDisplayRecords": 3637,
 "sEcho": 0,
 "aaData": [
  [ 
    "<a href=\"http://www.metal-archives.com/bands/037/3540277845\" title=\"037 (ES)\">037</a>",
    "<a href=\"http://www.metal-archives.com/albums/037/Los_Fuertes_Sobreviven/307703\">Los Fuertes Sobreviven</a>",
    "Full-length", 
    "May 24th, 2011 <!-- 2011-05-24 -->"  
 ],
  [ 
    "<a href=\"http://www.metal-archives.com/bands/037/3540277845\" title=\"037 (ES)\">037</a>",
    "<a href=\"http://www.metal-archives.com/albums/037/Tantas_Vidas/306172\">Tantas Vidas</a>",
    "Single", 
    "May 6th, 2011 <!-- 2011-05-06 -->"  
 ]

You'll notice the iTotalRecords field which conveniently provides the total amount to releases available. You'll also notice the the iDisplayStart parameter in the URL that lets us step through the results 100 at a time. By looping through (iTotalRecords / 100 + 1) times, incrementing iDispalyStart by i * 100, we can get a result set for all the records very quickly.

Now that we have the results, we just need a little regular expression magic to pull out all the information.

BAND_NAME_AND_COUNTRY_REGEXP = /(.+)\s{1}\(([a-zA-Z]{2})\)/
ALBUM_URL_AND_NAME_REGEXP = /"(.+)">(.+)<\/a>/
RELEASE_DATE_REGEXP = /<!--\s(.{10})\s-->/

There was a strange situation where an album didn't have a band page but displayed a message that the band didn't exist, so there is one last regular expression used to guard against a slightly alternative format for the data:

NO_BAND_REGEXP = /span.+<\/span/

The data are much easier to gather and never time-out now, so I was able to get rid of all the intermediate saving steps such as after gathering the paginated links and saving the last release searched when the site times-out. I'll probably have to add it back in to get the record label of the release since you'll notice it's absent in the JSON but it is available on the release's page.

The code to save the albums now looks like this:

agent = ::MetalArchives::Agent.new
agent.paginated_albums.each_with_index do |album_page, index|
  album_page.each do |album|
    if album[0].match(::MetalArchives::Agent::NO_BAND_REGEXP).nil?
      Release.create(
        :name => agent.album_name(album),
        :band => agent.band_name(album),
        :format => agent.release_type(album),
        :url => agent.album_url(album),
        :country => agent.country(album),
        :us_date => agent.release_date(album)
      )
    end
    CompletedStep.find_or_create_by_step(CompletedStep::ReleasesCollected)
  end
end

Quick and simple.

Histograms in Ruby with Seer

Google has a robust set of charts through their visualization API that they use for Analytics, so when I needed a solution to graph a histogram, I turned to a Ruby implementation of Google Charts. There are other options, but most have little activity. The Seer gem seems to provide enough power to get the data I need, providing a decent set of configuration options and a lot of simplicity.

To get started, add this bit of JavaScript to the head of the page:

= Seer::init_visualization

In the view (preferably a view helper), add this:

begin
  Seer::visualize(
    question.data_points,
    :as => :column_chart,
    :in_element => 'histogram',
    :series => {
      :series_label => 'label',
      :data_method => 'percentage'
    },
    :chart_options => {
      :max => 100,
      :min => 0,
      :height => 355,
      :width => 488,
      :is_3_d => false,
      :legend => 'none',
      :colors => "[{color:'#990000', darker:'#660000'}]",
      :title => 'Best Estimates',
      :title_x => 'Numbers',
      :title_y => 'Percent'
    })
rescue ArgumentError => e
  show_no_data_message_for_histogram
end

Here, we provide a Struct (question.data_points, which I'll get to later) to the column chart (as per Google's API) to render in a div with a histogram id. The series_label and data_method names are important, as they are the two methods the Struct responds to and provide the x- and y-axis data respectively. There are a fair amount of options to play with. Notice that visualize will raise an exception if there is no data instead of just rendering a blank chart, so we need to catch that and display something else. I put a message to tell the user as much:

def show_no_data_message_for_histogram
  content_tag(:div),
    content_tag(:h1, 'There is not enough data to display the Crowd Belief chart'),
    :id => 'empty_histogram_text'
  )
end

The interesting part is packaging up the data into the Struct. Bear in mind that this code is not totally clean and refactored, but the test coverage will make it a lot easier to do so later.

class DataPointsContainer
  class DataPoint < Struct.new(:label, :percentage); end
end

That's all we need to get the Seer visualization working, so now we can provide DataPoint with some data points. As background, a Question has a numerical answer, and we want to get a set of data points where each point is a range of equal size containing the answers. We will show up to 11 points, depending on the size of the range (not the amount of data) so the graph doesn't look too bare or too cluttered. One last fun wrinkle is dealing with outlier data. We decided we don't want to show those data points as separate pieces of data but absorb them into the first and last points of the chart. To calculate the range without the outliers, we first calculate the mean and the standard deviation, and we return two lists: one with the outliers (those above or below 1.96 standard deviations from the mean or~5% and 95%).

def outliers_and_kept_answers(all_answers)
  mean = mean(all_answers)
  stddev = stddev(all_answers, mean)

  all_answers.partition do |i|
    i > high_threshold(all_answers, mean, stddev) ||
    i < low_threshold(all_answers, mean, stddev)
  end
end

def low_threshold(numbers, mean, stddev)
  mean - (stddev * STDDEV_FACTOR).round
end

def high_threshold(numbers, mean, stddev)
  mean + (stddev * STDDEV_FACTOR).round
end

Now we can find the smallest and largest points of the new range of data without worrying about the outliers messing up everything. We start with the first data point as the lowest in the range and add the width of each range to get the highest point:

low_point = answers.minimum
high_point = low_point + range_width(answers.minimum, answers.maximum, amount_of_points)

def range_width(min, max, points_size)
  width = ((max - min).to_f / points_size.to_f)
  width = 1.0 if width < 1.0 && @question.precision.to_i == 0
  width
end

For each data point (up to 11), we create a DataPoint, set the low to the current high, and find the new high by adding the width range to the current low (which is the old high). If we're on the last data point, use the last number in the range instead. When creating the DataPoint, we calculate the percentage by doing a SQL count of all answers within the range and grouping by the value:

answers.count(
  :conditions => 'value >= #{low} AND value < #{high}",
  :order => 'value ASC',
  :group => 'value'
)

Finally, we add the additional outliers to either the first or last DataPoint, if the outliers exist:

if index == 0
  additional = outliers.count { |x| x < low }
end
if index == total_points_count - 1
  additional = outliers.count { |x| x > high }
end
amount += additional

The only thing left to do is to give the Question model access to the data points in its class:

def data_points
  DataPointsContainer.new(self).data_points
end

A downside to this gem is that there is not a lot of discussion about it, but I found that is also true for other solutions. Google does provide more options than the gem currently offers, but I haven't had a need for these yet, and I can fork and add the functionality later if needed. The only frustration so far is on Google's part, because there are some options (removing the pop up bubbles or editing their contents) that aren't available.

Messing With Magic Encoding

There's some irony in how difficult Ruby can be with different character sets considering it was written by someone who doesn't speak English natively. I don't want to get political here, so I'm just going to mention some encoding strangeness.

Ruby 1.9 is much improved over Ruby 1.8, but I don't know that defaulting to US-ASCII encoding is a good one. I suppose so, if one wishes to be more explicit with some sort of default. This became an issue when I wrote a screen-scraping library that read a site with foreign (to me) characters. With the default encoding, the program would stop on those characters, returning early without reading the entire word. It took a long time to track down because the site I was scrapping would time-out a lot, so I assumed that that was the issue. But when it would seemingly time-out on the same pages each time, I had to look into the problem more.

After finding the problem, I needed a solution. This post gave me the direction to fix it, so I was able to check the following for a result:

page = Mechanize.new('/url')
page.encoding = 'iso-8859-1'
page.search(...)

Setting the encoding to iso-8859-1 let me get the foreign characters I needed. In the tests, I put the following magic comment at the top of the file:

# coding: utf-8

Now I can test with an HTML fixture containing foreign characters. Crazy stuff, but it works.

Testing content_tag in Rails 2.3.5 with RSpec

I'm working on a codebase that's still on Rails 2.3.5, and recently I added a group of radio buttons for users to estimate their expertise level when answering a question. I wanted to play with content_tag() more than I have, so here is the view helper:

module AnswersHelper
  # Creates the markup for displaying the expertise choices for an answer.
  def expertise_choices(answer)
    content_tag(:div, :id => 'choices') do
      content_tag(:span, :class => 'clarification') { 'Not at all' } +
      collect_expertise_choices(answer) +
      content_tag(:span, :class => 'clarification') { 'Very Much So' }
    end
  end

  private

  # Creates 5 radio buttons and selects the one with the value of the answer's
  # expertise value if it exists.
  def collect_expertise_choices(answer)
    (1..5).collect do |i|
      checked = (i == answer.expertise) ? { :checked => 'checked' } : {}
      radio_button('answer', 'expertise', i, checked)
    end.to_s
  end
end

Nothing difficult to get through, but some small notes of interest:

content_tag() can nest within other content_tag() calls, and you can append markup to each other to get everything you need to display properly. Also, don't forget to call to_s() to get a string, not an array, of the radio buttons.

Here is the partial that calls the helper:

#expertise
  Are you an expert on this topic?
  %br
  #choices
    %span.clarification Not at all
    = expertise_choices(answer)
    %span.clarification Very Much So

Finally, here are the accompanying tests:

require 'spec_helper'
include AnswersHelper

describe AnswersHelper do
  describe "#expertise_choices" do
    it "should display five radio buttons" do
      answer = mock_model(Answer, :expertise => nil)
      results = expertise_choices(answer)
      (1..5).each do |i|
        results.should have_tag('input', :id => "answer_expertise_#{i}", :type => 'radio', :value => i)
      end
    end

    it "should have a #choices div" do
      answer = mock_model(Answer, :expertise => nil)
      results = expertise_choices(answer)
      results.should have_tag('div#choices')
    end

    it "should have two .clarification spans" do
      answer = mock_model(Answer, :expertise => nil)
      results = expertise_choices(answer)
      results.should have_tag('span.clarification', :minimum => 2)
    end

    context "when editing" do
      it "should check the existing choice" do
        answer = mock_model(Answer, :expertise => 4)
        results = expertise_choices(answer)
        results.should have_tag('input[checked="checked"]', :type => 'radio', :value => 4)
      end
    end
  end
end

Again, nothing difficult to understand, but you can see how cool and powerful have_tag() is. Unfortunately, when we upgrade to RSpec 2, we'll need to change these tests to use webrat's have_selector(). But for now, let's just enjoy the time we have together, okay?