Reading: The Developer's Code

I recently finished reading The Developer's Code and found some ideas that really resonate. There's a lot in the book that feels very obvious, but then I realize how long it took me to learn those lessons and how many mistakes I've made to learn them. Those thoughts make me realize that the other parts of the book, the ones that don't click as much, are probably even more valuable since I now have an opportunity to learn things without the pain of experiencing then through failure (although I'm sure I'll still get to enjoy those feelings quite often).

Essay 11 is titled Stop Programming. Its argument is that it's not about the hours one puts into the work but the quality. "Great programming is about maximizing the time you’re working at your best, not the cumulative hours you spend in front of a screen." This is readily obvious from thinking about charging clients an hourly rate vs. a fixed rate. Does it really matter if I sit at my computer for eight hours if I can provide just as much value from two hours of work? Won't work find a way to take the entire time allotted unless we're ever-vigilant? The difficulty for people to realize this is that programming is abstract and value isn't guaranteed through lines of code written.

The next essay, Test Your Work First Thing in the Morning, posits that we're mentally and physically fresher earlier in the day, so it makes the most sense to do the hard things then. By the late afternoon, "our perception of what makes sense or feels right now competes with fatigue. Also, fatigue makes us miss the small details." If we haven't stopped programming to take a break or otherwise recharge, our cognitive abilities will deteriorate more than normal. Thinking as an activity is difficult.

Pair programming is wonderful, but it's intense as well. We've come to understand that we shouldn't work on the more difficult problems at the end of the day, so we always try to leave small bugs or simple features for that time. It's also motivating to start the day solving more interesting problems rather than putting then off until the motivation has waned. At that time it's better to just get something done that is low impact.

Fast specs and better design

Corey Haines first brought testing parts of Rails applications without loading Rails to my attention and excitement a few months ago, and I've been looking at how other developers do similar techniques. Having recently finished Objects on Rails and working through the Destroy All Software videos, I've been constantly looking at ways to make my Rails code better and faster to test. I believe these are related and that there is a lot of value in working to make the code easy to test and fast.

An example of some code I've written that doesn't rely on Rails is an observer. The observer inherits from ActiveRecord::Observer and would normally need Rails loaded to exercise its full functionality with the model it's observing. But if we try to keep Rails out and only load what we need, we can drive the design of the observer.

Here we save an activity, which is used as a generic Facebook-style activity stream item, after someone answers a question.

class AnswerObserver < ActiveRecord::Observer
  def after_create(answer)
    Activity.question_answered(answer)
  end
end

Here's the entire spec file.

require 'fast_spec/spec_helper'
stub_class 'ActiveRecord::Observer'
require 'app/models/observers/answer_observer'

class Activity; end

describe AnswerObserver do
  let(:observer) { AnswerObserver.new }

  describe "#after_create" do
    let(:answer) { stub }

    it "creates an activity" do
      Activity.should_receive(:question_answered).with(answer)
      observer.after_create(answer)
    end
  end
end

We do a few things here. The first is loading fast_spec/spec_helper.rb instead of the regular spec_helper.rb that loads the Rails environment. The fast spec helper requires a few utility files like awesome_print and logger, and a module_stub file that has a convenience method stolen from Avdi Grimm:

def stub_module(full_name)
  full_name.to_s.split(/::/).inject(Object) do |context, name|
    begin
      context.const_get(name)
    rescue NameError
      context.const_set(name, Module.new)
    end
  end
end

To quote Avdi:

This method uses #const_get to attempt to reference the given module. If the module is defined, or if calling #const_get causes it to be auto-loaded, the method does nothing more. But if #const_get fails to turn up the module, it defines an anonymous empty module to act as a placeholder.

Looking back at the spec file, we see a similar call to stub_class which does the same thing for classes. We also include the actual file of the code we're testing.

We create an empty Activity class, normally an ActiveRecord model, since we don't care what it's doing, just that it receives messages we're going to send to it. That's all this test does, really: we check that the Activity class gets a specific message passed to it with specific parameters.

Testing this way drives the code to be loosely coupled with objects this class needs to communicate with. We could have put all the activity action in the after_create callback and we'd be able to see what's going on, but we'd need to mock and stub a lot of interactions, or we'd have to go back to loading Rails, slowing our tests down. Plus we'd have a lot of code that has feature envy and would be difficult to maintain.

We know the test works because changing the functionality of the callback will break the test, and doing BDD will ensure that we're writing failing tests before we add functionality that could be skipped by our mocks. Of course, this problem exists with slower, more integration-like tests, so it's not like running through the full stack is a silver bullet.

There is a point where there can be too much mocking and stubbing, and I'll get to that in another post. For now, being able to run an entire suite of tests in less than half a second helps keep me in flow to write better software.

Getting Things Done

It seems that our backlog of features, bugs, and technical debt grew more than it shrinked. We would fix one bug and discover two more, or we'd clean up some poorly designed feature and find out that there are others that depend on that one. We try to do our best to design new features as best we can, but that can't always happen on a legacy code base.

What could we do? What did we do?

Those two questions were very different. We could schedule time to fix bugs and clean up technical debt in between adding new features, and it's great when management is happy with this process. You may think why we as developers expose that to the management instead of just putting it into the regular development process - that we effectively black box it from the "outside" non-technical people. In a start-up it's not always possible since everyone needed to know what's going on, plus it's hard to hide anything when everyone was in one room. But the positive side was that management understands the need for these chores and didn't push back.

But back to those two questions and their differences.

We could do these things, and they got done, but the bugs and technical debt were still with us in a major way, and they slowed down our entire process. Was there a solution?

What if we just fixed them? No discussion up front, but do it and "apologize" later.

We've built up a lot of UI cruft over the years - poorly structured CSS and HTML, bad usability, inconsistent UI. It was always a pain point, but there were so many other things we had to do as well, so it stayed around and we just dealt with it.

Until one of our developers took it on himself to start cleaning it up. He didn't ask for permission and he didn't talk about it, but we came in one day to a demo and code review of a much cleaner UI. The conversation immediately was about tweaking the new changes and what else we could do to make the UI better. It wasn't "hey, why did you do this?" but was just accepted that it was there and we would now deal with it. It probably helps that it was user-facing and so immediate that everyone was able to weigh in with their opinion, that it wasn't a code cleanup that only developers would recognize.

It was a big win and we were all much happier about the design, but could this go too far? The developer did this on his own time, either staying late or working on it at home. I don't want to seem like a 5:01 developer, since, as Scott prefers, I am excited about technology and I do go home to work on personal projects and learn new things, and these extracurriculars do help me be a better developer for personal and professional reasons. But they do not necessarily immediately impact what I'm doing at work. I feel this is where we should draw a distinction and concern ourselves with doing too much work-related development and not enough personal development. It's a fine line between the two and how much to balance work and life, and there is no simple solution. We should be aware of it and realize how it can become the norm.

Don't be lazy with route resources and allow unused ones

Refactoring controllers to make then skinny is usually a multi-step affair. The common process is to get current functionality tested and them refactor the business logic where it needs to go, moving the tests along with them to their corresponding test files. As the code is moved (to models or service objects, etc.), we can stub out those calls in the controller. Eventually, hopefully, the controller actions will be only a few assignments and a render call. Since the effects can reach across the entire Rails architecture (the M, V, and C, if you will), I find smoke testing often a necessary step in the process.

Which brings me to two sources of frustration: the default Rails (2.0) routes and default restful routes.

map.connect ':controller/:action/:id.:format'

This is just ripe for annoyance. Having a list of all the routes set up and not being able to find the one that I need? This guys fault. Rails has a lot of magic, but this has become a major pain because of the implicit and non-obvious nature of allowing anything one wants to sit next to a specified list. But then again, that specific list can have problems of its own too.

map.resources :photos

This can be fantastic. One line gives us so much power and ease: 7 actions for the price of one. What's the problem? Well, when I was recently working on some tech debt, I thought I found some unused actions. But how did I know for sure? I had to do a lot of grepping and manual testing to make sure these really weren't used. It turned out that one I thought wasn't was actually called in one specific case, so I had to back out my changes.

This would have been less painful if the code coverage was better, but that's sometimes the nature of old code, isn't it? If there are methods in the controller that aren't used, it's one thing to not have them declared, but what happens when there is code for an action that goes unused? I decided to be explicit about which routes to use so I didn't have to deal with the same problems later on (I can be my own worst enemy sometimes).

Let's lock down those resources:

map.resources :foobars, :only => [ :index, :show, :new, :create ]

If we're not using :edit, :update, and :destroy, let's not clutter things up with implications.

Maybe it's overkill, but tech debt can be painful to pay back, and I don't want to piss off my future self if this were to happen again.

Vimprovements

After the improvements I made to my Vim setup, I realized I could get that setup in a repeatable state without too many problems. Here's what I did.

The first decision was to not use Janus. I enjoyed my time with it, but I wanted to have full control over Vim and have a reason for each new piece of functionality. I went through the .vimrc and .gvimrc files and the list of plugins and took only the ones I used for day-to-day development. If I have a pain point and want more functionality, I'll find a solution organically, but I didn't want to have a kitchen sink setup where I didn't know if the functionality is Vim's or Janus'.

Don't feel that this means I dislike Janus, as I think it's a great environment, especially for those new to Vim, but it's time to move on.

I then installed pathogen for easy plugin management. I used submodules to install all the plugins I wanted so I'm able to easily update them through git. The two non-trivial installations were Command-T and javaScriptLint. I included instructions in the README for getting those installed. For the latter plugin, it's a nice thing to add the following line to your .vimrc file to get rid of the rather strong highlighting color:

let g:jslint_highlight_color=""

To get the .vimrc and .gvimrc files in the same folder, I renamed them without the dots and created symlinks with the original names in the root directory that pointed to them. Now everything was under version control and can go on whatever machine I want.

You can find my setup on github. Feel free to take what you like.

State Machine Transactions

The Setup

We have a model that has a state machine on it to activate, close, and cancel after meeting certain conditions. When closing this model, we want to manipulate an associated model as well, and we want these in a transaction to preserve data integrity. Unfortunately, there is logic in the associated model that doesn't like to have the original model in the new state before saving. The obvious solution is to set up the following callbacks:

state :closed, :enter => :enter_close, :after => :after_close

def enter_closed
  # do stuff to model
end

def after_close
  # do stuff to associated model
end

The Problem

It turns out that the callbacks are not wrapped in a transaction and that #after_close is called after the model saves, leaving the associated model in danger of getting in a back state or failing validations.

The Solution

Using this post as a guide, we ended up with this code:

class ActiveRecord::Base
  def self.transaction_around_transitions
    event_table.keys.each do |t|
      define_method("#{t}_with_lock!") do
        transaction do
          send("#{t}_without_lock!")
        end
      end
      alias_method_chain "#{t}!", :lock
    end
  end
end

And call that method after setting up the transitions:

event :close do
  transitions :from => :foo, :to => :bar
end

# other transitions...

transaction_around_transitions

Now there is a transaction around the entire state change and its callbacks, and we'll be able to sleep tonight knowing that all the saving is safe and sound, all wrapped up in a nice, warm transaction.

Can We Have Too Many Tools?

Vim is a great editor. There are so many plugins that make it even better and increase my productivity. But can there be a saturation point where it's not worth finding the next plugin to shave a keystroke off of a command? I've been trying to find the sweet spot for the right amount of tools for the job.

Bigger Toolbox

Having more developers to talk to and work with, I have been exposed to different methods of development. Yan in particular, has been a tremendous Vim resource. He's made it his focus to optimize working with this editor as much as possible, and it's quite impressive. I've adopted some of his ideas, no longer content to be "good enough" with Vim.

Janus has been a tremendous help. It exposed me to command-T, which is now my favorite thing ever. I no longer use a buffer explorer because it's easier to just find it with command-T than search through the buffer list. I also rarely use NERD tree now, since I was using that mostly as a convenience to open project files. I'll still use it for looking in directories, but it's not open by default anymore.

From Yan, I've install git-grep and mapped K to search for the word under the cursor, and that's been such a pleasant, and faster experience, than using vimgrep. Yan is also cleaning up a plugin for rspec integration that provides some nice wins.

Too Big?

But how far do I take this path? I could continue to add plugins and map commonly used keystrokes to further increase efficiency, but when do I start to get diminishing returns? For example, I could map a letter to :GitGrep and save six keystrokes, but I haven't found the need to do that yet. Usually when I'm searching for something, I'm thinking about what I want to search for while typing the command, so I'm effectively multitasking and not wasting time with all that extra typing. Sometimes slowing down can be a good thing to allow that planning. Plus each new mapping or plugin is something new to learn, and it can occasionally become overwhelming with all the new options.

Just Right, For Now

I'll definitely continue to improve my Vim Fu, but I'm not in a hurry to continually add to my repertoire until I've mastered what I currently have available. I've come this far on a basic setup, and Vim has been around long enough that there have been many others who have gotten by with less, so I'm not going to stress out about not optimizing every single key I type.