After doing a fresh installation with Mountain Lion, I've had a hell of a time properly install MySQL. I've had to use a few sources to figure this one out, and here are the steps I needed to take.
I'm not 100% sure that these steps are needed, but they couldn't hurt
Now test with
mysql -uroot
A recent article from Los Techies resonated with something I also did recently at work, and it's nice to see others come up with similar ideas independently. Hopefully that means they're good ideas.
"Eventual consistency in interaction design" presents the idea of giving the user immediate feedback for an action that is not immediate. It combines a synchronous call (a message to a user after he interacts with a page) with an asynchronous one (processing the results of that action in a background task). The benefits are obvious: the user gets immediate feedback that something happened, and he knows that, while it may not have happened right away, he's not stuck in spinner hell, waiting for some sort of response.
When a user does certain actions on our site, we create an activity to put it in a feed to either display or to send it off to a third party. We create the activity immediately and put it in a job queue to do whatever functionality it requires later. The job queue is fast, but it can take a minute or so before it processes the new activity.
That's a minute the user shouldn't need to wait for.
So when we schedule the job, we also save the related data, the data we're sending off to a third party site, so the user sees the results immediately. To him, his action happened in real time and he never notices the difference.
One obvious question is what if the user tries to do something on our site with the data that is supposedly linked to another site because he assumes they've been synchronized? Since everything that goes to the other site is done asynchronously, his other actions just get added to the job queue after the first action. If he goes to the other site, the job is generally fast enough that it will have processed the original activity (again, since most jobs run in under a minute). If it becomes a problem where users are getting to the other site too quickly, we'll be able to come up with language more akin to "we're processing your request." But this current implementation provides a nice solution for now.
Like the article mentions, "If you have users that have to wait to have the view model updated to see their results, you have built the wrong user interface." In our case, the user does not have to wait for an update, even if that update hasn't happened yet.
We found a bug at work where a css style changed, but it wasn't immediately obvious where it had been introduced. To fix this bug, I went back through the git history and checked out one in the recent past to see if the bug had been introduced there. This only took a few tries, and then I worked back toward the present to find when the styling had been modified.
I found the offending commit by checking out the sass file of that particular commit:
git checkout {commit_number} path/to/stylesheet.sass
Now I've got the most recent code plus the file that fixes the problem. To see what had been changed, I did a diff on the two files: the older one without the bug and the next one with the bug:
git diff {older_commit} {newer_commit} path/to/stylesheet.sass
There are a lot of changes, so I looked for anything that would be obvious (margins in this case). After that, it was modifying the styling to fix the bug and done.
Git plays nice.
In our continued path to faster tests, we've tried doing controller tests without loading Rails and using only what's minimally required to fully test the functionality. the idea is that not only will the tests be faster, but they will drive the controller's design to be thinner.
How did it work out? First the background.
We implemented a controller that received an OAuth callback from another site to allow their users to directly login to Crowdcast. The required code was minimal, so we tried to test minimally as well.
Since we're not loading Rails, we need to manually load some required libraries for the controller. We need enough to load the controller and its dependencies (in this case, a url generator class and the controller itself) successfully.
require 'uri' require 'active_support' require 'fast_spec/spec_helper' require 'lib/crowdcast/url_generator' require 'app/controllers/oauth_controller'
We also need to stub out calls to models, both ActiveRecord and plain old Ruby objects the controller uses. Here is where we realize how many of these we need and use the pain of stubbing yet another class to drive the design to remove that dependency.
stub_class 'UserSession' stub_active_record_model 'Site' stub_active_record_model 'SalesforceConnection'
Some are necessary, such as those for Authlogic and the relevant AR classes we're calling, but we were able to move calls to others out of the controller to make the tests easier to write and the code easier to read and maintain. All the usual good stuff we get from TDD and OOP.
Here's an example:
describe OauthController do
let(:controller) { OauthController.new }
describe "#callback" do
let(:user_attributes) {{ :email => "email" }}
let(:user) { stub("user", :single_access_token => "foo") }
let(:site) { stub("site") }
let(:redirect_to_url) { "https://foo.example.com:8080?foo=bar&baz=quux" }
let(:state) { CGI.escape({:url => redirect_to_url, :subdomain => "foo" }.to_json) }
before do
controller.stub(:params) {{ :code => "foo", :state => state }}
controller.stub(:current_user) { user }
controller.stub(:session) {{ :return_to => "/return_path" }}
Crowdcast::Config.stub(:salesforce) {{ :id => "bar" }}
Salesforce::UserResolver.stub(:user_attributes_by_auth_code).with({ :id => "bar" }, "foo").and_return(user_attributes)
SalesforceConnection.stub(:connect).and_return(user)
Site.stub(:find_by_subdomain).with("foo").and_return(site)
end
it "creates a SalesforceConnection connection" do
SalesforceConnection.should_receive(:connect).once.with(user, user_attributes)
controller.callback
end
it "redirects to the return_to path with user's single access token" do
controller.should_receive(:redirect_to).with(redirect_to_url + "&token=foo")
controller.callback
end
end
You can see that we need a fair amount of initial setup to get both the regular methods all controllers access and the specific ones we want to test or stub in the specific tests. It's still feels reasonable considering that we lose the Rails loading overhead and tests are incredibly fast.
Here is the controller:
def callback
if params[:code]
cc_user = connect_user_to_salesforce
redirect_to Crowdcast::UrlGenerator.url(:url => return_to_url_from_callback, :params => { :token => cc_user.single_access_token })
else
flash[:error] = params[:error_description] if params[:error_description]
redirect_to return_to_url_from_callback
end
end
private
def connect_user_to_salesforce
SalesforceConnection.connect(existing_or_autoregistered_user)
end
def existing_or_autoregistered_user
current_user || Salesforce::UserAutoRegistrar.register(current_site)
end
def return_to_url_from_callback
state_from_callback["url"]
end
def state_from_callback
JSON.parse(CGI.unescape(params[:state]))
end
And now a problem.
We want to add some more Rails controller goodness in case there are exceptions (they're always where you least expect them). Check this out.
rescue_from Salesforce::UserResolver::UnsupportedAccountTypeError, :with => :account_error rescue_from Salesforce::TokenResolver::AuthCodeExpiredError, :with => :expired_auth_code def account_error render :status => :precondition_failed, :action => "account_error", :layout => "not_found" end
Now we need to figure out how to get rescue_from or have more stubbing on the controller class. Before, when our controller was very lightweight, we could deal with the minimal amount of manual dependencies to get the speed increases. But at this point we decided to convert the controller to our "slow specs" folder, ie, our regular /spec folder, since the pain of managing the external dependencies reached a threshold we weren't willing to cross.
How did we decide this was the time? It wasn't anything specific but the overall feel of the code getting too complicated and stub-happy; we weren't getting pain from bad design but from using a framework.
Conclusions
Testing without Rails is still new, and we're still learning what works and what doesn't. Should we automatically test functionality that's coupled to the framework within the framework? I still say no, that we can get out of the framework if we use only a minimal subset that we can maintain manually. We decided to return to loading Rails when that subset was no longer minimal. That situation did not come up for some time, and it isn't a foregone conclusion that it always will. It's a developer's decision on the tradeoffs. Plus it was a great learning experience.
I recently finished reading The Developer's Code and found some ideas that really resonate. There's a lot in the book that feels very obvious, but then I realize how long it took me to learn those lessons and how many mistakes I've made to learn them. Those thoughts make me realize that the other parts of the book, the ones that don't click as much, are probably even more valuable since I now have an opportunity to learn things without the pain of experiencing then through failure (although I'm sure I'll still get to enjoy those feelings quite often).
Essay 11 is titled Stop Programming. Its argument is that it's not about the hours one puts into the work but the quality. "Great programming is about maximizing the time you’re working at your best, not the cumulative hours you spend in front of a screen." This is readily obvious from thinking about charging clients an hourly rate vs. a fixed rate. Does it really matter if I sit at my computer for eight hours if I can provide just as much value from two hours of work? Won't work find a way to take the entire time allotted unless we're ever-vigilant? The difficulty for people to realize this is that programming is abstract and value isn't guaranteed through lines of code written.
The next essay, Test Your Work First Thing in the Morning, posits that we're mentally and physically fresher earlier in the day, so it makes the most sense to do the hard things then. By the late afternoon, "our perception of what makes sense or feels right now competes with fatigue. Also, fatigue makes us miss the small details." If we haven't stopped programming to take a break or otherwise recharge, our cognitive abilities will deteriorate more than normal. Thinking as an activity is difficult.
Pair programming is wonderful, but it's intense as well. We've come to understand that we shouldn't work on the more difficult problems at the end of the day, so we always try to leave small bugs or simple features for that time. It's also motivating to start the day solving more interesting problems rather than putting then off until the motivation has waned. At that time it's better to just get something done that is low impact.
Corey Haines first brought testing parts of Rails applications without loading Rails to my attention and excitement a few months ago, and I've been looking at how other developers do similar techniques. Having recently finished Objects on Rails and working through the Destroy All Software videos, I've been constantly looking at ways to make my Rails code better and faster to test. I believe these are related and that there is a lot of value in working to make the code easy to test and fast.
An example of some code I've written that doesn't rely on Rails is an observer. The observer inherits from ActiveRecord::Observer and would normally need Rails loaded to exercise its full functionality with the model it's observing. But if we try to keep Rails out and only load what we need, we can drive the design of the observer.
Here we save an activity, which is used as a generic Facebook-style activity stream item, after someone answers a question.
class AnswerObserver < ActiveRecord::Observer
def after_create(answer)
Activity.question_answered(answer)
end
end
Here's the entire spec file.
require 'fast_spec/spec_helper'
stub_class 'ActiveRecord::Observer'
require 'app/models/observers/answer_observer'
class Activity; end
describe AnswerObserver do
let(:observer) { AnswerObserver.new }
describe "#after_create" do
let(:answer) { stub }
it "creates an activity" do
Activity.should_receive(:question_answered).with(answer)
observer.after_create(answer)
end
end
end
We do a few things here. The first is loading fast_spec/spec_helper.rb instead of the regular spec_helper.rb that loads the Rails environment. The fast spec helper requires a few utility files like awesome_print and logger, and a module_stub file that has a convenience method stolen from Avdi Grimm:
def stub_module(full_name)
full_name.to_s.split(/::/).inject(Object) do |context, name|
begin
context.const_get(name)
rescue NameError
context.const_set(name, Module.new)
end
end
end
To quote Avdi:
This method uses #const_get to attempt to reference the given module. If the module is defined, or if calling #const_get causes it to be auto-loaded, the method does nothing more. But if #const_get fails to turn up the module, it defines an anonymous empty module to act as a placeholder.
Looking back at the spec file, we see a similar call to stub_class which does the same thing for classes. We also include the actual file of the code we're testing.
We create an empty Activity class, normally an ActiveRecord model, since we don't care what it's doing, just that it receives messages we're going to send to it. That's all this test does, really: we check that the Activity class gets a specific message passed to it with specific parameters.
Testing this way drives the code to be loosely coupled with objects this class needs to communicate with. We could have put all the activity action in the after_create callback and we'd be able to see what's going on, but we'd need to mock and stub a lot of interactions, or we'd have to go back to loading Rails, slowing our tests down. Plus we'd have a lot of code that has feature envy and would be difficult to maintain.
We know the test works because changing the functionality of the callback will break the test, and doing BDD will ensure that we're writing failing tests before we add functionality that could be skipped by our mocks. Of course, this problem exists with slower, more integration-like tests, so it's not like running through the full stack is a silver bullet.
There is a point where there can be too much mocking and stubbing, and I'll get to that in another post. For now, being able to run an entire suite of tests in less than half a second helps keep me in flow to write better software.
It seems that our backlog of features, bugs, and technical debt grew more than it shrinked. We would fix one bug and discover two more, or we'd clean up some poorly designed feature and find out that there are others that depend on that one. We try to do our best to design new features as best we can, but that can't always happen on a legacy code base.
What could we do? What did we do?
Those two questions were very different. We could schedule time to fix bugs and clean up technical debt in between adding new features, and it's great when management is happy with this process. You may think why we as developers expose that to the management instead of just putting it into the regular development process - that we effectively black box it from the "outside" non-technical people. In a start-up it's not always possible since everyone needed to know what's going on, plus it's hard to hide anything when everyone was in one room. But the positive side was that management understands the need for these chores and didn't push back.
But back to those two questions and their differences.
We could do these things, and they got done, but the bugs and technical debt were still with us in a major way, and they slowed down our entire process. Was there a solution?
What if we just fixed them? No discussion up front, but do it and "apologize" later.
We've built up a lot of UI cruft over the years - poorly structured CSS and HTML, bad usability, inconsistent UI. It was always a pain point, but there were so many other things we had to do as well, so it stayed around and we just dealt with it.
Until one of our developers took it on himself to start cleaning it up. He didn't ask for permission and he didn't talk about it, but we came in one day to a demo and code review of a much cleaner UI. The conversation immediately was about tweaking the new changes and what else we could do to make the UI better. It wasn't "hey, why did you do this?" but was just accepted that it was there and we would now deal with it. It probably helps that it was user-facing and so immediate that everyone was able to weigh in with their opinion, that it wasn't a code cleanup that only developers would recognize.
It was a big win and we were all much happier about the design, but could this go too far? The developer did this on his own time, either staying late or working on it at home. I don't want to seem like a 5:01 developer, since, as Scott prefers, I am excited about technology and I do go home to work on personal projects and learn new things, and these extracurriculars do help me be a better developer for personal and professional reasons. But they do not necessarily immediately impact what I'm doing at work. I feel this is where we should draw a distinction and concern ourselves with doing too much work-related development and not enough personal development. It's a fine line between the two and how much to balance work and life, and there is no simple solution. We should be aware of it and realize how it can become the norm.
map.connect ':controller/:action/:id.:format'This is just ripe for annoyance. Having a list of all the routes set up and not being able to find the one that I need? This guys fault. Rails has a lot of magic, but this has become a major pain because of the implicit and non-obvious nature of allowing anything one wants to sit next to a specified list. But then again, that specific list can have problems of its own too.
map.resources :photosThis can be fantastic. One line gives us so much power and ease: 7 actions for the price of one. What's the problem? Well, when I was recently working on some tech debt, I thought I found some unused actions. But how did I know for sure? I had to do a lot of grepping and manual testing to make sure these really weren't used. It turned out that one I thought wasn't was actually called in one specific case, so I had to back out my changes.
map.resources :foobars, :only => [ :index, :show, :new, :create ]If we're not using :edit, :update, and :destroy, let's not clutter things up with implications.
let g:jslint_highlight_color=""To get the .vimrc and .gvimrc files in the same folder, I renamed them without the dots and created symlinks with the original names in the root directory that pointed to them. Now everything was under version control and can go on whatever machine I want.
state :closed, :enter => :enter_close, :after => :after_close def enter_closed # do stuff to model end def after_close # do stuff to associated model end
class ActiveRecord::Base
def self.transaction_around_transitions
event_table.keys.each do |t|
define_method("#{t}_with_lock!") do
transaction do
send("#{t}_without_lock!")
end
end
alias_method_chain "#{t}!", :lock
end
end
end
And call that method after setting up the transitions:event :close do transitions :from => :foo, :to => :bar end # other transitions... transaction_around_transitionsNow there is a transaction around the entire state change and its callbacks, and we'll be able to sleep tonight knowing that all the saving is safe and sound, all wrapped up in a nice, warm transaction.