Network integration tests with MiniTest::Spec and VCR

The problem

If your application relies on a remote network interface and you write an integration test against that code, you’ll notice that the speed of your tests will plumet due to the network delay you now have within your tests. Further whenever your coding in a place without internet connection your tests will fail leading you to ignoring them because you assume that it is a network issue. And thats a path you never want to go down with your tests.

VCR to the rescue

Jup the archaic technology for video recording will help you having a pleasent smile while writing integration tests for code that uses remote network interfaces. Further it will prevent you from having failing tests due to a flaky network connection.

VCR is a Gem by Myron Marston, which just recently hit the 2.0 version, and can be used with either the FakeWeb or Webmock gem.

Lets take a look at this integration test I’ve written in MiniTest spec.

it "returns the name of an episode for a season" do
  series_info = SeriesNamer::SeriesInfo.new("Burn Notice/Season 1", "Burn Notice", "Season 1")
  #series_name = "Burn Notice"
  season = 1
  episode = 3

  series_info = SeriesNamer::GetEpisodeNames.new(series_info)
  series_info.episode_name(season, episode).must_equal "S01E03 Fight or Flight"
end

So this test calls a class that gets the episode name of a series episode. It uses the TvDB to get the information.

Now how do we speed this test up? Well first lets go to our Gemfile and add the following gems:

group :test do
  gem 'vcr', '~>2.0.0'
  gem 'webmock', '~>1.8.3'
end

Then bundle install and we’re ready to go adding VCR (cassettes) to our project

Foreach testfile you want to use VCR in, you’ll have to add the following lines. You may want to put this into your spec_helper.rb [1] file to save you some work and help keeping your tests DRY.

require 'vcr'

VCR.configure do |c|
  c.cassette_library_dir = 'spec/cassettes'
  c.hook_into :webmock
end

Now we’re nearly there to get our speed boost with VCR. Only thing left is adding the VCR call VCR.use_cassette to our test:

it "returns the name of an episode for a season" do
  series_info = SeriesNamer::SeriesInfo.new("Burn Notice/Season 1", "Burn Notice", "Season 1")
  #series_name = "Burn Notice"
  season = 1
  episode = 3

  VCR.use_cassette('burn_notice_episode_1_3') do
    series_info = SeriesNamer::GetEpisodeNames.new(series_info)
    series_info.episode_name(season, episode).must_equal "S01E03 Fight or Flight"
  end
end

When executing this test the first time you will not see any changes but the next time and from there on you will no longer see your tests dragging along, actually you don’t even have to be connected to the net anymore to run your tests. VCR stored all the communication locally on your machine. How, when and where? Well lets have a look.

VCR records all the traffic between the do…end block and stores this information in a yaml file. The string passed into the use_cassette method is used by VCR as the name for the cassette i.e. the yaml file.

We defined where to store the cassettes in the spec_helper.rb file. With the line c.cassette_library_dir = 'spec/cassettes'. If we open that directory PATH_TO_PROJECT/spec/cassettes we see that there is a file burn_notice_episode_1_3.yml which contains all the communication that took place between your app and the network API.

So if you want to refresh your recording, just delete the directory and the next time you run your tests all the communication will be freshly recorded again.

Happy API integration testing!

Note: As long as the request doesn’t change you can reuse a cassette for multiple tests. As soon as the request/answer differs make sure to use a fresh cassette i.e. pass in a different string for the cassette name.

References

Railscasts #291
Rubyinside

—

[1] the spec_helper.rb file is a helper file you can add manually to your spec directory, if you haven’t done so already. All common groundwork code you use in different places within your tests you can put here or if the need arises split it up into finer grained helper classes. In case your using MiniTest/UnitTest without specs, you would name this file test_helper.rb and store it in the test directory.

Run all your tests with rake

When writing automated tests one of the things you will always want to do is execute them. When using RSpec the only thing you have to do is type rspec into your console and of they go. All your tests i.e. specs get executed. Now you might have already heard about rake a.k.a. Ruby make. Within rake there is a test task that can be used to execute tests in the RSpec style described before with only a few lines of Ruby.

So lets started, open your Rakefile[1] and add the following lines of code, I’m using MiniTest::Spec for some of my projects but I’ll explain in a bit how you can modify the task to suit your needs.

  
    require 'rake/testtask'

    Rake::TestTask.new do |t|
      t.libs = ["lib"]
      t.warning = true
      t.verbose = true
      t.test_files = FileList['spec/*_spec.rb']
    end
  

This lets us run all spec files that are in the spec directory and the filenames end with _spec.rb by typing rake test into the console.

If your using another test library e.g. TestUnit you can change the seventh line to t.test_files = FileList['test/*_test.rb'], which will run all your tests analog to the spec command but in another directory i.e. with a different filename ending.

Now what are all the other configurations for? Well lets go through them one by one.

• libs: In the example case loads the lib directory into as default environment, this allows you to use require 'my_class' in your unit tests instead of having to use relative paths e.g. require_relative '../lib/myclass'.
• waring: This runs your tests with the warning flag enabled i.e. ruby -w, this shows you possible problems and we don’t want to give other people potential problems do we?
• verbose: Verbose output e.g. files used in test

It’s up to you which flags you want to enable and disable the one argument you should provide is test_files.

Separating slow tests

Now not every test you write may be a unit test and those tests that aren’t have the tendency to run slower then a unit test, which leads to favoring separation of unit tests and for example integration tests.

I like to add an additional directory for my integration tests within the spec directory. Which then looks something like this:

  
    |-spec
       |---integration
  

Our original rake task will not work with this structure i.e. it will just ignore the specs we store under spec/integration. Further we would actually like to run our unit tests and integration tests separately. So lets add a new task for our integration tests.

  
    #previous code omitted

    Rake::TestTask.new do |t|
      t.libs = ["lib", "spec"]
      t.name = "test:integration"
      t.warning = true
      t.test_files = FileList['spec/integration/*_spec.rb']
    end
  

If we now type rake test:integration into the console all our integration tests will be run. Note that by adding the spec folder you can require 'spec_helper.rb' in the integration tests.

Naming a test task allows you to choose between them, but there is a more convenient way of naming a test task. Let me show you by adding a further task that will execute all our specs/tests in the spec directory including sub folders.

  
    #previous code omitted

    #integration test code omitted

    Rake::TestTask.new("test:all") do |t|
      t.libs = ["lib", "spec"]
      t.warning = true
      t.test_files = FileList['spec/**/*_spec.rb']
    end
  

With rake test:all we can now run all our tests in one go. And if we decide to add another category of tests they will also be run with this approach.

References

Rake:TestTask
Ruby Inside MiniTest::Spec

Further reading rake

A good point to get some more rake skills is the Rake Tutorial by Jason Seifer or the Official Rake Tutorial.

—-

[1] If not present just create a file named Rakefile and continue on reading.

Getting started with RSpec in Rails 3.2

Installing RSpec with Rails is pretty straight forward. The only task left since Bundler is to add RSpec to the Gemfile. And there is more good news, there is a special RSpec gem for Rails which has some handy shortcuts ready for you, I’ll be showing you some of those in future blog entries.

Quickest way is to append the gem to the Gemfile using the Console or Terminal just execute the following command in your project root directory(1):

echo "gem "rspec-rails", :group => [:test, :development]" >> Gemfile

Once you appended the RSpec gem install the gem via bundler i.e. bundle install.

We now got a new Rails generator commands:

Rspec:
  rspec:install

So lets install RSpec for our Rails project by running the following command rails g rspec:install

This will setup the RSpec environment in your Rails directory. Now we are ready to start cranking out RSpec tested code.

You may have noticed that we did not only add the gem to the test group but additionally to the development. You may ask yourself why? Well by adding it to the development group we get some more RSpec goodness. If you now use e.g.:

rails g model Something some_name some_description
      invoke  active_record

      create    db/migrate/20120207071500_create_somethings.rb
      create    app/models/something.rb
      invoke    rspec
      create      spec/models/something_spec.rb

Your Rails generator will automatically generate you the RSpec spec file. Let the BDD begin..

Sources:

RSpec book
Railscasts
Tekpub Rails 3 intro

—
(1)Or just simply open the Gemfile in the editor of choice and add the line: gem "rspec-rails", :group => [:test, :development]

Ruby Unit Tests breaking dependencies

One definition of a unit test is that it has to run fast or in the words of Michael Feathers in his book “Working Effectively with Legacy Code”:

A unit test that takes 1/10th of a second to run is a slow unit test.

If all operations happen in memory they tend to work rather quickly.(1)
This changes when we start to use files or network connections in classes that we put under unit tests. IO is costly as every roundtrip to the hard drive uses compared to RAM access forever LINK COMPARISON HDD TO RAM. So lets say we want to have a Ruby class that lists all entries of a directory and puts it into a nicely formatted string ready to print out to the console:

class DirLister
  def list_dir( path )
    raise ArgumentError, "Path #{path} does not exist." unless Dir.exists?( path )

    Dir.entries( path )
  end
end

To avoid the hit in execution time of our tests consider using default parameters for your IO classes in the constructor. So we would have to change our little dummy class to the following:

class DirLister
  @directory #defaults to Rubys Dir, see constructor

  def initialize( directory = Dir )
    @directory = directory
  end

  def list_dir( path )
    raise ArgumentError, "Path #{path} does not exist." unless @directory.exists?( path )

    @directory.entries( path )
  end
end

With these changes our test can now pass in a stub or if needed even a mock object. So how would we do this? Well remember Ruby is dynamic so if it quacks, walks and looks like a duck your all good when it comes to stubbing a object in Ruby. Here is an example of how the original MiniTest Spec could look like:

describe DirLister do
  it "lists all entries of directory" do
    test_path = "/tmp/test_path"
    expected_result = ["one", "two", "three"]

    DirLister.new().list_dir(test_path).must_equal expected_result
  end
end

Now if we wanted to pass in a fake dir object we could use the built in Mock from MiniTest which would result in the following code:

describe DirLister do
  it "lists all entries of directory" do
    test_path = "/tmp/test_path"
    expected_result = ["one", "two", "three"]

    dir_mock = MiniTest::Mock.new

    #MockMethod   MethodName, ReturnValue, Parameter(s)
    dir_mock.expect(:exists?, true, [test_path])
    dir_mock.expect(:entries, expected_result, [test_path])

    DirLister.new(dir_mock).list_dir(test_path).must_equal expected_result
  end
end

This approach of passing in system classes to your class can also be used to replace the Time class to simulate different times etc. as the current time given on the system.

One may argue that using a Mock class in this case is overkill. Another alternative would be to pass in a fake class object that would behave the same way we set up the MiniTest mock class.

(1) There are of course algorithms or big datablocks that tend to take more time. Consider in those cases to only tests aspects of the algorithm and move the more complex and time consuming test to your integration tests.