Network integration tests with MiniTest::Spec and VCR

The problem

If your application relies on a remote network interface and you write an integration test against that code, you’ll notice that the speed of your tests will plumet due to the network delay you now have within your tests. Further whenever your coding in a place without internet connection your tests will fail leading you to ignoring them because you assume that it is a network issue. And thats a path you never want to go down with your tests.

VCR to the rescue

Jup the archaic technology for video recording will help you having a pleasent smile while writing integration tests for code that uses remote network interfaces. Further it will prevent you from having failing tests due to a flaky network connection.

VCR is a Gem by Myron Marston, which just recently hit the 2.0 version, and can be used with either the FakeWeb or Webmock gem.

Lets take a look at this integration test I’ve written in MiniTest spec.

it "returns the name of an episode for a season" do
  series_info ="Burn Notice/Season 1", "Burn Notice", "Season 1")
  #series_name = "Burn Notice"
  season = 1
  episode = 3

  series_info =
  series_info.episode_name(season, episode).must_equal "S01E03 Fight or Flight"

So this test calls a class that gets the episode name of a series episode. It uses the TvDB to get the information.

Now how do we speed this test up? Well first lets go to our Gemfile and add the following gems:

group :test do
  gem 'vcr', '~>2.0.0'
  gem 'webmock', '~>1.8.3'

Then bundle install and we’re ready to go adding VCR (cassettes) to our project

Foreach testfile you want to use VCR in, you’ll have to add the following lines. You may want to put this into your spec_helper.rb [1] file to save you some work and help keeping your tests DRY.

require 'vcr'

VCR.configure do |c|
  c.cassette_library_dir = 'spec/cassettes'
  c.hook_into :webmock

Now we’re nearly there to get our speed boost with VCR. Only thing left is adding the VCR call VCR.use_cassette to our test:

it "returns the name of an episode for a season" do
  series_info ="Burn Notice/Season 1", "Burn Notice", "Season 1")
  #series_name = "Burn Notice"
  season = 1
  episode = 3

  VCR.use_cassette('burn_notice_episode_1_3') do
    series_info =
    series_info.episode_name(season, episode).must_equal "S01E03 Fight or Flight"

When executing this test the first time you will not see any changes but the next time and from there on you will no longer see your tests dragging along, actually you don’t even have to be connected to the net anymore to run your tests. VCR stored all the communication locally on your machine. How, when and where? Well lets have a look.

VCR records all the traffic between the do…end block and stores this information in a yaml file. The string passed into the use_cassette method is used by VCR as the name for the cassette i.e. the yaml file.

We defined where to store the cassettes in the spec_helper.rb file. With the line c.cassette_library_dir = 'spec/cassettes'. If we open that directory PATH_TO_PROJECT/spec/cassettes we see that there is a file burn_notice_episode_1_3.yml which contains all the communication that took place between your app and the network API.

So if you want to refresh your recording, just delete the directory and the next time you run your tests all the communication will be freshly recorded again.

Happy API integration testing!

Note: As long as the request doesn’t change you can reuse a cassette for multiple tests. As soon as the request/answer differs make sure to use a fresh cassette i.e. pass in a different string for the cassette name.


Railscasts #291

[1] the spec_helper.rb file is a helper file you can add manually to your spec directory, if you haven’t done so already. All common groundwork code you use in different places within your tests you can put here or if the need arises split it up into finer grained helper classes. In case your using MiniTest/UnitTest without specs, you would name this file test_helper.rb and store it in the test directory.

Run all your tests with rake

When writing automated tests one of the things you will always want to do is execute them. When using RSpec the only thing you have to do is type rspec into your console and of they go. All your tests i.e. specs get executed. Now you might have already heard about rake a.k.a. Ruby make. Within rake there is a test task that can be used to execute tests in the RSpec style described before with only a few lines of Ruby.

So lets started, open your Rakefile[1] and add the following lines of code, I’m using MiniTest::Spec for some of my projects but I’ll explain in a bit how you can modify the task to suit your needs.

    require 'rake/testtask' do |t|
      t.libs = ["lib"]
      t.warning = true
      t.verbose = true
      t.test_files = FileList['spec/*_spec.rb']

This lets us run all spec files that are in the spec directory and the filenames end with _spec.rb by typing rake test into the console.

If your using another test library e.g. TestUnit you can change the seventh line to t.test_files = FileList['test/*_test.rb'], which will run all your tests analog to the spec command but in another directory i.e. with a different filename ending.

Now what are all the other configurations for? Well lets go through them one by one.

  • libs: In the example case loads the lib directory into as default environment, this allows you to use require 'my_class' in your unit tests instead of having to use relative paths e.g. require_relative '../lib/myclass'.
  • waring: This runs your tests with the warning flag enabled i.e. ruby -w, this shows you possible problems and we don’t want to give other people potential problems do we?
  • verbose: Verbose output e.g. files used in test

It’s up to you which flags you want to enable and disable the one argument you should provide is test_files.

Separating slow tests

Now not every test you write may be a unit test and those tests that aren’t have the tendency to run slower then a unit test, which leads to favoring separation of unit tests and for example integration tests.

I like to add an additional directory for my integration tests within the spec directory. Which then looks something like this:


Our original rake task will not work with this structure i.e. it will just ignore the specs we store under spec/integration. Further we would actually like to run our unit tests and integration tests separately. So lets add a new task for our integration tests.

    #previous code omitted do |t|
      t.libs = ["lib", "spec"] = "test:integration"
      t.warning = true
      t.test_files = FileList['spec/integration/*_spec.rb']

If we now type rake test:integration into the console all our integration tests will be run. Note that by adding the spec folder you can require 'spec_helper.rb' in the integration tests.

Naming a test task allows you to choose between them, but there is a more convenient way of naming a test task. Let me show you by adding a further task that will execute all our specs/tests in the spec directory including sub folders.

    #previous code omitted

    #integration test code omitted"test:all") do |t|
      t.libs = ["lib", "spec"]
      t.warning = true
      t.test_files = FileList['spec/**/*_spec.rb']

With rake test:all we can now run all our tests in one go. And if we decide to add another category of tests they will also be run with this approach.


Ruby Inside MiniTest::Spec

Further reading rake

A good point to get some more rake skills is the Rake Tutorial by Jason Seifer or the Official Rake Tutorial.


[1] If not present just create a file named Rakefile and continue on reading.