Automated Testing

It is critical to write automated tests for as much of your back-end code as possible. We use RSpec and Cucumber tests. The RSpec tests are in the spec folder and the Cucumber tests are in the features folder.

Ideally, you will first write tests for your code, and then write your code. This is called Test-Driven Development (TDD).

Table of Contents Unit Tests Integration Tests Running Tests Known Test Failures Pending Tests

Unit Tests

Use RSpec for unit testing Ruby code. Unit tests match to code functionality and not to features, and run quickly. They are helpful to ensure that you're not changing behavior accidentally when refactoring or adding new functionality, and that your code is working the way you'd expect it to.

Writing unit tests makes it easier to catch mistakes before checking them in, and reduces the burden on manual testing (human tester volunteers).

Resources on writing good tests with RSpec:

• Everyday Rails Testing with RSpec

Integration Tests

We write specification in Cucumber. These specify behavior and are easy to understand if you're not a developer.

They run much more slowly than RSpec; don't introduce too many unnecessary steps, and be mindful of unnecessary database calls!

Resources on writing good tests with Cucumber:

• Coming Soon!

Running Tests

Before you run the automated tests, you may need to prepare your test database using bundle exec rake db:test:prepare. This is usually only necessary the first time you run tests or after running a migration.

• Run all RSpec tests: bundle exec rspec spec
• Run a specific RSpec test: bundle exec rspec spec/my_model_spec.rb
• Run all Cucumber tests: bundle exec cucumber features
• Run a specific Cucumber test: bundle exec cucumber features/my_archive_process.feature
• Run a specific scenario within a Cucumber test: bundle exec cucumber features/my_archive_process.feature:10, where :10 is the line number of the scenario

We use Travis CI to run tests automatically on pull requests before and after they're merged. Code that is merged is also tested automatically on Codeship before being automatically deployed to our staging environment.

Known Test Failures

Some tests currently fail when run locally, but not in CI, due to environment differences. These include:

• LogfileReader with some logfiles reads rows correctly with various patterns (OS specific)
• WorkSearch tests (can't convert SearchResult to Array)

Other tests fail intermittently, particularly when run on Travis:

• Bookmarks
  • cucumber features/bookmarks/bookmark_create.feature:386 # Scenario: Editing a bookmark's tags should update the bookmark blurb
• Collections
  • cucumber features/collections/collection_anonymity.feature:7 # Scenario: Create a hidden collection, add new and existing works to it, reveal works
  • cucumber features/collections/collection_anonymity.feature:161 # Scenario: Create an anonymous collection, add new and existing works to it, reveal authors
  • cucumber features/collections/collection_navigation.feature:4 # Scenario: Create a collection and check the links
• Other B
  • cucumber features/other_b/series.feature:7 # Scenario: Three ways to add a work to a series
  • cucumber features/other_b/series.feature:96 # Scenario: Three ways to add a work to a series: a user with more than one pseud
  • cucumber features/other_b/subscriptions_fandoms.feature:57 # Scenario: Author of anonymous work is not shown in feed
• Tags and Wrangling
  • cucumber features/tags_and_wrangling/tag_comment.feature:6 # Scenario: Comment on a tag and get taken to right page and see right date
• Users
  • cucumber features/users/user_dashboard.feature:70 # Scenario: When a user has more works, series, or bookmarks than the maximum displayed on dashboards (5), the "Recent" listbox for that type of item should contain a link to that user's page for that type of item (e.g. Works (6), Bookmarks (10)). The link should go to the user's works, series, or bookmarks page. That link should not exist for any pseuds belonging to that user until the pseud has 5 or more works/series/bookmarks, and then the pseud's link should go to the works/series/bookmarks page for that pseud.

Pending Tests

Mark failing tests pending if you need to merge something urgently, a test is written but isn't working for known reasons, and you don't want to break the build.

In RSpec, do this using xit:

xit "should only get works under that tag" do
  get :index, tag_id: @fandom.name
  expect(assigns(:works).items).to include(@work)
  expect(assigns(:works).items).not_to include(@work2)
end 

In Cucumber, use the When /^"([^\"]*)" is fixed$/ step and comment out any failing steps:

Scenario: Post Without Preview
  Given I am logged in as "whoever" with password "whatever"
    And I add the work "public" to series "be_public"
    And I follow "be_public"
    And "Issue 2169" is fixed
  # Then I should not see the "title" text "Restricted" within "h2"