Automated Testing

It is critical to write automated tests for as much of your back-end code as possible. We use RSpec and Cucumber tests. The RSpec tests are in the spec folder and the Cucumber tests are in the features folder.

Ideally, you will first write tests for your code, and then write your code. This is called Test-Driven Development (TDD).

Table of Contents Code Coverage Unit Tests Continuous Integration Known Test Failures Pending Tests

Code Coverage

Coveralls indicates how much of the codebase is actually being tested.

Unit Tests

Use RSpec for unit testing Ruby code. Unit tests match to code functionality and not to features, and run quickly. They are helpful to ensure that you're not changing behavior accidentally when refactoring or adding new functionality, and that your code is working the way you'd expect it to.

Writing unit tests makes it easier to catch mistakes before checking them in, and reduces the burden on manual testing (human tester volunteers).

Resources on writing good tests with RSpec:

• Everyday Rails Testing with RSpec

Continuous Integration

We write specification in Cucumber. These specify behavior and are easy to understand if you're not a developer.

They run much more slowly than RSpec; don't introduce too many unnecessary steps, and be mindful of unnecessary database calls!

We use Travis CI to run tests automatically on pull requests.

Resources on writing good tests with Cucumber:

• Coming Soon!

Known Test Failures

Some tests currently fail when run locally, but not in CI, due to environment differences. These include:

• LogfileReader with some logfiles reads rows correctly with various patterns (OS specific)
• WorkSearch tests (can't convert SearchResult to Array)

Other tests fail intermittently, particularly when run on Travis:

• Bookmarks
  • cucumber features/bookmarks/bookmark_create.feature:386 # Scenario: Editing a bookmark's tags should update the bookmark blurb
• Collections
  • cucumber features/collections/collection_anonymity.feature:7 # Scenario: Create a hidden collection, add new and existing works to it, reveal works
  • cucumber features/collections/collection_navigation.feature:4 # Scenario: Create a collection and check the links
• Other B
  • cucumber features/other_b/series.feature:7 # Scenario: Three ways to add a work to a series
  • cucumber features/other_b/series.feature:96 # Scenario: Three ways to add a work to a series: a user with more than one pseud
  • cucumber features/other_b/subscriptions_fandoms.feature:57 # Scenario: Author of anonymous work is not shown in feed
• Tags and Wrangling
  • cucumber features/tags_and_wrangling/tag_comment.feature:6 # Scenario: Comment on a tag and get taken to right page and see right date
• Users
  • cucumber features/users/user_dashboard.feature:70 # Scenario: When a user has more works, series, or bookmarks than the maximum displayed on dashboards (5), the "Recent" listbox for that type of item should contain a link to that user's page for that type of item (e.g. Works (6), Bookmarks (10)). The link should go to the user's works, series, or bookmarks page. That link should not exist for any pseuds belonging to that user until the pseud has 5 or more works/series/bookmarks, and then the pseud's link should go to the works/series/bookmarks page for that pseud.

Pending Tests

Mark failing tests pending if you need to merge something urgently, a test is written but isn't working for known reasons, and you don't want to break the build.

In RSpec, do this using xit:

xit "should only get works under that tag" do
  get :index, tag_id: @fandom.name
  expect(assigns(:works).items).to include(@work)
  expect(assigns(:works).items).not_to include(@work2)
end 

In Cucumber, use the When /^"([^\"]*)" is fixed$/ step and comment out any failing steps:

Scenario: Post Without Preview
  Given I am logged in as "whoever" with password "whatever"
    And I add the work "public" to series "be_public"
    And I follow "be_public"
    And "Issue 2169" is fixed
  # Then I should not see the "title" text "Restricted" within "h2"