Skip to content

Improve reporting for passing "todo" tests #1813

@Krinkle

Description

@Krinkle

Status quo

When a todo test is failing, we currently report it as follows in the CLI and in TAP:

/* global QUnit */
QUnit.test('apple', function (assert) {
  assert.true(true, 'stem');
  assert.true(true, 'skin');
});
QUnit.test.todo('banana', function (assert) {
  assert.true(true, 'foo');
  assert.equal('This is actual.', 'This is expected.', 'example sentence');
  assert.true(true, 'bar');
  assert.true(true, 'baz');
});
QUnit.test('sauerkraut', function (assert) {
  assert.true(true, 'acid rock');
});
QUnit.test('dampfnudel', function (assert) {
  assert.true(true, 'stream');
  assert.true(true, 'bun');
});
TAP version 13
ok 1 apple
not ok 2 banana # TODO
  ---
  message: example sentence
  severity: todo
  actual  : This is actual.
  expected: This is expected.
  stack: at test/fixtures/qunit-todo-skip.js:8:10
  ...
ok 3 sauerkraut
ok 4 dampfnudel
1..4
# pass 3
# skip 0
# todo 1
# fail 0

... and in the HTML Reporter:

Default (collapsed, because not failing) Expected
Image Image

Problem

Then, when it starts passing, it is reported as folllows.

  • There is nothing here to indicate that it is/was a TODO test.
  • There is nothing here to explain why it is marked as failure.
  • There is nothing here to suggest removing the "TODO" marker to make it pass.
TAP version 13
ok 1 apple
not ok 2 banana
ok 3 sauerkraut
ok 4 dampfnudel
1..4
# pass 3
# skip 0
# todo 0
# fail 1
Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    Component: CoreFor module, test, hooks, and reporters.Type: EnhancementNew idea or feature request.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions