Show which rows failed in `expect_` functions

[lbm364dl]

Prework

• Read and agree to the code of conduct and contributing guidelines.
• If there is already a relevant issue, whether open or closed, comment on the existing thread instead of posting a new issue.

Question

Apologies if this just sounds like a rant. I searched as I could through issues and documentation, but didn't find anything addressing this. I'm interested in knowing how people use the expect_ functions in testthat tests. I understand the main idea for these calls is just to pass or fail, but I find this behaviour not so helpful. In most testthat situations, if your test fails, you get some hopefully useful, but at least some feedback on e.g. which values failed to match. With pointblank expect_ functions we only get the number of failed rows. How could I automatically see which rows failed if the expectation test fails? I guess something like this is achievable with non expect_ functions by using get_agent_report interactively or some variant of get_data_extracts non interactively. If we have the tools, I guess it could be possible to easily see the failed rows in the expect_ calls?

Right now I get something like this:

── Failure (test_polities.R:38:3): get_polities is consistent ──────────────────
The `expect_col_vals_expr()` validation failed beyond the absolute threshold lev
el (1).
* failure level (1) >= failure threshold (1)
Backtrace:
    ▆
 1. ├─.expect_consistent_code_info(...) at test_polities.R:38:3
 2. │ ├─dplyr::select(...) at test_polities.R:2:3
 3. │ ├─pointblank::expect_col_vals_expr(...) at dplyr/R/select.R:54:3
 4. │ │ ├─... %>% .$validation_set at pointblank/R/col_vals_expr.R:432:3
 5. │ │ └─pointblank::create_agent(tbl = object, label = "::QUIET::") at pointbl
ank/R/col_vals_expr.R:350:3
 6. │ └─pointblank::expect_col_vals_expr(...) at pointblank/R/create_agent.R:510
:3
 7. ├─pointblank::interrogate(.) at magrittr/R/pipe.R:136:3
 8. └─pointblank::col_vals_expr(...) at pointblank/R/interrogate.R:169:3
 9.   └─pointblank:::is_a_table_object(x) at pointblank/R/col_vals_expr.R:350:3
[ FAIL 1 | WARN 0 | SKIP 0 | PASS 11 ]

This is not that helpful for debugging, as I would have to add some manual tweaks to see which rows might have failed, while automatically printing the failed rows (at least a few of them) in the test might make this faster to debug. Some questions:

1. What's the easiest way to give meaningful feedback in my tests with current pointblank functionality?
2. Why was the initial decision to not include any individual failed rows in the output when the expect_ fails?
3. Why is there no label argument for expect_ calls? Couldn't this be useful for showing a description of the expectation? I would usually add several expectations per testthat test.
4. Am I just using pointblank the wrong way? Do people usually just perform their validations in bare scripts?

If it helps, my typical testthat tests with pointblank would look something like this:

...
get_polities() |>
  pointblank::expect_col_exists(all_of(cols)) |>
  pointblank::expect_col_is_integer(all_of(int_cols)) |>
  pointblank::expect_col_is_character(all_of(other_cols)) |>
  pointblank::expect_col_vals_not_null(all_of(non_null_cols)) |>
  pointblank::expect_rows_distinct(polity_name) |>
  pointblank::expect_rows_distinct(polity_code) |>
  pointblank::expect_rows_distinct(display_code) |>
  pointblank::expect_col_vals_expr(~ start_year <= end_year) |>
  # More expect_ calls here
  .expect_consistent_code_info()
...

Anything that you think might help me regarding this is welcome.

Reproducible example

• For any problems you identify, post a minimal reproducible example so the maintainer can troubleshoot. A reproducible example is:
  • Runnable: post enough R code and data so any onlooker can create the error on their own computer.
  • Minimal: reduce runtime wherever possible and remove complicated details that are irrelevant to the issue at hand.
  • Readable: format your code according to the tidyverse style guide.