Skip to content

Conversation

@xmkg
Copy link
Member

@xmkg xmkg commented May 6, 2025

Right now, it's hard to tell which expect_log call is failed to be satisfied because the output is ambiguous:

 error: Actual function call count doesn't match EXPECT_CALL(*this, log(lvl, _, HasSubstr(substr)))...
         Expected: to be called once
           Actual: never called - unsatisfied and active

The patch fixes that by utilizing the matcher description:

 error: Actual function "my expected log string" call count doesn't match EXPECT_CALL(*this, log(lvl, _, HasSubstr(substr)))...
         Expected: to be called once
           Actual: never called - unsatisfied and active
```

@codecov
Copy link

codecov bot commented May 6, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.50%. Comparing base (b086fb7) to head (e8e9756).
⚠️ Report is 444 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4074   +/-   ##
=======================================
  Coverage   89.50%   89.50%           
=======================================
  Files         259      259           
  Lines       14740    14740           
=======================================
  Hits        13193    13193           
  Misses       1547     1547           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Collaborator

@ricab ricab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, but the sentence looks a little strange. Could do Description(fmt::format("log with {}", substr)) or maybe just use operator << ?

@xmkg
Copy link
Member Author

xmkg commented May 7, 2025

Good idea, but the sentence looks a little strange. Could do Description(fmt::format("log with {}", substr)) or maybe just use operator << ?

Hmm, EXPECT_CALL does not seem to implement "operator <<". I agree that we could use a nicer message, though. How about sth like:

Actual function "expect_log(level: debug, substr: "message", times: 1)" call count doesn't match EXPECT_CALL(*this, log(lvl, _, HasSubstr(substr))).

@ricab
Copy link
Collaborator

ricab commented May 7, 2025

Hmm, it's log whose call count doesn't match, not expect_log, right?

@xmkg
Copy link
Member Author

xmkg commented May 7, 2025

Hmm, it's log whose call count doesn't match, not expect_log, right?

The main problem comes from the assumed placement of the description. It's right between the [actual] and [expected], and whatever we write into the description will be associated with the actual part as a cognitive default. Given that we don't have an easy way to manipulate this ordering, let's ditch the function name altogether then. How about:

Actual function "level: debug, substr: `message`" call count doesn't match EXPECT_CALL(*this, log(lvl, _, HasSubstr(substr))

@ricab
Copy link
Collaborator

ricab commented May 7, 2025

There is maybe something I am missing, but why don't you say this?

Actual function "log(level: debug, substr: "message")" call count doesn't match EXPECT_CALL(*this, log(lvl, _, HasSubstr(substr))).

IOW, the actual call count for function log with those arguments didn't match the expectation. That would be accurate, no?

@xmkg
Copy link
Member Author

xmkg commented May 7, 2025

There is maybe something I am missing, but why don't you say this?

Actual function "log(level: debug, substr: "message")" call count doesn't match EXPECT_CALL(*this, log(lvl, _, HasSubstr(substr))).

IOW, the actual call count for function log with those arguments didn't match the expectation. That would be accurate, no?

I suppose so. There's a slight disconnect between what we actually write in unit tests (expect_log) vs. what's actually being logged (log), but ok. I could live with that.

Copy link
Collaborator

@ricab ricab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense to me, thanks!

Regarding the disconnect you mention, perhaps I haven't been sleeping enough and I may be short-circuiting, but expect_log is what the unit tests call to set a call-count expectation on log... That is what the message is saying doesn't match. If anything, it's EXPECT_CALL that could be replaced with expect_log (not necessary).

@xmkg
Copy link
Member Author

xmkg commented May 7, 2025

Regarding the disconnect you mention, perhaps I haven't been sleeping enough and I may be short-circuiting, but expect_log is what the unit tests call to set a call-count expectation on log... That is what the message is saying doesn't match. If anything, it's EXPECT_CALL that could be replaced with expect_log (not necessary).

Yeah, log() is hidden behind the expect_log() call, so what I'm saying is it might be slightly harder to correlate the output with the actual unit test case, but that's not that important. I'd take any single clue that would lead to the correct expect_log() line when something goes wrong, so I don't have any strong preference on that.

Regarding the EXPECT_CALL part: unfortunately we can't easily touch the EXPECT_CALL part of the description. I agree that it would look much cleaner, but IMO it's not worth the effort.

@ricab
Copy link
Collaborator

ricab commented May 8, 2025

Yeah, log() is hidden behind the expect_log() call [...]

EXPECT_CALL is — log() itself is called in the code being tested. But I understand what you mean better now. It would be good to have an indication of where the failed expect_log() was called in the tests. I agree what you have is good enough though 👍

@ricab
Copy link
Collaborator

ricab commented May 9, 2025

Hey @xmkg to keep the web a little simpler, we normally avoid merging the target branch back into the topic branch and rebase instead (except if there are many commits with conflicts). Would you mind doing that here, please?

xmkg added 2 commits May 9, 2025 16:24
Right now, it's hard to tell which expect_log call is failed to
be satisfied because the output is ambiguous:

 error: Actual function call count doesn't match EXPECT_CALL(*this, log(lvl, _, HasSubstr(substr)))...
         Expected: to be called once
           Actual: never called - unsatisfied and active

The patch fixes that by utilizing the matcher description:

 error: Actual function "my expected log string" call count doesn't match EXPECT_CALL(*this, log(lvl, _, HasSubstr(substr)))...
         Expected: to be called once
           Actual: never called - unsatisfied and active

Signed-off-by: Mustafa Kemal Gilor <mustafa.gilor@canonical.com>
Signed-off-by: Mustafa Kemal Gilor <mustafa.gilor@canonical.com>
@xmkg xmkg force-pushed the enhancement/add-description-to-expect-log branch from 9bdb3c5 to e8e9756 Compare May 9, 2025 13:24
@xmkg
Copy link
Member Author

xmkg commented May 9, 2025

Hey @xmkg to keep the web a little simpler, we normally avoid merging the target branch back into the topic branch and rebase instead (except if there are many commits with conflicts). Would you mind doing that here, please?

Done

@ricab
Copy link
Collaborator

ricab commented May 9, 2025

Thank you!

@xmkg
Copy link
Member Author

xmkg commented May 22, 2025

@georgeliao, can I get a secondary review when you're available? The changes are trivial.

Copy link
Collaborator

@sharder996 sharder996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 from me

@xmkg
Copy link
Member Author

xmkg commented May 22, 2025

+1 from me

Thanks Scott!

@xmkg xmkg added this pull request to the merge queue May 22, 2025
@ricab ricab removed this pull request from the merge queue due to a manual request May 22, 2025
@ricab
Copy link
Collaborator

ricab commented May 22, 2025

Hey @xmkg, I removed this from the queue to avoid having to rebase #4082 again, which is tricky due to the different authors and GPG keys. Sorry about that. We can add it back once that is done.

@xmkg xmkg added this pull request to the merge queue May 23, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 23, 2025
@xmkg xmkg added this pull request to the merge queue May 23, 2025
Merged via the queue into main with commit c8b704d May 23, 2025
19 checks passed
@xmkg xmkg deleted the enhancement/add-description-to-expect-log branch May 23, 2025 08:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants