An intuitive interaction-testing implementation

[jafarre-bi]

Hello,

The first time I knew about Spock, it took me 5 minutes to fall in love with it and decide it was the testing framework I would use from that moment on. And I haven't used anything else since then.
One of the most valuable features of Spock is its intuitive way of expressing specifications. It's so easy to read and understand them that they also serve as documentation.

And then you try interaction testing.

I am aware of the technical reasons behind it, but the behaviour of interaction testing is (at least to me) counterintuitive. And this is IMO the opposite of what Spock tries to provide.

I'd say the reasons roughly boil down to two:

• If you want to specify both the return values and the interactions, it must be done in the same sentence. When specifying return values goes naturally in the given or when clauses, and interactions intuitively fall in the then clause.
• Interactions aren't validated in the place where you declare them, leading to bizarre behaviours. For example, you have several when - then blocks in sequence and define the interactions that should happen in each of them, but then realise it's validating an interaction defined for a later then block before another interaction defined in the current then block.

I have been thinking about it, and IMO, these limitations come from the way it's been implemented, not from any functional design decision or testing formalism. I also think it wouldn't be that difficult to implement an intuitive behaviour, so let me summarise my take on an intuitive way to implement it:

1. Mock objects record interactions per invoked method. They could naively record all interactions or just the ones with methods for which we have written at least an interaction check. Interactions are no longer moved to the given block, but become actual tests in the then block. By default, I would expect interactions to happen in the same order I've written them. This means that the check is as easy as looking into the first recorded interaction, consuming it (i.e., deleting it from the list of recorded interactions), and checking whether it matches or not. If it doesn't match, you don't need to check any other recorded interaction, because even if it matched one, it wouldn't follow the specified sequence, so it's a failure.
2. The previous behaviour can be generalised by providing a means (something like anyOrder { ... }) to specify that a group of interactions can happen in any order. Let's say you specify three interactions in such a block. You only need to get the first three recorded interactions and match them with the specified ones in any order, consuming them. Again, if there aren't enough or not all of them match, that's a failure.
3. With this way of testing interactions in place, it's straightforward to allow specifying return values independently. You maintain a list of return values and provide them in sequence while recording interactions. Nothing else.

There must be many things I'm missing, but this proposal feels sound and relatively easy to implement, to the point that I could even consider contributing to it, if my limited time allows.
More importantly, now that you are about to release Spock 2.4 it's precisely the right time for a bigger change like this.

Sorry if this topic has been raised too many times before, but to me, this is the most important improvement to make Spock the Most Intuitive Way to specify behaviour and testing in the market.

Thank you very much for your fantastic work.

[Vampire]

I agree that getting the interactions right is often not the most intuitive thing, but I don't think there can too much be done without dropping significant functionality and breaking most tests using interactions out there. The limitations are because of how it is implemented, yes, but changing this would most probably be a huuuuge refactoring of the implementation logic. Feel free to try implementing something that makes it feel more natural to you, but I think it will be a very non-trivial change and most probably not something you will finish for 2.4 from what my gut feeling tells, if Leonard at all will accept it to the codebase in the end.
For example you cannot know for what a method an interaction will match at runtime, because you can for example have things like 1 * subscriber./r.*e/("hello") that match all methods that start with an r and end with an e and so on.

[jafarre-bi]

Right, well, the brute-force way would be to record every interaction. And yes, it must be a significant change, it's just that the idea seems so simple that it's tempting :)
But you're right that, if suddenly interactions become conditions in the then block, we would face a problem with existing interactions defined in the given block, for instance. It could be an opt-in alternative.
For me, it isn't only about being more or less easy to get them right, it's also that the final result is often not as easy to read as it could be in theory.
I'll look around and see...