feat: add zio spark test assertion system

[dylandoamaral]

🚧 On hold, it's not clear if this is useful as it is. It creates a lot of issue, given it doesn't really compose at the moment. 🚧

Here is the missing step for now :

• Create a way to simply create spark macros close to ZIO way of doing tests
• Validate the design
• Add macro for assertSpark
• Modify macro to get a better output message

[ahoy-jon]

Why does it need a macro ? Spark produce values (seq(jsonoids)) that need ...

Do you have an example for the syntax?

[dylandoamaral]

Why does it need a macro ? Spark produce values (seq(jsonoids)) that need ...

Do you have an example for the syntax?

For the moment it generates something like this :
/**

• What we have :
• Assertion failed:
  ✗ false was not true
  Dataset was not empty
  Dataset(1, 2, 3) did not satisfy isEmpty
  Dataset(1, 2, 3) = false
  at ...
  */

You need the macro to retrieve the value (Dataset(1, 2, 3)) and the assertion (isEmpty) from our custom assert function.

[ahoy-jon]

Reviewed, macro are not needed for this use case.
Are we adding them for something else later on? or related to quill / iskra?

I don't see the use case here, unless we are moving fast to explain advanced matchers :

df.expectAtLeast(
  row(__, 13), 
  row.as[Person].assertTrue(_.name.contains("toto") || _.age > 10)
)

But that doesn't look like the actual direction.

@dylandoamaral could you explain why we need the macro for this PR ? Can the macro be reserved for another PR for advanced predicate checks?

I will take a look at zio-test to see if they use macro outside code Location or assertTrue.

[ahoy-jon]

pb, the user need the macro as a dependencie, as well?

[dylandoamaral]

Yes it it the same expectation for zio: https://github.com/zio/zio/blob/series/2.x/project/BuildHelper.scala#L254.

[dylandoamaral]

Reviewed, macro are not needed for this use case. Are we adding them for something else later on? or related to quill / iskra?

I don't see the use case here, unless we are moving fast to explain advanced matchers :

df.expectAtLeast(
  row(__, 13), 
  row.as[Person].assertTrue(_.name.contains("toto") || _.age > 10)
)

But that doesn't look like the actual direction.

@dylandoamaral could you explain why we need the macro for this PR ? Can the macro be reserved for another PR for advanced predicate checks?

I will take a look at zio-test to see if they use macro outside code Location or assertTrue.

I explained above why I use the macro, it is exactly the same code from zio assert.