How to tell if blaze has kicked in?

[mattwparas]

I'd assume that if I were to check the query plan I'd see things related to "native" stages - is that true? Is it guaranteed that something like parquet scan will translate to a native plan?

More or less - I'm not sure if my setup is incorrect or if I'm looking in the right place to check things

[merrily01]

Checking the query plan is indeed an effective way to determine if Blaze has intervened.

I think if the "native" phase appears in the query plan and you have enabled Blaze as described above, then Blaze can be considered effective.

[cxzl25]

You need to set spark.sql.adaptive.enabled=false and then use explain.

set spark.sql.adaptive.enabled=false;

EXPLAIN FORMATTED SQL

planName=$(echo "$plan" | grep -E '^\([0-9]+\)')
echo "$planName" | wc -l
echo "$planName" | grep -E  "Native" | wc -l