gemma 27b slow on v2

[terrykong]

I am noticing that gemma 27b is significantly slower on the v2 backend. I don't observe such a large slowdown with other models.

Here are three sets of runs:

https://wandb.ai/nvidia/nemo-rl?nw=drf8mhln88

• green: v2
• blue, red: v1

There is some in-run variance in perf (~8%), but the diff from v1 is much larger