Skip to content
Discussion options

You must be logged in to vote

Hi @mps-hlx,

This is intentional. We discuss this in the preprint, see end of section 3.2.1.

We recommend the following parametric testing workflow: repeated measures ANOVA followed by the Tukey HSD test. During the repeated CV procedure, competing methods are being fit to the same splits of data. To appropriately account for this dependency, we perform repeated measures ANOVA, and then provide the sum of squared errors output to the Tukey HSD procedure. This results in a test with higher statistical power than TukeyHSD alone.

The TukeyHSD assumes independence of samples, which for repeated CV will result in CIs that are too large. Repeated measures ANOVA appropriately accounts for the …

Replies: 1 comment

Comment options

You must be logged in to vote
0 replies
Answer selected by jrash
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants