Small sample inference for GLMM?

[jarbet]

Best practices have been developed for doing inference with linear mixed models (LMMs) given small/moderate sample size. Briefly, Luke 2017 recommends using REML with degrees of freedom estimated by Kenward-Roger or Satterthwaite methods.

My question is: does theparameters R package implement any methods for doing better small sample inference on fixed effects in GLMMs, for example, logistic mixed effect model?

I believe SAS implements a Kenward-Rodger method for GLMM, but I cannot find any R packages that implement this.

[bwiernik]

You can use REML with your estimating package (lme4, glmmTMB, etc) with generalized models. You can apply various degrees of freedom approximation methods with the ci_method argument in parameters and other packages in easystats. Note that all of these methods were developed for Gaussian models and they don't have clear theoretical foundation for generalized models (but do often perform quite well).

[jarbet]

Are you sure that parameters supports this for glmmTMB? Here's a quick example. Note the output shows Z-tests, not t-tests with Kenward Rodgers DF.

# Load required packages
library(glmmTMB)
library(parameters)

# Simulated data
set.seed(123)
df <- data.frame(
  outcome = rbinom(100, 1, 0.5),
  group = gl(10, 10),
  x = rnorm(100)
)

# Fit logistic mixed model
model <- glmmTMB(outcome ~ x + (1 | group), data = df, family = binomial(), REML = TRUE)

# Get summary with Kenward-Roger approximation
model_summary <- model_parameters(model, df_method = "kenward")

# View results
print(model_summary)
#> # Fixed Effects
#> 
#> Parameter   | Log-Odds |   SE |        95% CI |     z |     p
#> -------------------------------------------------------------
#> (Intercept) |    -0.14 | 0.23 | [-0.59, 0.30] | -0.64 | 0.524
#> x           |    -0.35 | 0.22 | [-0.79, 0.08] | -1.60 | 0.110
#> 
#> # Random Effects
#> 
#> Parameter             | Coefficient |       95% CI
#> --------------------------------------------------
#> SD (Intercept: group) |        0.31 | [0.02, 4.61]
#> 
#> Uncertainty intervals (equal-tailed) and p-values (two-tailed) computed
#>   using a Wald z-distribution approximation.
#> 
#> The model has a log- or logit-link. Consider using `exponentiate =
#>   TRUE` to interpret coefficients as ratios.

^{Created on 2025-05-09 with reprex v2.1.1}

My understanding from the SAS documentation is that SAS PROC GLIMMIX does support KR and Satterwaite methods for approximating DF in GLMM. But I haven't found any R software that supports this, it be awesome if parameters could! I'm not going to use SAS lol.

[strengejacke]

There's an issue open to implement it in glmmTMB:
glmmTMB/glmmTMB#536

For GLMMs, parameters has "ml1" and "betwithin" as alternative

[jarbet]

For GLMMs, parameters has "ml1" and "betwithin" as alternative

Does your implementation of the ml1 method work for any GLMM? My understanding is this method does not work for more complex random effect structures. The function help page says the following:

Limitations of the "m-l-1" Heuristic
Note that the "m-l-1" heuristic is not applicable (or at least less accurate) for complex multilevel designs, e.g. with cross-classified clusters. In such cases, more accurate approaches like the Kenward-Roger approximation (dof_kenward()) is recommended. However, the "m-l-1" heuristic also applies to generalized mixed models, while approaches like Kenward-Roger or Satterthwaite are limited to linear mixed models only.

If I'm understanding the above quote, it sounds like model_parameters(model, ci_method = "ml1") will work for any GLMM although it may be less accurate for complex random effect structures... but probably still better than assuming infinite degrees of freedom?