problem with viterbi() (Error in sub_obs_probs[1, ] : incorrect number of dimensions)

[anchan-research]

Hello,

I was wondering if anyone had ran into this problem. After I fitted the model
hmm <- HMM$fit()

getting the decoded states

decoded_states <- hmm$viterbi()

I get the following error:

Error in sub_obs_probs[1, ] : incorrect number of dimensions

Any thoughts? Would be happy to share any info as needed.

I fitted a 2-state hmm model with two obs data streams. (gamma and beta distribution)

[TheoMichelot]

Hi, could you try running the following example, which fits a 2-state HMM with gamma and beta distributions to simulated data?

library(hmmTMB)

# Simulate data from 2-state HMM
n <- 1000
tpm <- matrix(c(0.9, 0.1, 0.1, 0.9), 2, 2)
S <- rep(1, n)
for(i in 2:n) S[i] <- sample(1:2, size = 1, prob = tpm[S[i-1],])
z1 <- rgamma(n = n, shape = c(0.5, 5)[S], scale = c(1/3, 1)[S])
z2 <- rbeta(n = n, shape1 = c(1, 3)[S], shape2 = c(0.5, 5)[S])
data <- data.frame(z1 = z1, z2 = z2)

# Define hidden state model
hid <- MarkovChain$new(data = data, n_states = 2)

# Define observation model
dists <- list(z1 = "gamma", z2 = "beta")
par0 <- list(z1 = list(shape = c(0.3, 4), scale = c(0.5, 1)),
             z2 = list(shape1 = c(2, 4), shape2 = c(1, 4)))
obs <- Observation$new(data = data, dists = dists, n_states = 2, par = par0)

# Fit HMM
hmm <- HMM$new(obs = obs, hid = hid)
hmm$fit()

# Compare estimated states to true states
states <- hmm$viterbi()
table(true = S, estimated = states)
#    estimated
# true   1   2
#    1 504   1
#    2   1 494

If this runs fine, then it seems that the problem is coming from something that is specific to your model (e.g., numerical problems during model fitting). Could you post the output of hmm$tmb_rep()?

[anchan-research]

Hello,

Thank you for your very quick response. The simulation fit fine. The output from hmm$tmb_rep() is
hmm$tmp_rep()

sdreport(.) result
Estimate Std. Error
coeff_fe_obs 0.63311841 0.04589000
coeff_fe_obs 1.27139524 0.03838375
coeff_fe_obs 5.45834139 0.06539222
coeff_fe_obs 6.29077976 0.03826637
coeff_fe_obs -0.28994545 0.04469784
coeff_fe_obs -0.06537758 0.03449165
coeff_fe_obs -0.06892487 0.04807179
coeff_fe_obs -0.06343957 0.03447673
coeff_fe_hid -3.49569688 0.23466938
coeff_fe_hid -3.00881162 0.13491875
Maximum gradient component: 0.007174004

My gamma variable ranges from ~5 to 5000 and I want my first state to be around 5 - 300 and second to over the rest. It might be too big of an ask?

[TheoMichelot]

Everything looks fine in the output. What are you setting initial_state to when creating the MarkovChain object?

[anchan-research]

So I figured it out - I was giving "estimated" for the initial state, but changed it to a potential state based on two observational data. I also couldn't give initial transitional probabilities for some reason, but the issue was resolved when I took the argument out, and the model seems fine. Sorry for the delayed response on this. Thank you so much for all your help and for creating this package!

Best,
Aung

[TheoMichelot]

Great, happy to hear you figured it out!