@@ -23,12 +23,12 @@ a two-step procedure:
23
23
In practice, neither the marginal posterior nor the conditional posterior
24
24
are available in closed form and so they must be approximated.
25
25
The marginal posterior can be written as $p(\phi \mid y) \propto p(y \mid \phi) p(\phi)$,
26
- where $p(y \mid \phi) = \int p(y \mid \phi, \theta) p(\theta) d\theta$ $
27
- is called marginal likelihood. The Laplace method approximates
26
+ where $p(y \mid \phi) = \int p(y \mid \phi, \theta) p(\theta) d\theta$ $
27
+ is called marginal likelihood. The Laplace method approximates
28
28
$p(y \mid \phi, \theta) p(\theta)$ with a normal distribution and the
29
29
resulting Gaussian integral can be evaluated analytically to obtain an
30
- approximation to the log marginal likelihood
31
- $\log \hat p(y \mid \phi) \approx \log p(y \mid \phi)$.
30
+ approximation to the log marginal likelihood
31
+ $\log \hat p(y \mid \phi) \approx \log p(y \mid \phi)$.
32
32
33
33
Combining this marginal likelihood with the prior in the ` model `
34
34
block, we can then sample from the marginal posterior $p(\phi \mid y)$
@@ -40,21 +40,21 @@ depends on the case.
40
40
41
41
To obtain posterior draws for $\theta$, we sample from the normal
42
42
approximation to $p(\theta \mid y, \phi)$ in ` generated quantities ` .
43
- The process of iteratively sampling from $p(\phi \mid y)$ (say, with MCMC) and
43
+ The process of iteratively sampling from $p(\phi \mid y)$ (say, with MCMC) and
44
44
then $p(\theta \mid y, \phi)$ produces samples from the joint posterior
45
45
$p(\theta, \phi \mid y)$.
46
46
47
47
The Laplace approximation is especially useful if $p(\theta)$ is
48
48
multivariate normal and $p(y \mid \phi, \theta)$ is
49
49
log-concave. Stan's embedded Laplace approximation is restricted to
50
- have multivariate normal prior $p(\theta)$ and ... likelihood
50
+ have multivariate normal prior $p(\theta)$ and ... likelihood
51
51
$p(y \mid \phi, \theta)$.
52
52
53
53
54
54
## Specifying the likelihood function
55
55
56
- The first step to use the embedded Laplace approximation is to write down a
57
- function in the ` functions ` block which returns the log joint likelihood
56
+ The first step to use the embedded Laplace approximation is to write down a
57
+ function in the ` functions ` block which returns the log joint likelihood
58
58
` \log p(y \mid \theta, \phi) ` . There are a few constraints on this function:
59
59
60
60
* The function return type must be ` real `
@@ -63,7 +63,7 @@ function in the `functions` block which returns the log joint likelihood
63
63
have type ` vector ` .
64
64
65
65
* The operations in the function must support higher-order automatic
66
- differentation (AD). Most functions in Stan support higher-order AD.
66
+ differentiation (AD). Most functions in Stan support higher-order AD.
67
67
The exceptions are functions with specialized calls for reverse-mode AD, and
68
68
these are higher-order functions (algebraic solvers, differential equation
69
69
solvers, and integrators) and the suite of hidden Markov model (HMM) functions.
@@ -74,7 +74,7 @@ real ll_function(vector theta, ...)
74
74
```
75
75
There is no type restrictions for the variadic arguments ` ... ` and each
76
76
argument can be passed as data or parameter. As always, users should use
77
- parameter arguments only when nescessary in order to speed up differentiation.
77
+ parameter arguments only when necessary in order to speed up differentiation.
78
78
In general, we recommend marking data only arguments with the keyword ` data ` ,
79
79
for example,
80
80
```
@@ -98,11 +98,11 @@ is implicitly defined as the collection of all non-data arguments passed to
98
98
## Approximating the log marginal likelihood $\log p(y \mid \phi)$
99
99
100
100
In the ` model ` block, we increment ` target ` with ` laplace_marginal ` , a function
101
- that approximates the log marginal likelihood $\log p(y \mid \phi)$.
101
+ that approximates the log marginal likelihood $\log p(y \mid \phi)$.
102
102
This function takes in the
103
103
user-specified likelihood and prior covariance functions, as well as their arguments.
104
104
These arguments must be passed as tuples, which can be generated on the fly
105
- using parenthesis.
105
+ using parenthesis.
106
106
We also need to pass an argument $\theta_0$ which serves as an initial guess for
107
107
the optimization problem that underlies the Laplace approximation,
108
108
$$
@@ -113,11 +113,11 @@ passed to `ll_function`.
113
113
114
114
The signature of the function is:
115
115
```
116
- target += laplace_marginal(function ll_function, tupple (...), vector theta_0,
117
- function K_function, tupple (...));
116
+ real laplace_marginal(function ll_function, tuple (...), vector theta_0,
117
+ function K_function, tuple (...));
118
118
```
119
- The tuple ` (...) ` after ` ll_function ` contains the arguments that get passed
120
- to ` ll_function ` * excluding $\theta$* . Likewise, the tuple ` (...) ` after
119
+ The ` tuple (...)` after ` ll_function ` contains the arguments that get passed
120
+ to ` ll_function ` * excluding $\theta$* . Likewise, the ` tuple (...)` after
121
121
` ll_function ` contains the arguments that get passed to ` K_function ` .
122
122
123
123
It also possible to specify control parameters, which can help improve the
@@ -159,8 +159,8 @@ maximum number of steps in the linesearch is reached. By default,
159
159
With these arguments at hand, we can call ` laplace_marginal_tol ` with the
160
160
following signature:
161
161
```
162
- target += laplace_margina_tol(function ll_function, tupple (...), vector theta_0,
163
- function K_function, tupple (...),
162
+ target += laplace_margina_tol(function ll_function, tuple (...), vector theta_0,
163
+ function K_function, tuple (...),
164
164
real tol, int max_steps, int hessian_block_size,
165
165
int solver, int max_steps_linesearch);
166
166
```
@@ -172,22 +172,22 @@ approximation of $p(\theta \mid \phi, y)$ using `laplace_latent_rng`.
172
172
The signature for ` laplace_latent_rng ` follows closely
173
173
the signature for ` laplace_marginal ` :
174
174
```
175
- vector theta =
176
- laplace_latent_rng(function ll_function, tupple (...), vector theta_0,
177
- function K_function, tupple (...));
175
+ vector theta =
176
+ laplace_latent_rng(function ll_function, tuple (...), vector theta_0,
177
+ function K_function, tuple (...));
178
178
```
179
179
Once again, it is possible to specify control parameters:
180
180
```
181
- vector theta =
182
- laplace_latent_tol_rng(function ll_function, tupple (...), vector theta_0,
183
- function K_function, tupple (...),
181
+ vector theta =
182
+ laplace_latent_tol_rng(function ll_function, tuple (...), vector theta_0,
183
+ function K_function, tuple (...),
184
184
real tol, int max_steps, int hessian_block_size,
185
185
int solver, int max_steps_linesearch);
186
186
```
187
187
188
188
## Built-in Laplace marginal likelihood functions
189
189
190
- Stan supports certain built-in Laplace marginal likelihood functions.
190
+ Stan supports certain built-in Laplace marginal likelihood functions.
191
191
This selection is currently
192
192
narrow and expected to grow. The built-in functions exist for the user's
193
193
convenience but are not more computationally efficient than specifying log
@@ -196,7 +196,7 @@ likelihoods in the `functions` block.
196
196
### Poisson with log link
197
197
198
198
Given count data, with each observed count $y_i$ associated with a group
199
- $g(i)$ and a corresponding latent variable $\theta_ {g(i)}$, and Poisson model,
199
+ $g(i)$ and a corresponding latent variable $\theta_ {g(i)}$, and Poisson model,
200
200
the likelihood is
201
201
$$
202
202
p(y \mid \theta, \phi) = \prod_i\text{Poisson} (y_i \mid \exp(\theta_{g(i)})).
@@ -211,18 +211,18 @@ The signatures for the embedded Laplace approximation function with a Poisson
211
211
likelihood are
212
212
```
213
213
real laplace_marginal_poisson_log_lpmf(array[] int y | array[] int y_index,
214
- vector theta0, function K_function, (...));
214
+ vector theta0, function K_function, tuple (...));
215
215
216
216
real laplace_marginal_tol_poisson_log_lpmf(array[] int y | array[] int y_index,
217
- vector theta0, function K_function, (...),
217
+ vector theta0, function K_function, tuple (...),
218
218
real tol, int max_steps, int hessian_block_size,
219
219
int solver, int max_steps_linesearch);
220
220
221
221
vector laplace_latent_poisson_log_rng(array[] int y, array[] int y_index,
222
- vector theta0, function K_function, (...));
222
+ vector theta0, function K_function, tuple (...));
223
223
224
224
vector laplace_latent_tol_poisson_log_rng(array[] int y, array[] int y_index,
225
- vector theta0, function K_function, (...),
225
+ vector theta0, function K_function, tuple (...),
226
226
real tol, int max_steps, int hessian_block_size,
227
227
int solver, int max_steps_linesearch);
228
228
```
@@ -237,37 +237,37 @@ The signatures for this function are:
237
237
```
238
238
real laplace_marginal_poisson2_log_lpmf(array[] int y | array[] int y_index,
239
239
vector x, vector theta0,
240
- function K_function, (...));
240
+ function K_function, tuple (...));
241
241
242
242
real laplace_marginal_tol_poisson2_log_lpmf(array[] int y | array[] int y_index,
243
243
vector x, vector theta0,
244
- function K_function, (...),
244
+ function K_function, tuple (...),
245
245
real tol, int max_steps, int hessian_block_size,
246
246
int solver, int max_steps_linesearch);
247
247
248
248
vector laplace_latent_poisson2_log_rng(array[] int y, array[] int y_index,
249
- vector x, vector theta0,
250
- function K_function, (...));
249
+ vector x, vector theta0,
250
+ function K_function, tuple (...));
251
251
252
252
vector laplace_latent_tol_poisson2_log_rng(array[] int y, array[] int y_index,
253
253
vector x, vector theta0,
254
- function K_function, (...),
254
+ function K_function, tuple (...),
255
255
real tol, int max_steps, int hessian_block_size,
256
256
int solver, int max_steps_linesearch);
257
257
```
258
258
259
259
260
260
### Negative Binomial with log link
261
261
262
- The negative Bionomial distribution generalizes the Poisson distribution by
262
+ The negative Binomial distribution generalizes the Poisson distribution by
263
263
introducing the dispersion parameter $\eta$. The corresponding likelihood is then
264
264
$$
265
265
p(y \mid \theta, \phi) = \prod_i\text{NegBinomial2} (y_i \mid \exp(\theta_{g(i)}), \eta).
266
266
$$
267
- Here we use the alternative paramererization implemented in Stan, meaning that
267
+ Here we use the alternative parameterization implemented in Stan, meaning that
268
268
$$
269
- \mathbb E(y_i) = \exp (\theta_{g(i)}), \\
270
- \text{Var}(y_i) = \mathbb E(y_i) + \frac{(\mathbb E(y_i))^2}{\eta}.
269
+ \mathbb E(y_i) = \exp (\theta_{g(i)}), \\
270
+ \text{Var}(y_i) = \mathbb E(y_i) + \frac{(\mathbb E(y_i))^2}{\eta}.
271
271
$$
272
272
The arguments for the likelihood function are:
273
273
@@ -279,23 +279,23 @@ group the $i^\text{th}$ observation belongs to.
279
279
The function signatures for the embedded Laplace approximation with a negative
280
280
Binomial likelihood are
281
281
```
282
- real laplace_marginal_neg_binomial_2_log_lpmf(array[] int y |
283
- array[] int y_index, real eta, vector theta0,
284
- function K_function, (...));
282
+ real laplace_marginal_neg_binomial_2_log_lpmf(array[] int y |
283
+ array[] int y_index, real eta, vector theta0,
284
+ function K_function, tuple (...));
285
285
286
- real laplace_marginal_tol_neg_binomial_2_log_lpmf(array[] int y |
287
- array[] int y_index, real eta, vector theta0,
288
- function K_function, (...),
286
+ real laplace_marginal_tol_neg_binomial_2_log_lpmf(array[] int y |
287
+ array[] int y_index, real eta, vector theta0,
288
+ function K_function, tuple (...),
289
289
real tol, int max_steps, int hessian_block_size,
290
290
int solver, int max_steps_linesearch);
291
291
292
- vector laplace_latent_neg_binomial_2_log_rng(array[] int y,
293
- array[] int y_index, real eta, vector theta0,
294
- function K_function, (...));
292
+ vector laplace_latent_neg_binomial_2_log_rng(array[] int y,
293
+ array[] int y_index, real eta, vector theta0,
294
+ function K_function, tuple (...));
295
295
296
- vector laplace_latent_tol_neg_binomial_2_log_rng(array[] int y,
297
- array[] int y_index, real eta, vector theta0,
298
- function K_function, (...),
296
+ vector laplace_latent_tol_neg_binomial_2_log_rng(array[] int y,
297
+ array[] int y_index, real eta, vector theta0,
298
+ function K_function, tuple (...),
299
299
real tol, int max_steps, int hessian_block_size,
300
300
int solver, int max_steps_linesearch);
301
301
```
@@ -314,23 +314,23 @@ group the $i^\text{th}$ observation belongs to.
314
314
315
315
The function signatures for the embedded Laplace approximation with a Bernoulli likelihood are
316
316
```
317
- real laplace_marginal_bernoulli_logit_lpmf(array[] int y |
318
- array[] int y_index, real eta, vector theta0,
319
- function K_function, (...));
317
+ real laplace_marginal_bernoulli_logit_lpmf(array[] int y |
318
+ array[] int y_index, real eta, vector theta0,
319
+ function K_function, tuple (...));
320
320
321
- real laplace_marginal_tol_bernoulli_logit_lpmf(array[] int y |
322
- array[] int y_index, real eta, vector theta0,
323
- function K_function, (...),
321
+ real laplace_marginal_tol_bernoulli_logit_lpmf(array[] int y |
322
+ array[] int y_index, real eta, vector theta0,
323
+ function K_function, tuple (...),
324
324
real tol, int max_steps, int hessian_block_size,
325
325
int solver, int max_steps_linesearch);
326
326
327
- vector laplace_latent_bernoulli_logit_rng(array[] int y,
328
- array[] int y_index, real eta, vector theta0,
329
- function K_function, (...));
327
+ vector laplace_latent_bernoulli_logit_rng(array[] int y,
328
+ array[] int y_index, real eta, vector theta0,
329
+ function K_function, tuple (...));
330
330
331
- vector laplace_latent_tol_bernoulli_logit_rng(array[] int y,
332
- array[] int y_index, real eta, vector theta0,
333
- function K_function, (...),
331
+ vector laplace_latent_tol_bernoulli_logit_rng(array[] int y,
332
+ array[] int y_index, real eta, vector theta0,
333
+ function K_function, tuple (...),
334
334
real tol, int max_steps, int hessian_block_size,
335
335
int solver, int max_steps_linesearch);
336
336
```
@@ -374,7 +374,7 @@ vector laplace_latent_tol_bernoulli_logit_rng(array[] int y,
374
374
<!-- and admits nearly the same arguments as `laplace_marginal`. A key difference -->
375
375
<!-- is that -->
376
376
<!-- ``` -->
377
- <!-- vector laplace_latent_rng(function ll_function, tupple (...), vector theta_0, -->
378
- <!-- function K_function, tupple (...)); -->
377
+ <!-- vector laplace_latent_rng(function ll_function, tuple (...), vector theta_0, -->
378
+ <!-- function K_function, tuple (...)); -->
379
379
<!-- ``` -->
380
380
0 commit comments