Merge pull request #97 from Julia-Tempering/fix-docs

fix documentation under the hood

Author: miguelbiron

docs/make.jl

@@ -28,7 +28,7 @@ makedocs(;
     sitename="Pigeons.jl",
     strict=true,
     format=Documenter.HTML(;
-        prettyurls=get(ENV, "CI", "false") == "true",
+        prettyurls=true, # always on, avoids confusion when building locally. If needed, serve the "build" folder locally with LiveServer. #get(ENV, "CI", "false") == "true",
         canonical="https://Julia-Tempering.github.io/Pigeons.jl",
         edit_link="main",
         assets=String[],

docs/src/correctness.md

@@ -7,7 +7,7 @@ CurrentModule = Pigeons
 It is notoriously difficult to implement correct parallel/distributed algorithms. 
 One strategy we use to address this is to guarantee that the code will output 
 precisely the same output no matter how many threads/machines are used. 
-We describe how this is done under the hood in the page [Distributed PT](distributed.html). 
+We describe how this is done under the hood in the page [Distributed PT](@ref distributed). 
 
 In practice, how is this useful? Let us say you developed a new target and you would like
 to make sure that it works correctly in a multi-threaded environment. To do so, add a flag to indicate to "check" one of the PT rounds as follows, and 

docs/src/distributed.md

@@ -2,7 +2,7 @@
 CurrentModule = Pigeons
 ```
 
-# Distributed and parallel implementation of PT 
+# [Distributed and parallel implementation of PT](@id distributed)
 
 ## Introduction
 
@@ -15,7 +15,7 @@ parallelized, and randomized algorithm.
     Read this page if you are interested in extending Pigeons or 
     understanding how it works under the hood. 
     Reading this page is not required to use Pigeons. Instead, refer to the 
-    [user guide](index.html). 
+    [user guide](@ref index). 
 
 In Distributed PT, one or several computers run MCMC simulations in parallel and 
 communicate with each other to improve MCMC efficiency. 
@@ -79,7 +79,7 @@ Let us start with a high-level picture of the distributed PT algorithm.
 The high-level code is the function [`pigeons()`](@ref) which is identical to the single-machine algorithm. 
 A first difference lay in the [`replicas`](@ref) datastructure taking on a different type. Also, as promised the 
 output is identical despite a vastly different swap logic: this can be checked using the `checked_round` 
-argument described in the [user guide](index.html). 
+argument described in the [user guide](@ref index). 
 A second difference between the execution of [`pigeons()`](@ref) in single vs many machine context is the behaviour 
 of [`swap!`](@ref) which is dispatched 
 based on the type of 

docs/src/index.md

@@ -2,14 +2,14 @@
 CurrentModule = Pigeons
 ```
 
-# Pigeons
+# [Pigeons](@id index)
 
 ## Summary
 
 `Pigeons` is a Julia package to approximate challenging posterior distributions, and more broadly, Lebesgue integration problems. Pigeons can be used in a multi-threaded context, and/or distributed over hundreds or thousands of MPI-communicating machines.
 
-Pigeons supports many [different ways to specify integration/expectation problems](input-overview.html) and 
-provides [rich and configurable output](output-overview.html). 
+Pigeons supports many [different ways to specify integration/expectation problems](@ref input-overview) and 
+provides [rich and configurable output](@ref output-overview). 
 
 Pigeons' core algorithm is a distributed and parallel implementation 
 of the following algorithms: 
@@ -22,7 +22,7 @@ These algorithms achieve state-of-the-art performance for approximation
 of challenging probability distributions.
 
 
-## Installing Pigeons
+## [Installing Pigeons](@id installing-pigeons)
 
 1. If you have not done so, install [Julia](https://julialang.org/downloads/). Julia 1.8 and higher are supported. 
 2. Install `Pigeons` using

docs/src/input-explorers.md

@@ -2,7 +2,7 @@
 CurrentModule = Pigeons
 ```
 
-# Custom explorers
+# [Custom explorers](@id input-explorers)
 
 Pigeons have several built-in [`explorer`](@ref) kernels such as 
 [`AutoMALA`](@ref) and a [`SliceSampler`](@ref). 
@@ -37,8 +37,8 @@ Pigeons.initialization(::MyLogPotential, ::AbstractRNG, ::Int) = [0.5, 0.5]
 
 We show how create a new explorer, 
 for pedagogy, a simple [independence Metropolis algorithm](https://bookdown.org/rdpeng/advstatcomp/metropolis-hastings.html#independence-metropolis-algorithm), applied to 
-our familiar [unidentifiable toy example](unidentifiable-example.html), 
-based on [Julia black-box implementation](input-julia.html). 
+our familiar [unidentifiable toy example](@ref unidentifiable-example), 
+based on [Julia black-box implementation](@ref input-julia). 
 
 ```@example explorer
 struct MyIndependenceSampler 

docs/src/input-julia.md

@@ -2,15 +2,15 @@
 CurrentModule = Pigeons
 ```
 
-# Julia code as input to pigeons
+# [Julia code as input to pigeons](@id input-julia)
 
 In typical Bayesian statistics applications, it is 
 easiest to specify the model in a modelling language, 
 such as Turing, but sometimes to get more flexibility or 
 speed it is useful to implement the density evaluation 
 manually as a "black-box" Julia function. 
 
-Here we show how this is done using our familiar [unidentifiable toy example](unidentifiable-example.html)
+Here we show how this is done using our familiar [unidentifiable toy example](@ref unidentifiable-example)
 [ported to the Stan language](https://github.com/Julia-Tempering/Pigeons.jl/blob/main/examples/stan/unid.stan).
 
 We first create a custom type, `MyLogPotential` to control dispatch on the interface [`target`](@ref).
@@ -119,14 +119,13 @@ Pigeons have several built-in [`explorer`](@ref) kernels such as
 However when the state space is neither the reals nor the integers, 
 or for performance reasons, it may be necessary to create custom 
 exploration MCMC kernels.
-This is described on the [custom explorers page](input-explorers.html).
+This is described on the [custom explorers page](@ref input-explorers).
 
 
 ## Manipulating the output
 
 Some 
-common post-processing are shown below, see [the section on output processing for more information](output-overview
-.html). 
+common post-processing are shown below, see [the section on output processing for more information](@ref output-overview). 
 
 ```@example julia
 using MCMCChains
@@ -146,7 +145,7 @@ samples
 ```
 
 ```@raw html
-<iframe src="julia_posterior_densities_and_traces.html" style="height:500px;width:100%;"></iframe>
+<iframe src="../julia_posterior_densities_and_traces.html" style="height:500px;width:100%;"></iframe>
 ```
 
 

docs/src/input-nonjulian.md

@@ -2,7 +2,7 @@
 CurrentModule = Pigeons
 ```
 
-# Targeting a non-Julian model 
+# [Targeting a non-Julian model](@id input-nonjulian)
 
 Suppose you have some code implementing vanilla MCMC, written in an arbitrary "foreign" language such as C++, Python, R, Java, etc. You would like to turn this vanilla MCMC code into a Parallel Tempering algorithm able to harness large numbers of cores, including distributing this algorithm over MPI. However, you do not wish to learn anything about MPI/multi-threading/Parallel Tempering.
 
@@ -48,7 +48,7 @@ Pigeons.setup_blang("blangDemos")
 
 Next, we run a  
 [Blang implementation](https://github.com/UBC-Stat-ML/blangDemos/blob/master/src/main/java/demos/UnidentifiableProduct.bl) of 
-our usual [unidentifiable toy example](unidentifiable-example.html):
+our usual [unidentifiable toy example](@ref unidentifiable-example):
 
 ```@example blang
 using Pigeons

docs/src/input-overview.md

@@ -2,18 +2,18 @@
 CurrentModule = Pigeons
 ```
 
-# Overview: inputting an integral/expectation problem into pigeons
+# [Overview: inputting an integral/expectation problem into pigeons](@id input-overview)
 
 Pigeons takes as input an expectation or integration problem.
 Pigeons supports a wide range of methods for specifying the input problem, 
 described in the pages below. 
 
-- [Turing.jl model](input-turing.html): a succinct specification of a joint distribution from which a posterior (target) and prior (reference) are extracted. 
-- [Black-box Julia function](input-julia.html): less automated, but more general and fully configurable. 
-- [Stan model](input-stan.html): a convenient adaptor for the most popular Bayesian modelling language. 
-- [MCMC code implemented in another language](input-nonjulian.html): bridging your MCMC code to pigeons to make it distributed and parallel. 
-- [Customize the MCMC explorers used by PT](input-explorers.html).
+- [Turing.jl model](@ref input-turing): a succinct specification of a joint distribution from which a posterior (target) and prior (reference) are extracted. 
+- [Black-box Julia function](@ref input-julia): less automated, but more general and fully configurable. 
+- [Stan model](@ref input-stan): a convenient adaptor for the most popular Bayesian modelling language. 
+- [MCMC code implemented in another language](@ref input-nonjulian): bridging your MCMC code to pigeons to make it distributed and parallel. 
+- [Customize the MCMC explorers used by PT](@ref input-explorers).
 
 We exemplify these different input methods on a recurrent example: 
 an unidentifiable toy model, 
-see [the page describing the recurrent example in more details](unidentifiable-example.html). 
+see [the page describing the recurrent example in more details](@ref unidentifiable-example). 

docs/src/input-stan.md

@@ -2,7 +2,7 @@
 CurrentModule = Pigeons
 ```
 
-# Stan model as input to pigeons
+# [Stan model as input to pigeons](@id input-stan)
 
 !!! note
 
@@ -17,7 +17,7 @@ To target the posterior distribution specified by
 a [Stan](https://mc-stan.org/) model, use 
 a [`StanLogPotential`](@ref). 
 
-Here we show how this is done using our familiar [unidentifiable toy example](unidentifiable-example.html)
+Here we show how this is done using our familiar [unidentifiable toy example](@ref unidentifiable-example)
 [ported to the Stan language](https://github.com/Julia-Tempering/Pigeons.jl/blob/main/examples/stan/unid.stan).
 
 ```@example stan
@@ -85,8 +85,7 @@ However, sample post-processing functions such as [`sample_array()`](@ref) and [
 convert back to the original ("constrained") parameterization via [`extract_sample()`](@ref). 
 
 As a result parameterization issues can be essentially ignored when post-processing, for example some 
-common post-processing are shown below, see [the section on output processing for more information](output-overview
-.html). 
+common post-processing are shown below, see [the section on output processing for more information](@ref output-overview). 
 
 ```@example stan
 using MCMCChains
@@ -105,7 +104,7 @@ samples
 ```
 
 ```@raw html
-<iframe src="stan_posterior_densities_and_traces.html" style="height:500px;width:100%;"></iframe>
+<iframe src="../stan_posterior_densities_and_traces.html" style="height:500px;width:100%;"></iframe>
 ```
 
 

docs/src/input-turing.md

@@ -2,7 +2,7 @@
 CurrentModule = Pigeons
 ```
 
-# Turing.jl model as input to pigeons
+# [Turing.jl model as input to pigeons](@id input-turing)
 
 To target the posterior distribution specified by 
 a [Turing.jl](https://github.com/TuringLang/Turing.jl) model use 
@@ -37,8 +37,7 @@ However, sample post-processing functions such as [`sample_array()`](@ref) and [
 convert back to the original ("constrained") parameterization via [`extract_sample()`](@ref). 
 
 As a result parameterization issues can be essentially ignored when post-processing, for example some 
-common post-processing are shown below, see [the section on output processing for more information](output-overview
-.html). 
+common post-processing are shown below, see [the section on output processing for more information](@ref output-overview). 
 
 ```@example turing
 using MCMCChains
@@ -56,6 +55,6 @@ samples
 ```
 
 ```@raw html
-<iframe src="turing_posterior_densities_and_traces.html" style="height:500px;width:100%;"></iframe>
+<iframe src="../turing_posterior_densities_and_traces.html" style="height:500px;width:100%;"></iframe>
 ```
 

docs/src/mpi.md

@@ -44,7 +44,7 @@ Create an issue if you would like another submission system included.
 
 Follow these instructions to run MPI over several machines:
 
-1. In the cluster login node, follow the [local installation instructions](index.html). 
+1. In the cluster login node, follow the [local installation instructions](@ref installing-pigeons). 
 2. Start Julia in the login node, and perform a one-time setup by calling [`setup_mpi()`](@ref). Its argument are the fields in [`MPISettings`](@ref), see the documentation there for details.
 3. Still in the Julia REPL running in the login node, use:
 
@@ -77,4 +77,4 @@ and cancel/kill a job using
 kill_job(mpi_run)
 ```
 
-To analyze the output, see the documentation page on [post-processing for MPI runs](output-mpi-postprocessing.html).
+To analyze the output, see the documentation page on [post-processing for MPI runs](@ref output-mpi-postprocessing).

docs/src/output-custom-types.md

@@ -2,7 +2,7 @@
 CurrentModule = Pigeons
 ```
 
-# Output for custom types
+# [Output for custom types](@id output-custom-types)
 
 The [`sample_array`](@ref) function assumes that the variables are real or integer (the latter coerced into the former) 
 and "flattened" into a uniform array. 
@@ -21,5 +21,5 @@ vector = get_sample(pt)
 length(vector) # = number of iterations = 2^10
 ```
 
-Another option is to use [off-memory processing](output-off-memory.html) which makes no assumption 
+Another option is to use [off-memory processing](@ref output-off-memory) which makes no assumption 
 either on the type of each individual sample.

docs/src/output-mpi-postprocessing.md

@@ -3,7 +3,7 @@ CurrentModule = Pigeons
 ```
 
 
-# Post-processing for MPI runs (plotting, summaries, etc) 
+# [Post-processing for MPI runs (plotting, summaries, etc)](@id output-mpi-postprocessing) 
 
 Two options are available to post-process samples produced from 
 MPI runs: (1) loading  
@@ -27,12 +27,12 @@ This will load the information distributed across several machines
 into the interactive node.
 
 Once you have a [`PT`](@ref) struct, proceed in the same way as 
-when running PT locally, e.g. [see the page on plotting](output-plotting.html), 
-[the page on online statistics](output-online.html), 
-and [the page on sample summaries and diagnostics](summaries.html).
+when running PT locally, e.g. [see the page on plotting](@ref output-plotting), 
+[the page on online statistics](@ref output-online), 
+and [the page on sample summaries and diagnostics](@ref output-numerical).
 
 For example, here is how to modify the posterior density and trace plot 
-example from [the plotting page](output-plotting.html) to run as a local MPI job 
+example from [the plotting page](@ref output-plotting) to run as a local MPI job 
 instead of in-process (the lines differing from the local version are marked 
 with (*)):
 
@@ -70,7 +70,7 @@ nothing # hide
 ```
 
 ```@raw html
-<iframe src="mpi_posterior_densities_and_traces.html" style="height:500px;width:100%;"></iframe>
+<iframe src="../mpi_posterior_densities_and_traces.html" style="height:500px;width:100%;"></iframe>
 ```
 
 # Perform post-processing by loading samples from disk one at a time
@@ -113,5 +113,5 @@ nothing # hide
 ```
 
 ```@raw html
-<iframe src="first_dim_of_each.html" style="height:500px;width:100%;"></iframe>
+<iframe src="../first_dim_of_each.html" style="height:500px;width:100%;"></iframe>
 ```

docs/src/output-normalization.md

@@ -2,7 +2,7 @@
 CurrentModule = Pigeons
 ```
 
-# Approximation of the normalization constant
+# [Approximation of the normalization constant](@id output-normalization)
 
 ## Background
 
@@ -22,7 +22,7 @@ In many applications, it is useful to approximate the constant ``Z``. For  examp
 As a side-product of parallel tempering, we automatically obtain an approximate the natural logarithm of the normalization constant ``\log Z``. This is done automatically using the 
 [stepping stone estimator](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3038348/) computed in [`stepping_stone()`](@ref). 
 
-It is shown in the [standard output report](output-report.html) produced at each round:
+It is shown in the [standard output report](@ref output-reports) produced at each round:
 
 ```@example constants
 using Pigeons
@@ -48,7 +48,7 @@ log of the *ratio*, ``\log (Z_1/ Z_0)`` where ``Z_1`` and ``Z_0`` are the normal
 
 Hence to estimate ``\log Z_1`` the reference distribution ``\pi_1`` should have a known normalization constant. In cases where the reference is a proper prior distribution, for example in Turing.jl models, this is typically the case. 
 
-In scenarios where the reference is specified manually, e.g. for black-box functions or Stan models, more care is needed. In such cases, one alternative is to use [variational PT](`variational.html`) in which case the built-in variational distribution is constructed so that its normalization constant is one. 
+In scenarios where the reference is specified manually, e.g. for black-box functions or Stan models, more care is needed. In such cases, one alternative is to use [variational PT](@ref variational-pt) in which case the built-in variational distribution is constructed so that its normalization constant is one. 
 
 !!! note "Normalization of Stan models"
 

docs/src/output-numerical.md

@@ -2,7 +2,7 @@
 CurrentModule = Pigeons
 ```
 
-# Numerical outputs and diagnostics
+# [Numerical outputs and diagnostics](@id output-numerical)
 
 Use [`sample_array()`](@ref) to convert target chain 
 samples into a format that can then be consumed by the 
@@ -44,7 +44,7 @@ samples
 ## Accessing individual diagnostics and summaries
 
 Computing a mean 
-(but see [online statistics](output-online.html) for 
+(but see [online statistics](@ref output-online) for 
 a constant memory alternative):
 
 ```example numerical

docs/src/output-off-memory.md

@@ -2,20 +2,20 @@
 CurrentModule = Pigeons
 ```
 
-# Off-memory processing
+# [Off-memory processing](@id output-off-memory)
 
 When the dimensionality of a model is large and/or the 
 number of MCMC samples is large, the samples may not 
 fit in memory. 
 In some situation, it may be possible to compute the 
 output in finite memory, as described in 
-[the online statistics documentation page](output-online.html). 
+[the online statistics documentation page](@ref output-online). 
 However not all situations admit sufficient statistics and 
 in this case it is necessary to store samples to disk. 
 We show here how to do so when pigeons is ran on a single 
 machine, but the interface is similar over MPI and 
 described in the 
-[MPI sample processing documentation page](output-mpi-postprocessing.html). 
+[MPI sample processing documentation page](@ref output-mpi-postprocessing). 
 
 
 ## Prepare the PT run with the disk recorder