Merge pull request #5 from yuta-nakahara/develop

Develop web pages

Author: yuta-nakahara

README.md

@@ -50,12 +50,12 @@ You can visualize the characteristics of the created model by the following meth
 gen_model.visualize_model()
 ```
 
->p:0.7  
->x0:[1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 0]  
->x1:[1 0 1 0 1 0 0 0 0 1 1 1 1 1 1 1 1 0 1 1]  
->x2:[1 0 1 0 1 1 1 1 1 1 1 0 0 0 0 1 1 0 1 1]  
->x3:[0 1 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 1 0]  
->x4:[1 1 1 0 1 1 0 1 1 1 1 1 1 1 1 1 0 1 1 0]  
+>theta:0.7  
+>x0:[1 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 0 1 1 1]  
+>x1:[1 0 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1]  
+>x2:[0 0 1 1 0 1 0 1 1 1 1 1 1 0 1 0 1 1 1 1]  
+>x3:[1 0 1 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0]  
+>x4:[1 1 0 1 0 1 1 1 0 1 1 1 0 0 1 1 1 1 1 1]  
 >![bernoulli_example1](./doc/images/README_ex_img1.png)
 
 After confirming that the frequency of occurrence of 1 is around `theta=0.7`, we generate a sample and store it to variable `x`.
@@ -108,6 +108,7 @@ Different settings of a loss function yield different optimal estimates.
 The following packages are currently available. In this library, a probabilistic data generative model, prior distribution, posterior distribution (or approximate posterior distribution), and predictive distribution (or approximate predictive distribution) are collectively called a model.
 
 * Bernoulli model
+* Categorical model
 * Poisson model
 * Normal model
 * Multivariate normal model

README_jp.md

@@ -50,11 +50,11 @@ gen_model.visualize_model()
 ```
 
 >theta:0.7  
->x0:[1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 0]  
->x1:[1 0 1 0 1 0 0 0 0 1 1 1 1 1 1 1 1 0 1 1]  
->x2:[1 0 1 0 1 1 1 1 1 1 1 0 0 0 0 1 1 0 1 1]  
->x3:[0 1 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 1 0]  
->x4:[1 1 1 0 1 1 0 1 1 1 1 1 1 1 1 1 0 1 1 0]  
+>x0:[1 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 0 1 1 1]  
+>x1:[1 0 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1]  
+>x2:[0 0 1 1 0 1 0 1 1 1 1 1 1 0 1 0 1 1 1 1]  
+>x3:[1 0 1 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0]  
+>x4:[1 1 0 1 0 1 1 1 0 1 1 1 0 0 1 1 1 1 1 1]  
 >![bernoulli_example1](./doc/images/README_ex_img1.png)
 
 1の出現頻度が`theta=0.7`程度であることを確認したら，サンプルを生成し変数`x`に保存します．
@@ -105,6 +105,7 @@ print(learn_model.estimate_params(loss='0-1'))
 現在，以下のモデルに関するパッケージが利用可能です．本ライブラリでは，データ生成確率モデル，事前分布，事後分布（または近似事後分布），予測分布（または近似予測分布）を合わせてモデルと呼んでいます．
 
 * ベルヌーイモデル
+* カテゴリカルモデル
 * ポアソンモデル
 * 正規モデル
 * 多変量正規モデル

bayesml/autoregressive/__init__.py

@@ -1,5 +1,6 @@
 # Document Author
 # Yuta Nakahara <yuta.nakahara@aoni.waseda.jp>
+# Koki Kazama <kokikazama@aoni.waseda.jp>
 r"""
 The linear autoregressive model with the normal-gamma prior distribution.
 
@@ -12,49 +13,66 @@
 * :math:`\boldsymbol{\theta} \in \mathbb{R}^{d+1}`: a regression coefficient parameter
 * :math:`\tau \in \mathbb{R}_{>0}`: a precision parameter of noise
 
-.. math:: 
-    \mathcal{N}(x_n|\boldsymbol{\theta}^\top \boldsymbol{x}'_{n-1}, \tau^{-1})
-    = \sqrt{\frac{\tau}{2 \pi}} \exp \left\{ -\frac{\tau}{2} (x_n - \boldsymbol{\theta}^\top \boldsymbol{x}'_{n-1})^2 \right\}.
+.. math::
+    p(x_n | \boldsymbol{x}'_{n-1}, \boldsymbol{\theta}, \tau) &= \mathcal{N}(x_n|\boldsymbol{\theta}^\top \boldsymbol{x}'_{n-1}, \tau^{-1}) \\
+    &= \sqrt{\frac{\tau}{2 \pi}} \exp \left\{ -\frac{\tau}{2} (x_n - \boldsymbol{\theta}^\top \boldsymbol{x}'_{n-1})^2 \right\}.
+
+.. math::
+    &\mathbb{E}[ x_n | \boldsymbol{x}'_{n-1},\boldsymbol{\theta},\tau] = \boldsymbol{\theta}^{\top} \boldsymbol{x}'_{n-1}, \\
+    &\mathbb{V}[ x_n | \boldsymbol{x}'_{n-1},\boldsymbol{\theta},\tau ] = \tau^{-1}.
+
 
 The prior distribution is as follows:
 
 * :math:`\boldsymbol{\mu}_0 \in \mathbb{R}^{d+1}`: a hyperparameter for :math:`\boldsymbol{\theta}`
 * :math:`\boldsymbol{\Lambda}_0 \in \mathbb{R}^{(d+1) \times (d+1)}`: a hyperparameter for :math:`\boldsymbol{\theta}` (a positive definite matrix)
 * :math:`| \boldsymbol{\Lambda}_0 | \in \mathbb{R}`: the determinant of :math:`\boldsymbol{\Lambda}_0`
-* :math:`a_0 \in \mathbb{R}_{>0}`: a hyperparameter for :math:`\tau`
-* :math:`b_0 \in \mathbb{R}_{>0}`: a hyperparameter for :math:`\tau`
+* :math:`\alpha_0 \in \mathbb{R}_{>0}`: a hyperparameter for :math:`\tau`
+* :math:`\beta_0 \in \mathbb{R}_{>0}`: a hyperparameter for :math:`\tau`
 * :math:`\Gamma(\cdot): \mathbb{R}_{>0} \to \mathbb{R}`: the Gamma function
 
 .. math::
-    &\mathcal{N}(\boldsymbol{\theta}|\boldsymbol{\mu}_0, (\tau \boldsymbol{\Lambda}_0)^{-1}) \text{Gam}(\tau|a_0,b_0)\\
+    p(\boldsymbol{\theta}, \tau) &= \mathcal{N}(\boldsymbol{\theta}|\boldsymbol{\mu}_0, (\tau \boldsymbol{\Lambda}_0)^{-1}) \mathrm{Gam}(\tau|\alpha_0,\beta_0)\\
     &= \frac{|\tau \boldsymbol{\Lambda}_0|^{1/2}}{(2 \pi)^{(d+1)/2}} 
     \exp \left\{ -\frac{\tau}{2} (\boldsymbol{\theta} - \boldsymbol{\mu}_0)^\top 
     \boldsymbol{\Lambda}_0 (\boldsymbol{\theta} - \boldsymbol{\mu}_0) \right\}
-    \frac{b_0^{a_0}}{\Gamma (a_0)} \tau^{a_0 - 1} \exp \{ -b_0 \tau \} .
+    \frac{\beta_0^{\alpha_0}}{\Gamma (\alpha_0)} \tau^{\alpha_0 - 1} \exp \{ -\beta_0 \tau \} .
+
+.. math::
+    \mathbb{E}[\boldsymbol{\theta}] &= \boldsymbol{\mu}_0 & \left( \alpha_0 > \frac{1}{2} \right), \\
+    \mathrm{Cov}[\boldsymbol{\theta}] &= \frac{\beta_0}{\alpha_0 - 1} \boldsymbol{\Lambda}_0^{-1} & (\alpha_0 > 1), \\
+    \mathbb{E}[\tau] &= \frac{\alpha_0}{\beta_0}, \\
+    \mathbb{V}[\tau] &= \frac{\alpha_0}{\beta_0^2}.
 
 The posterior distribution is as follows:
 
 * :math:`x^n := [x_1, x_2, \dots , x_n]^\top \in \mathbb{R}^n`: given data
 * :math:`\boldsymbol{X}_n = [\boldsymbol{x}'_1, \boldsymbol{x}'_2, \dots , \boldsymbol{x}'_n]^\top \in \mathbb{R}^{n \times (d+1)}`
 * :math:`\boldsymbol{\mu}_n \in \mathbb{R}^{d+1}`: a hyperparameter for :math:`\boldsymbol{\theta}`
 * :math:`\boldsymbol{\Lambda}_n \in \mathbb{R}^{(d+1) \times (d+1)}`: a hyperparameter for :math:`\boldsymbol{\theta}` (a positive definite matrix)
-* :math:`a_n \in \mathbb{R}_{>0}`: a hyperparameter for :math:`\tau`
-* :math:`b_n \in \mathbb{R}_{>0}`: a hyperparameter for :math:`\tau`
+* :math:`\alpha_n \in \mathbb{R}_{>0}`: a hyperparameter for :math:`\tau`
+* :math:`\beta_n \in \mathbb{R}_{>0}`: a hyperparameter for :math:`\tau`
 
 .. math::
-    &\mathcal{N}(\boldsymbol{\theta}|\boldsymbol{\mu}_n, (\tau \boldsymbol{\Lambda}_n)^{-1}) \text{Gam}(\tau|a_n,b_n)\\
+    p(\boldsymbol{\theta}, \tau | x^n) &= \mathcal{N}(\boldsymbol{\theta}|\boldsymbol{\mu}_n, (\tau \boldsymbol{\Lambda}_n)^{-1}) \mathrm{Gam}(\tau|\alpha_n,\beta_n)\\
     &= \frac{|\boldsymbol{\tau \Lambda}_n|^{1/2}}{(2 \pi)^{(d+1)/2}}
     \exp \left\{ -\frac{\tau}{2} (\boldsymbol{\theta} - \boldsymbol{\mu}_n)^\top 
     \boldsymbol{\Lambda}_n (\boldsymbol{\theta} - \boldsymbol{\mu}_n) \right\}
-    \frac{b_n^{a_n}}{\Gamma (a_n)} \tau^{a_n - 1} \exp \{ -b_n \tau \} .
+    \frac{\beta_n^{\alpha_n}}{\Gamma (\alpha_n)} \tau^{\alpha_n - 1} \exp \{ -\beta_n \tau \} .
+
+.. math::
+    \mathbb{E}[\boldsymbol{\theta} | x^n] &= \boldsymbol{\mu}_n & \left( \alpha_n > \frac{1}{2} \right), \\
+    \mathrm{Cov}[\boldsymbol{\theta} | x^n] &= \frac{\beta_n}{\alpha_n - 1} \boldsymbol{\Lambda}_n^{-1} & (\alpha_n > 1), \\
+    \mathbb{E}[\tau | x^n] &= \frac{\alpha_n}{\beta_n}, \\
+    \mathbb{V}[\tau | x^n] &= \frac{\alpha_n}{\beta_n^2},
 
 where the updating rules of the hyperparameters are
 
 .. math::
     \boldsymbol{\Lambda}_n &= \boldsymbol{\Lambda}_0 + \boldsymbol{X}_n^\top \boldsymbol{X}_n,\\
     \boldsymbol{\mu}_n &= \boldsymbol{\Lambda}_n^{-1} (\boldsymbol{\Lambda}_0 \boldsymbol{\mu}_0 + \boldsymbol{X}_n^\top x^n),\\
-    a_n &= a_0 + \frac{n}{2},\\
-    b_n &= b_0 + \frac{1}{2} \left( -\boldsymbol{\mu}_n^\top \boldsymbol{\Lambda}_n \boldsymbol{\mu}_n 
+    \alpha_n &= \alpha_0 + \frac{n}{2},\\
+    \beta_n &= \beta_0 + \frac{1}{2} \left( -\boldsymbol{\mu}_n^\top \boldsymbol{\Lambda}_n \boldsymbol{\mu}_n 
     + (x^n)^\top x^n + \boldsymbol{\mu}_0^\top \boldsymbol{\Lambda}_0 \boldsymbol{\mu}_0 \right).
 
 The predictive distribution is as follows:
@@ -65,17 +83,21 @@
 * :math:`\nu_\mathrm{p} \in \mathbb{R}_{>0}`: a parameter
 
 .. math::
-    \text{St}(x_{n+1}|m_\mathrm{p}, \lambda_\mathrm{p}, \nu_\mathrm{p})
+    \mathrm{St}(x_{n+1}|m_\mathrm{p}, \lambda_\mathrm{p}, \nu_\mathrm{p})
     = \frac{\Gamma (\nu_\mathrm{p}/2 + 1/2)}{\Gamma (\nu_\mathrm{p}/2)}
     \left( \frac{m_\mathrm{p}}{\pi \nu_\mathrm{p}} \right)^{1/2}
     \left[ 1 + \frac{\lambda_\mathrm{p}(x_{n+1}-m_\mathrm{p})^2}{\nu_\mathrm{p}} \right]^{-\nu_\mathrm{p}/2 - 1/2}.
 
+.. math::
+    \mathbb{E}[x_{n+1} | x^n] &= m_\mathrm{p} & (\nu_\mathrm{p} > 1), \\
+    \mathbb{V}[x_{n+1} | x^n] &= \frac{1}{\lambda_\mathrm{p}} \frac{\nu_\mathrm{p}}{\nu_\mathrm{p}-2} & (\nu_\mathrm{p} > 2),
+
 where the parameters are obtained from the hyperparameters of the posterior distribution as follows.
 
 .. math::
     m_\mathrm{p} &= \mu_n^\top \boldsymbol{x}'_n,\\
-    \lambda_\mathrm{p} &= \frac{a_n}{b_n} (1 + (\boldsymbol{x}'_n)^\top \boldsymbol{\Lambda}_n^{-1} \boldsymbol{x}'_n)^{-1},\\
-    \nu_\mathrm{p} &= 2 a_n.
+    \lambda_\mathrm{p} &= \frac{\alpha_n}{\beta_n} (1 + (\boldsymbol{x}'_n)^\top \boldsymbol{\Lambda}_n^{-1} \boldsymbol{x}'_n)^{-1},\\
+    \nu_\mathrm{p} &= 2 \alpha_n.
 """
 
 from ._autoregressive import GenModel

bayesml/bernoulli/__init__.py

@@ -1,54 +1,71 @@
 # Document Author
 # Yuta Nakahara <yuta.nakahara@aoni.waseda.jp>
+# Koki Kazama <kokikazama@aoni.waseda.jp>
 r"""
 The Bernoulli distribution with the beta prior distribution.
 
 The stochastic data generative model is as follows:
 
 * :math:`x \in \{ 0, 1\}`: a data point
-* :math:`p \in [0, 1]`: a parameter 
+* :math:`\theta \in [0, 1]`: a parameter
 
-.. math:: \text{Bern}(x|p) = p^x (1-p)^{1-x}.
+.. math::
+    p(x | \theta) = \mathrm{Bern}(x|\theta) = \theta^x (1-\theta)^{1-x}.
+
+.. math::
+    \mathbb{E}[x | \theta] &= \theta, \\
+    \mathbb{V}[x | \theta] &= \theta (1 - \theta).
 
 The prior distribution is as follows:
 
 * :math:`\alpha_0 \in \mathbb{R}_{>0}`: a hyperparameter
 * :math:`\beta_0 \in \mathbb{R}_{>0}`: a hyperparameter
 * :math:`B(\cdot,\cdot): \mathbb{R}_{>0} \times \mathbb{R}_{>0} \to \mathbb{R}_{>0}`: the Beta function
 
-.. math:: \text{Beta}(p|\alpha_0,\beta_0) = \frac{1}{B(\alpha_0, \beta_0)} p^{\alpha_0} (1-p)^{\beta_0}.
+.. math::
+    p(\theta) = \mathrm{Beta}(\theta|\alpha_0,\beta_0) = \frac{1}{B(\alpha_0, \beta_0)} \theta^{\alpha_0} (1-\theta)^{\beta_0}.
+
+.. math::
+    \mathbb{E}[\theta] &= \frac{\alpha_0}{\alpha_0 + \beta_0}, \\
+    \mathbb{V}[\theta] &= \frac{\alpha_0 \beta_0}{(\alpha_0 + \beta_0)^2 (\alpha_0 + \beta_0 + 1)}.
 
 The posterior distribution is as follows:
 
 * :math:`x^n = (x_1, x_2, \dots , x_n) \in \{ 0, 1\}^n`: given data
 * :math:`\alpha_n \in \mathbb{R}_{>0}`: a hyperparameter
 * :math:`\beta_n \in \mathbb{R}_{>0}`: a hyperparameter
 
-.. math:: \text{Beta}(p|\alpha_n,\beta_n) = \frac{1}{B(\alpha_n, \beta_n)} p^{\alpha_n} (1-p)^{\beta_n},
+.. math::
+    p(\theta | x^n) = \mathrm{Beta}(\theta|\alpha_n,\beta_n) = \frac{1}{B(\alpha_n, \beta_n)} \theta^{\alpha_n} (1-\theta)^{\beta_n},
+
+.. math::
+    \mathbb{E}[\theta | x^n] &= \frac{\alpha_n}{\alpha_n + \beta_n}, \\
+    \mathbb{V}[\theta | x^n] &= \frac{\alpha_n \beta_n}{(\alpha_n + \beta_n)^2 (\alpha_n + \beta_n + 1)}.
 
 where the updating rule of the hyperparameters is
 
 .. math::
     \alpha_n = \alpha_0 + \sum_{i=1}^n I \{ x_i = 1 \},\\
-    \beta_n = \beta_0 + \sum_{i=1}^n I \{ x_i = 0 \}.    
+    \beta_n = \beta_0 + \sum_{i=1}^n I \{ x_i = 0 \}.
 
 The predictive distribution is as follows:
 
-* :math:`x \in \{ 0, 1\}`: a new data point
+* :math:`x_{n+1} \in \{ 0, 1\}`: a new data point
 * :math:`\alpha_\mathrm{p} \in \mathbb{R}_{>0}`: a parameter
 * :math:`\beta_\mathrm{p} \in \mathbb{R}_{>0}`: a parameter
+* :math:`\theta_\mathrm{p} \in [0,1]`: a parameter
+
+.. math::
+    p(x_{n+1} | x^n) = \mathrm{Bern}(x_{n+1}|\theta_\mathrm{p}) =\theta_\mathrm{p}^{x_{n+1}}(1-\theta_\mathrm{p})^{1-x_{n+1}}
 
 .. math::
-    p(x|\alpha_\mathrm{p}, \beta_\mathrm{p}) = \begin{cases}
-    \frac{\alpha_\mathrm{p}}{\alpha_\mathrm{p} + \beta_\mathrm{p}} & x = 1,\\
-    \frac{\beta_\mathrm{p}}{\alpha_\mathrm{p} + \beta_\mathrm{p}} & x = 0,
-    \end{cases}
+    \mathbb{E}[x_{n+1} | x^n] &= \theta_\mathrm{p}, \\
+    \mathbb{V}[x_{n+1} | x^n] &= \theta_\mathrm{p} (1 - \theta_\mathrm{p}).
 
-where the parameters are abtained from the hyperparameters of the posterior distribution as follows.
+where the parameters are obtained from the hyperparameters of the posterior distribution as follows.
 
 .. math::
-    \alpha_\mathrm{p} &= \alpha_n,\\
-    \beta_\mathrm{p} &= \beta_n
+    \theta_\mathrm{p} = \frac{\alpha_n}{\alpha_n + \beta_n}
 """
 
 from ._bernoulli import GenModel