Skip to content

Commit 132f30a

Browse files
yuta-nakaharayuta-nakahara
committed
Remove unnecessary files
1 parent d24d01a commit 132f30a

File tree

6 files changed

+144
-1774
lines changed

6 files changed

+144
-1774
lines changed

‎bayesml/metatree/__init__.py

Lines changed: 128 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,131 @@
1+
# Document Author
2+
# Shota Saito <shota.s@gunma-u.ac.jp>
3+
# Yuta Nakahara <yuta.nakahara@aoni.waseda.jp>
4+
r"""
5+
The stochastic data generative model is as follows:
6+
7+
* :math:`\mathcal{X}` : a space of an explanatory variable (a finite set)
8+
* :math:`\boldsymbol{x}=[x_1, \ldots, x_d] \in \mathcal{X}^d` : an explanatory variable
9+
* :math:`\mathcal{Y}` : a space of an objective variable
10+
* :math:`y \in \mathcal{Y}` : an objective variable
11+
* :math:`D_\mathrm{max} \in \mathbb{N}` : the maximum depth of trees
12+
* :math:`T` : :math:`|\mathcal{X}|`-ary regular tree whose depth is smaller than or equal to :math:`D_\mathrm{max}`, where "regular" means that all inner nodes have :math:`k` child nodes.
13+
* :math:`\mathcal{T}` : a set of :math:`T`
14+
* :math:`s` : a node of a tree
15+
* :math:`\mathcal{S}` : a set of :math:`s`
16+
* :math:`\mathcal{I}(T)` : a set of inner nodes of :math:`T`
17+
* :math:`\mathcal{L}(T)` : a set of leaf nodes of :math:`T`
18+
* :math:`\boldsymbol{k}=(k_s)_{s \in \mathcal{S}}` : feature assign vector where :math:`k_s \in \{1,2,\ldots,d\}`
19+
* :math:`\boldsymbol{\theta}=(\theta_s)_{s \in \mathcal{S}}` : a set of parameter
20+
* :math:`s(\boldsymbol{x}) \in \mathcal{L}(T)` : a leaf node of :math:`T` corresponding to :math:`\boldsymbol{x}`
21+
22+
.. math::
23+
p(y | \boldsymbol{x}, \boldsymbol{\theta}, T, \boldsymbol{k})=p(y | \theta_{s(\boldsymbol{x})})
24+
25+
The prior distribution is as follows:
26+
27+
* :math:`g_{0,s} \in [0,1]` : a hyperparameter assigned to :math:`s \in \mathcal{S}`
28+
* :math:`M_{T, \boldsymbol{k}}` : a meta-tree for :math:`(T, \boldsymbol{k})`
29+
* :math:`\mathcal{T}_{M_{T, \boldsymbol{k}}}` : a set of :math:`T` represented by a meta-tree :math:`M_{T, \boldsymbol{k}}`
30+
* :math:`B \in \mathbb{N}` : the number of meta-trees
31+
* :math:`\mathcal{M}=\{(T_1, \boldsymbol{k}_1), (T_2, \boldsymbol{k}_2), \ldots, (T_B, \boldsymbol{k}_B) \}` for :math:`B` meta-trees :math:`M_{T_1, \boldsymbol{k}_1}, M_{T_2, \boldsymbol{k}_2}, \dots, M_{T_B, \boldsymbol{k}_B}`. (These meta-trees must be given beforehand by some method, e.g., constructed from bootstrap samples similar to the random forest.)
32+
33+
For :math:`T' \in M_{T, \boldsymbol{k}}`,
34+
35+
.. math::
36+
p(T')=\prod_{s \in \mathcal{I}(T')} g_{0,s} \prod_{s' \in \mathcal{L}(T')} (1-g_{0,s'}),
37+
38+
where :math:`g_{0,s}=0` for a leaf node :math:`s` of a meta-tree :math:`M_{T, \boldsymbol{k}}`.
39+
40+
For :math:`\boldsymbol{k}_b \in \{\boldsymbol{k}_1, \boldsymbol{k}_2, \ldots, \boldsymbol{k}_B \}`,
41+
42+
.. math::
43+
p(\boldsymbol{k}_b) = \frac{1}{B}.
44+
45+
The posterior distribution is as follows:
46+
47+
* :math:`n \in \mathbb{N}` : a sample size
48+
* :math:`\boldsymbol{x}^n = \{ \boldsymbol{x}_1, \boldsymbol{x}_2, \ldots, \boldsymbol{x}_n \}`
49+
* :math:`y^n = \{ y_1, y_2, \ldots, y_n \}`
50+
* :math:`g_{n,s} \in [0,1]` : a hyperparameter
51+
52+
For :math:`T' \in M_{T, \boldsymbol{k}}`,
53+
54+
.. math::
55+
p(T' | \boldsymbol{x}^n, y^n, \boldsymbol{k})=\prod_{s \in \mathcal{I}(T')} g_{n,s} \prod_{s' \in \mathcal{L}(T')} (1-g_{n,s'}),
56+
57+
where the updating rules of the hyperparameter are as follows.
58+
59+
.. math::
60+
g_{i,s} =
61+
\begin{cases}
62+
g_{0,s} & (i = 0),\\
63+
\frac{g_{i-1,s} \tilde{q}_{s_{\mathrm{child}}}(y_i | \boldsymbol{x}_i, \boldsymbol{x}^{i-1}, y^{i-1}, M_{T, \boldsymbol{k}})}{\tilde{q}_s(y_i | \boldsymbol{x}_i, \boldsymbol{x}^{i-1}, y^{i-1}, M_{T, \boldsymbol{k}})} &(\mathrm{otherwise}),
64+
\end{cases}
65+
66+
where :math:`s_{\mathrm{child}}` is the child node of :math:`s` on the path corresponding to :math:`\boldsymbol{x}_{i}` in :math:`M_{T, \boldsymbol{k}}` and
67+
68+
.. math::
69+
&\tilde{q}_s(y_{i} | \boldsymbol{x}_{i}, \boldsymbol{x}^{i-1}, y^{i-1}, M_{T, \boldsymbol{k}}) \\
70+
&= \begin{cases}
71+
q_s(y_{i} | \boldsymbol{x}_{i}, \boldsymbol{x}^{i-1}, y^{i-1}, \boldsymbol{k}),& (s \ {\rm is \ the \ leaf \ node \ of} \ M_{T, \boldsymbol{k}}),\\
72+
(1-g_{i-1,s}) q_s(y_{i} | \boldsymbol{x}_{i}, \boldsymbol{x}^{i-1}, y^{i-1}, \boldsymbol{k}) \\
73+
\qquad + g_{i-1,s} \tilde{q}_{s_{\mathrm{child}}}(y_{i} | \boldsymbol{x}_{i}, \boldsymbol{x}^{i-1}, y^{i-1}, M_{T, \boldsymbol{k}}),& ({\rm otherwise}),
74+
\end{cases}
75+
76+
.. math::
77+
q_s(y_{i} | \boldsymbol{x}_{i}, \boldsymbol{x}^{i-1}, y^{i-1}, \boldsymbol{k})=\int p(y_i | \boldsymbol{x}_i, \boldsymbol{\theta}, T, \boldsymbol{k}) p(\boldsymbol{\theta} | \boldsymbol{x}^{i-1}, y^{i-1}, T, \boldsymbol{k}) \mathrm{d} \boldsymbol{\theta}, \quad s \in \mathcal{L}(T)
78+
79+
For :math:`\boldsymbol{k}_b \in \{\boldsymbol{k}_1, \boldsymbol{k}_2, \ldots, \boldsymbol{k}_B \}`,
80+
81+
.. math::
82+
p(\boldsymbol{k}_b | \boldsymbol{x}^n, y^n)\propto \prod_{i=1}^n \tilde{q}_{s_{\lambda}}(y_{i}|\boldsymbol{x}_{i},\boldsymbol{x}^{i-1}, y^{i-1}, M_{T_b, \boldsymbol{k}_b}),
83+
84+
where :math:`s_{\lambda}` is the root node of :math:`M_{T, \boldsymbol{k}_b}`.
85+
86+
The predictive distribution is as follows:
87+
88+
.. math::
89+
p(y_{n+1}| \boldsymbol{x}_{n+1}, \boldsymbol{x}^n, y^n) = \sum_{b = 1}^B p(\boldsymbol{k}_b | \boldsymbol{x}^n, y^n) \tilde{q}_{s_{\lambda}}(y_{n+1}|\boldsymbol{x}_{n+1},\boldsymbol{x}^n, y^n, M_{T_b, \boldsymbol{k}_b})
90+
91+
The expectation of the predictive distribution can be calculated as follows.
92+
93+
.. math::
94+
\mathbb{E}_{p(y_{n+1}| \boldsymbol{x}_{n+1}, \boldsymbol{x}^n, y^n)} [Y_{n+1}| \boldsymbol{x}_{n+1}, \boldsymbol{x}^n, y^n] = \sum_{b = 1}^B p(\boldsymbol{k}_b | \boldsymbol{x}^n, y^n) \mathbb{E}_{\tilde{q}_{s_{\lambda}}(y_{n+1}|\boldsymbol{x}_{n+1},\boldsymbol{x}^n, y^n, M_{T_b, \boldsymbol{k}_b})} [Y_{n+1}| \boldsymbol{x}_{n+1}, \boldsymbol{x}^n, y^n, \boldsymbol{k}_b],
95+
96+
where the expectation for :math:`\tilde{q}` is recursively given as follows.
97+
98+
.. math::
99+
&\mathbb{E}_{\tilde{q}_s(y_{n+1} | \boldsymbol{x}_{n+1}, \boldsymbol{x}^n, y^n, M_{T_b, \boldsymbol{k}_b})} [Y_{n+1} | \boldsymbol{x}_{n+1}, \boldsymbol{x}^n, y^n, \boldsymbol{k}_b] \\
100+
&= \begin{cases}
101+
\mathbb{E}_{q_s(y_{n+1} | \boldsymbol{x}_{n+1}, \boldsymbol{x}^n, y^n, \boldsymbol{k}_b)} [Y_{n+1} | \boldsymbol{x}_{n+1}, \boldsymbol{x}^n, y^n, \boldsymbol{k}_b],& (s \ {\rm is \ the \ leaf \ node \ of} \ M_{T_b, \boldsymbol{k}_b}),\\
102+
(1-g_{n,s}) \mathbb{E}_{q_s(y_{n+1} | \boldsymbol{x}_{n+1}, \boldsymbol{x}^n, y^n, \boldsymbol{k}_b)} [Y_{n+1} | \boldsymbol{x}_{n+1}, \boldsymbol{x}^n, y^n, \boldsymbol{k}_b] \\
103+
\qquad + g_{n,s} \mathbb{E}_{\tilde{q}_{s_{\mathrm{child}}}(y_{n+1} | \boldsymbol{x}_{n+1}, \boldsymbol{x}^n, y^n, M_{T_b, \boldsymbol{k}_b})} [Y_{n+1} | \boldsymbol{x}_{n+1}, \boldsymbol{x}^n, y^n, \boldsymbol{k}_b] ,& ({\rm otherwise}).
104+
\end{cases}
105+
106+
The maximum value of the predictive distribution can be calculated as follows.
107+
108+
.. math::
109+
\max_{y_{n+1}} p(y_{n+1}| \boldsymbol{x}_{n+1}, \boldsymbol{x}^n, y^n) = \max_{b = 1, \dots , B} \left\{ p(\boldsymbol{k}_b | \boldsymbol{x}^n, y^n) \max_{y_{n+1}} \tilde{q}_{s_{\lambda}}(y_{n+1}|\boldsymbol{x}_{n+1},\boldsymbol{x}^n, y^n, M_{T_b, \boldsymbol{k}_b}) \right\},
110+
111+
where the maximum value of :math:`\tilde{q}` is recursively given as follows.
112+
113+
.. math::
114+
&\max_{y_{n+1}} \tilde{q}_s(y_{n+1} | \boldsymbol{x}_{n+1}, \boldsymbol{x}^n, y^n, M_{T_b, \boldsymbol{k}_b}) \\
115+
&= \begin{cases}
116+
\max_{y_{n+1}} q_s(y_{n+1} | \boldsymbol{x}_{n+1}, \boldsymbol{x}^n, y^n, \boldsymbol{k}_b),& (s \ {\rm is \ the \ leaf \ node \ of} \ M_{T_b, \boldsymbol{k}_b}),\\
117+
\max \{ (1-g_{n,s}) \max_{y_{n+1}} q_s(y_{n+1} | \boldsymbol{x}_{n+1}, \boldsymbol{x}^n, y^n, \boldsymbol{k}_b), \\
118+
\qquad \qquad g_{n,s} \max_{y_{n+1}} \tilde{q}_{s_{\mathrm{child}}}(y_{n+1} | \boldsymbol{x}_{n+1}, \boldsymbol{x}^n, y^n, M_{T_b, \boldsymbol{k}_b}) \} ,& ({\rm otherwise}).
119+
\end{cases}
120+
121+
The mode of the predictive distribution can be also calculated by using the above equation.
122+
123+
References
124+
125+
* Dobashi, N.; Saito, S.; Nakahara, Y.; Matsushima, T. Meta-Tree Random Forest: Probabilistic Data-Generative Model and Bayes Optimal Prediction. *Entropy* 2021, 23, 768. https://doi.org/10.3390/e23060768
126+
* Nakahara, Y.; Saito, S.; Kamatsuka, A.; Matsushima, T. Probability Distribution on Full Rooted Trees. *Entropy* 2022, 24, 328. https://doi.org/10.3390/e24030328
127+
"""
1128
from ._metatree_x_discrete import GenModel
2129
from ._metatree_x_discrete import LearnModel
3-
from ._metatree_x_discrete import _LearnNode
4130

5-
__all__ = ["GenModel", "LearnModel","_LearnNode"]
131+
__all__ = ["GenModel", "LearnModel"]

0 commit comments

Comments
 (0)