Skip to content

Commit a551884

Browse files
authored
[MRG] DOC Add dropdowns to Module 1.13 Feature Selection (scikit-learn#26662)
1 parent a5620f4 commit a551884

File tree

1 file changed

+38
-27
lines changed

1 file changed

+38
-27
lines changed

doc/modules/feature_selection.rst

Lines changed: 38 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -201,31 +201,36 @@ alpha parameter, the fewer features selected.
201201

202202
.. _compressive_sensing:
203203

204-
.. topic:: **L1-recovery and compressive sensing**
205-
206-
For a good choice of alpha, the :ref:`lasso` can fully recover the
207-
exact set of non-zero variables using only few observations, provided
208-
certain specific conditions are met. In particular, the number of
209-
samples should be "sufficiently large", or L1 models will perform at
210-
random, where "sufficiently large" depends on the number of non-zero
211-
coefficients, the logarithm of the number of features, the amount of
212-
noise, the smallest absolute value of non-zero coefficients, and the
213-
structure of the design matrix X. In addition, the design matrix must
214-
display certain specific properties, such as not being too correlated.
215-
216-
There is no general rule to select an alpha parameter for recovery of
217-
non-zero coefficients. It can by set by cross-validation
218-
(:class:`~sklearn.linear_model.LassoCV` or
219-
:class:`~sklearn.linear_model.LassoLarsCV`), though this may lead to
220-
under-penalized models: including a small number of non-relevant variables
221-
is not detrimental to prediction score. BIC
222-
(:class:`~sklearn.linear_model.LassoLarsIC`) tends, on the opposite, to set
223-
high values of alpha.
224-
225-
**Reference** Richard G. Baraniuk "Compressive Sensing", IEEE Signal
204+
|details-start|
205+
**L1-recovery and compressive sensing**
206+
|details-split|
207+
208+
For a good choice of alpha, the :ref:`lasso` can fully recover the
209+
exact set of non-zero variables using only few observations, provided
210+
certain specific conditions are met. In particular, the number of
211+
samples should be "sufficiently large", or L1 models will perform at
212+
random, where "sufficiently large" depends on the number of non-zero
213+
coefficients, the logarithm of the number of features, the amount of
214+
noise, the smallest absolute value of non-zero coefficients, and the
215+
structure of the design matrix X. In addition, the design matrix must
216+
display certain specific properties, such as not being too correlated.
217+
218+
There is no general rule to select an alpha parameter for recovery of
219+
non-zero coefficients. It can by set by cross-validation
220+
(:class:`~sklearn.linear_model.LassoCV` or
221+
:class:`~sklearn.linear_model.LassoLarsCV`), though this may lead to
222+
under-penalized models: including a small number of non-relevant variables
223+
is not detrimental to prediction score. BIC
224+
(:class:`~sklearn.linear_model.LassoLarsIC`) tends, on the opposite, to set
225+
high values of alpha.
226+
227+
.. topic:: Reference
228+
229+
Richard G. Baraniuk "Compressive Sensing", IEEE Signal
226230
Processing Magazine [120] July 2007
227231
http://users.isr.ist.utl.pt/~aguiar/CS_notes.pdf
228232

233+
|details-end|
229234

230235
Tree-based feature selection
231236
----------------------------
@@ -282,6 +287,10 @@ instead of starting with no features and greedily adding features, we start
282287
with *all* the features and greedily *remove* features from the set. The
283288
`direction` parameter controls whether forward or backward SFS is used.
284289

290+
|details-start|
291+
**Detail on Sequential Feature Selection**
292+
|details-split|
293+
285294
In general, forward and backward selection do not yield equivalent results.
286295
Also, one may be much faster than the other depending on the requested number
287296
of selected features: if we have 10 features and ask for 7 selected features,
@@ -299,16 +308,18 @@ cross-validation requires fitting `m * k` models, while
299308
:class:`~sklearn.feature_selection.SelectFromModel` always just does a single
300309
fit and requires no iterations.
301310

302-
.. topic:: Examples
303-
304-
* :ref:`sphx_glr_auto_examples_feature_selection_plot_select_from_model_diabetes.py`
305-
306-
.. topic:: References:
311+
.. topic:: Reference
307312

308313
.. [sfs] Ferri et al, `Comparative study of techniques for
309314
large-scale feature selection
310315
<https://citeseerx.ist.psu.edu/doc_view/pid/5fedabbb3957bbb442802e012d829ee0629a01b6>`_.
311316
317+
|details-end|
318+
319+
.. topic:: Examples
320+
321+
* :ref:`sphx_glr_auto_examples_feature_selection_plot_select_from_model_diabetes.py`
322+
312323
Feature selection as part of a pipeline
313324
=======================================
314325

0 commit comments

Comments
 (0)