@@ -201,31 +201,36 @@ alpha parameter, the fewer features selected.
201
201
202
202
.. _compressive_sensing :
203
203
204
- .. topic :: **L1-recovery and compressive sensing**
205
-
206
- For a good choice of alpha, the :ref: `lasso ` can fully recover the
207
- exact set of non-zero variables using only few observations, provided
208
- certain specific conditions are met. In particular, the number of
209
- samples should be "sufficiently large", or L1 models will perform at
210
- random, where "sufficiently large" depends on the number of non-zero
211
- coefficients, the logarithm of the number of features, the amount of
212
- noise, the smallest absolute value of non-zero coefficients, and the
213
- structure of the design matrix X. In addition, the design matrix must
214
- display certain specific properties, such as not being too correlated.
215
-
216
- There is no general rule to select an alpha parameter for recovery of
217
- non-zero coefficients. It can by set by cross-validation
218
- (:class: `~sklearn.linear_model.LassoCV ` or
219
- :class: `~sklearn.linear_model.LassoLarsCV `), though this may lead to
220
- under-penalized models: including a small number of non-relevant variables
221
- is not detrimental to prediction score. BIC
222
- (:class: `~sklearn.linear_model.LassoLarsIC `) tends, on the opposite, to set
223
- high values of alpha.
224
-
225
- **Reference ** Richard G. Baraniuk "Compressive Sensing", IEEE Signal
204
+ |details-start |
205
+ **L1-recovery and compressive sensing **
206
+ |details-split |
207
+
208
+ For a good choice of alpha, the :ref: `lasso ` can fully recover the
209
+ exact set of non-zero variables using only few observations, provided
210
+ certain specific conditions are met. In particular, the number of
211
+ samples should be "sufficiently large", or L1 models will perform at
212
+ random, where "sufficiently large" depends on the number of non-zero
213
+ coefficients, the logarithm of the number of features, the amount of
214
+ noise, the smallest absolute value of non-zero coefficients, and the
215
+ structure of the design matrix X. In addition, the design matrix must
216
+ display certain specific properties, such as not being too correlated.
217
+
218
+ There is no general rule to select an alpha parameter for recovery of
219
+ non-zero coefficients. It can by set by cross-validation
220
+ (:class: `~sklearn.linear_model.LassoCV ` or
221
+ :class: `~sklearn.linear_model.LassoLarsCV `), though this may lead to
222
+ under-penalized models: including a small number of non-relevant variables
223
+ is not detrimental to prediction score. BIC
224
+ (:class: `~sklearn.linear_model.LassoLarsIC `) tends, on the opposite, to set
225
+ high values of alpha.
226
+
227
+ .. topic :: Reference
228
+
229
+ Richard G. Baraniuk "Compressive Sensing", IEEE Signal
226
230
Processing Magazine [120] July 2007
227
231
http://users.isr.ist.utl.pt/~aguiar/CS_notes.pdf
228
232
233
+ |details-end |
229
234
230
235
Tree-based feature selection
231
236
----------------------------
@@ -282,6 +287,10 @@ instead of starting with no features and greedily adding features, we start
282
287
with *all * the features and greedily *remove * features from the set. The
283
288
`direction ` parameter controls whether forward or backward SFS is used.
284
289
290
+ |details-start |
291
+ **Detail on Sequential Feature Selection **
292
+ |details-split |
293
+
285
294
In general, forward and backward selection do not yield equivalent results.
286
295
Also, one may be much faster than the other depending on the requested number
287
296
of selected features: if we have 10 features and ask for 7 selected features,
@@ -299,16 +308,18 @@ cross-validation requires fitting `m * k` models, while
299
308
:class: `~sklearn.feature_selection.SelectFromModel ` always just does a single
300
309
fit and requires no iterations.
301
310
302
- .. topic :: Examples
303
-
304
- * :ref: `sphx_glr_auto_examples_feature_selection_plot_select_from_model_diabetes.py `
305
-
306
- .. topic :: References:
311
+ .. topic :: Reference
307
312
308
313
.. [sfs ] Ferri et al, `Comparative study of techniques for
309
314
large-scale feature selection
310
315
<https://citeseerx.ist.psu.edu/doc_view/pid/5fedabbb3957bbb442802e012d829ee0629a01b6> `_.
311
316
317
+ |details-end |
318
+
319
+ .. topic :: Examples
320
+
321
+ * :ref: `sphx_glr_auto_examples_feature_selection_plot_select_from_model_diabetes.py `
322
+
312
323
Feature selection as part of a pipeline
313
324
=======================================
314
325
0 commit comments