Skip to content

[Model] Sample weights do not work in mlf.cross_validation with refit=False #497

@slamao

Description

@slamao

Sample weights do not work in mlf.cross_validation with refit=False

Hi,

I found out that mlforecast's cross validation method does not work in one specific case: when using sample weights and when refit=False. I think that this is a correct setting and that the cross validation should work in this case as well (please correct me if I am wrong).

There is a reproducible example below:

from mlforecast.utils import generate_daily_series
from mlforecast import MLForecast
from sklearn.linear_model import ElasticNet

series = generate_daily_series(2)
series['weight'] = np.random.default_rng(seed=0).random(series.shape[0])
series['exog'] = np.random.default_rng(seed=0).random(series.shape[0])

fcst = MLForecast(
    models={
        'lr': ElasticNet(alpha=0.1)
    },
    freq='D',
    lags=[1],
    date_features=['dayofweek'],
)

# cross validation throws error
fcst.cross_validation(series, weight_col='weight', static_features=[], n_windows=2, h=7, refit=False)

Image

Similar error is also thrown when not using any exogenous variables.

Versions / Dependencies

python: 3.12
mlforecast: 1.0.1
pandas: 2.2.3
scikit-learn: 1.6.1

Reproduction script

from mlforecast.utils import generate_daily_series
from mlforecast import MLForecast
from sklearn.linear_model import ElasticNet

series = generate_daily_series(2)
series['weight'] = np.random.default_rng(seed=0).random(series.shape[0])
series['exog'] = np.random.default_rng(seed=0).random(series.shape[0])

fcst = MLForecast(
    models={
        'lr': ElasticNet(alpha=0.1)
    },
    freq='D',
    lags=[1],
    date_features=['dayofweek'],
)

# cross validation throws error
fcst.cross_validation(series, weight_col='weight', static_features=[], n_windows=2, h=7, refit=False)

Issue Severity

Medium: It is a significant difficulty but I can work around it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions