Plans for daal4py to support missing values in tree models

[tpboudreau]

When training data contains missing values, XGBoost will construct trees that direct missing valued observations down whichever path produces the greater gain. Whenever this results in missing values joining the "yes" (or, "less than the breakpoint for the feature") branch, daal4py refuses to load the resulting model for prediction.

For example, using the csv training dataset in the zip file below and running this script:

import numpy as np
import pandas as pd
import xgboost as xg
import daal4py as d4
from sklearn.metrics import mean_squared_error as mse

if __name__ == '__main__':

    df = pd.read_csv('./data_with_some_missing_values.csv',
                     names=['y', 'x0', 'x1', 'x2', 'x3', 'x4', 'x5', 'x6', 'x7', 'x8', 'x9'],
                     header=None,
                     dtype=np.float32)
    X = df.drop('y', axis=1)
    y = df['y']
    dt = xg.DMatrix(data=X, label=y)
    print("dt: {}x{}".format(dt.num_row(), dt.num_col()))

    pm = {
      'objective': 'reg:squarederror',
      'max_depth': 3,
      'tree_method': 'hist',
    }
    print(pm)

    md = xg.train(params=pm, dtrain=dt, num_boost_round=10, evals=[(dt, "dt")])
    md.dump_model('trained_with_some_missing_values.json', dump_format='json')

    dm = d4.get_gbt_model_from_xgboost(md)
    yh = d4.gbt_regression_prediction().compute(X, dm).prediction
    ye = np.sqrt(mse(y, yh))
    print(ye)

results in the following error:

Traceback (most recent call last):
File "predict_with_some_missing_values.py", line 29, in
dm = d4.get_gbt_model_from_xgboost(md)
File "build/daal4py_cy.pyx", line 21293, in _daal4py.get_gbt_model_from_xgboost
TypeError: Missing values are not supported in daal4py Gradient Boosting Trees

(The resulting tree model is also included in the zip file, showing that missing values occasionally join the "yes" observations)

d4p.zip

Since missing values are not uncommon in our data, this limitation reduces daal4py's usefulness in our environment.

Are there any plans for daal4py to support such models in the near future?

Thanks!

EDIT: I'm running daal4py 2021.2.3 (from conda-forge) on Linux, but I don't believe this is a platform specific issue.

[PivovarA]

Hi @tpboudreau
Thank you very much for pointing this out. Unfortunately, we had no plans to add this feature in the near future.
However, I would like to clarify how urgently you need support for such models?

[napetrov]

this was implemented some time ago - #1276