Support for more `BinningProcess.binning_transform_params` parameters

[WonderfulAnastasia]

Hello!

I've run into some issues while converting OptBinning to PMML format. Specifically, I used the Binning Process class and specified binning_transform_params with metric_missing='empirical'. After inspecting the resulting binning table, I noticed that the Weight-of-Evidence (WoE) value for missing entries was non-zero (it was equal to -0.182322). However, when examining the exported PMML file, the attribute mapMissingTo was set to 0 (Discretize field="age" mapMissingTo="0.0"), which contradicts my treatment of missing values.
Could you please clarify if it's possible to configure mapMissingTo reflects the empirical WoE values for missing entries instead of being 0?

Thanks!

import pandas as pd
import numpy as np
from optbinning import BinningProcess
from sklearn2pmml import sklearn2pmml
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
df = pd.DataFrame({
    'age':[np.nan, 80, 54, 31, 32, 79, 43, np.nan, 22, 48, 62, 66, 33, 76, 60, 68, 47, 72, 20, 51, 44, 38, 25, 64, 63, 39, 52,
            65, 59, 53, 73, 78, 45, 27, 57, 21, 34, 24, 42, np.nan],
    'y':[1, 1, 0, 1, 0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 0,
       1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 1, 1,
       0, 0, 1, 1, 0, 0]
})
pipeline = Pipeline([
    ('binning',  BinningProcess(variable_names = ['age'], 
                                binning_transform_params = {
                                    'age' : {'metric' : 'woe',
                                             'metric_missing':'empirical'}
                                })),
    ("logistic_regression", LogisticRegression(random_state=42))
])
pipeline.fit(df[['age']], df['y'])

P.S. The same problem with metric_special

[vruusmann]

The Discretize@mapMissingTo value is currently set to 0 always:
https://github.com/jpmml/jpmml-sklearn/blob/1.9.7/pmml-sklearn-extension/src/main/java/optbinning/OptimalBinning.java#L177-L178

The existing codebase does not check for the metric_missing parameter of the OptimalBinning.binning_transform_params attribute (it checks for metric and special_codes parameters). By any chance, is the metric_missing parameter some newer OptBinning feature? I haven't followed its development lately.

The same problem with metric_special

What do you mean, is it also a OptimalBinning.binning_transform_params parameter, or something else?

[WonderfulAnastasia]

Thank you very much for such a quick response!

As far as I'm aware, this functionality has already been presented in version 0.10.0, so it isn't so new.

The metric_special = 'empirical' is indeed another OptimalBinning.binning_transform_params parameter. It's designed to transform special codes in the input vector, assigning them not just a fixed value of 0. The "empirical” value is derived from the results of the binning process itself: it treats the special codes as a distinct category and computes a unique WoE specifically for them.

I would greatly appreciate it if, in the future, it became possible to work with the values of metric_missing and metric_special

[vruusmann]

I would greatly appreciate it if, in the future, it became possible to work with the values of metric_missing and metric_special

Doesn't seem like much work to make them supported. Maybe your favourite LLM can suggest a quick code change that you can test out with your JPMML-SkLearn checkout.

I'm busy with other stuff for one more week or so. However, if you happen to represent a paying customer, then you should let me know, and perhaps I can give it a quicker turnaround then.

[WonderfulAnastasia]

Thank you