How to Combine Special Bins with Regular Bins in OptBinning

[priyankamishra31]

Hi,

Thank you for developing and maintaining this amazing library.

I have a question about handling special bins and merging them with regular bins after the optimization process. In my scenario I want to merge the 'Special_Value' bins with one of the regular bins.

Example:
I am working with a numerical feature Income with the following details:

Special values: [-999, -1] (representing missing or erroneous data).
Target: Binary classification (e.g., 0 = no default, 1 = default).

After fitting the data using the OptimalBinning class, I obtained the following binning table:

Bin	Count	Event Rate	WoE	IV
[-999, -1]	2	0.50	0.20	0.01
[5000, 7000)	1	0.00	-0.50	0.02
[7000, 15000)	2	0.50	0.10	0.05
[15000, 30000)	1	1.00	0.70	0.07

My goal is to merge the special bin ([-999, -1]) with the second bin ([7000, 15000)) to create the following updated binning table:

Bin	Count	Event Rate	WoE	IV
[5000, 7000)	1	0.00	-0.50	0.02
[-999, -1, 7000, 15000)	4	0.375	0.05	0.15
[15000, 30000)	1	1.00	0.70	0.07

• Is there a way to achieve this directly using the OptBinning library's functionality?

• If not, what is the recommended workflow or best practice for post-processing the binning table while ensuring the consistency of metrics like Event Rate and WoE?

[guillermo-navas-palencia]

Hi @priyankamishra31.

Not ideal, but there is a way to that with OptBinning:

1. Fit OptimalBinning
2. Transform to "bins". Then you have a categorical variable as a result.
3. Perform OptimalBinning again using the new categorical variable as x, passing user_splits and user_splits_fixed, and removing any constraint, e.g., monotonic_trend

I hope that helps