Monotone constraints in vector-leaf models
Both monotone constraints and vector-leaf models can act as regularizers by limiting the allowed splits.
- Monotone constraints allow embedding domain knowledge into the model and prevent noise-derived splits.
- Vector Leaf models prefer splits that are good for many targets also preventing noise-derived splits.
However using both at the same time is sometimes impossible with the current API.
Consider the case where a feature has positive effect for target 1 and negative effect for target 2. This is a very common scenario, for example with multi-class classifiers. Because we can only supply a single monotone_constraints vector we can't describe these contradicting effects to the model. It is however trivial when training multiple trees (pass different monotone_constraints for each train).
Is there any planned work on making these two features work in tandem?
It would likely require receiving an vector of monotone_constraints vectors, one for each target (or equivalent dictionary format).
I don't know how easy would be to adapt the split selection algorithm to comply with those constraints.
Issue created from discussion #12505
Source: dmlc/xgboost