Usage of scalar-leaf trees with multi-output objectives
In https://github.com/dmlc/xgboost/issues/12278 , it was mentioned:
In one tree-per-class mode, one idea would be to recompute the gradients after each tree, instead of after each group of trees. This turns it into a coordinate descent algorithm which is much better at dealing with curvature.
Following this example: https://xgboost.readthedocs.io/en/stable/tutorials/advanced_custom_obj.html
If setting parameters num_boost_round=1 and base_score=0 without any base_score, then calling something like:
booster.inplace_predict(X, predict_type="margin")I would expect from that comment and from the documentation to get a single tree making predictions for a single column of the output, but .predict() produces an output where each column has a non-zero value, even though the custom objective is called only once (verified by putting prints on it) and there is a single tree, as verified by:
import json
trees = json.loads(booster.save_raw(raw_format="json"))["learner"]["gradient_booster"]["model"]["trees"]
len(trees)What's happening there?
Source: dmlc/xgboost