Table 2 Optimal hyperparameters for gradient-boosted models
Hyperparameter | Search space | Optimal parameters | |||
|---|---|---|---|---|---|
GBR | LGBM | XGB | CBT | ||
No of estimators | 10–1000 | 520 | 780 | 890 | 980 |
Learning rate | 0.01–0.5 | 0.12 | 0.13 | 0.12 | 0.12 |
Max features | 0.5–1 | 0.7 | 1 | 0.7 | Default |
Subsample | 0.7–1 | 0.8 | Default | 0.8 | Default |
Max depth (depth) | 1–10 | 5 | 5 | 5 | 5 |
Num leaves | 0–10 | – | 10 | Default | Default |
alpha (L1 Reg) | 0.01–2.0 | – | 0.14 | 0.6 | – |
Lambda (L2 Reg) | 0.01–2.0 | – | Default | Default | 1 |