Regression
The hgboost method consists 3 regression methods: xgboost_reg, catboost_reg, lightboost_reg.
Each algorithm provides hyperparameters that must very likely be tuned for a specific dataset.
Although there are many hyperparameters to tune, some are more important the others. The parameters used in hgboost are lised below:
- Parameters
The number of trees or estimators.
The learning rate.
The row and column sampling rate for stochastic models.
The maximum tree depth.
The minimum tree weight.
The regularization terms alpha and lambda.
xgboost
The specific list of parameters used for xgboost: hgboost.hgboost.hgboost.xgboost_reg()
# Parameters:
'learning_rate' : hp.quniform('learning_rate', 0.05, 0.31, 0.05)
'max_depth' : hp.choice('max_depth', np.arange(5, 30, 1, dtype=int))
'min_child_weight' : hp.choice('min_child_weight', np.arange(1, 10, 1, dtype=int))
'gamma' : hp.choice('gamma', [0, 0.25, 0.5, 1.0])
'reg_lambda' : hp.choice('reg_lambda', [0.1, 1.0, 5.0, 10.0, 50.0, 100.0])
'subsample' : hp.uniform('subsample', 0.5, 1)
'n_estimators' : hp.choice('n_estimators', range(20, 205, 5))
'early_stopping_rounds' : 25
catboost
The specific list of parameters used for catboost: hgboost.hgboost.hgboost.catboost_reg()
'learning_rate' : hp.quniform('learning_rate', 0.05, 0.31, 0.05),
'max_depth' : hp.choice('max_depth', np.arange(2, 16, 1, dtype=int)),
'colsample_bylevel' : hp.choice('colsample_bylevel', np.arange(0.3, 0.8, 0.1)),
'n_estimators' : hp.choice('n_estimators', range(20, 205, 5)),
'early_stopping_rounds' : 10
lightboost
The specific list of parameters used for lightboost: hgboost.hgboost.hgboost.lightboost_reg()
# Parameters:
'learning_rate' : hp.quniform('learning_rate', 0.05, 0.31, 0.05),
'max_depth' : hp.choice('max_depth', np.arange(5, 30, 1, dtype=int)),
'min_child_weight' : hp.choice('min_child_weight', np.arange(1, 8, 1, dtype=int)),
'subsample' : hp.uniform('subsample', 0.8, 1),
'n_estimators' : hp.choice('n_estimators', range(20, 205, 5)),
'eval_metric' : 'l2'
'early_stopping_rounds' : 25