Natural Gradient Boosting for Probabilistic Prediction
ngboost is a Python library that implements Natural Gradient Boosting, as described in “NGBoost: Natural Gradient Boosting for Probabilistic Prediction”. It is built on top of Scikit-Learn, and is designed to be scalable and modular with respect to choice of proper scoring rule, distribution, and base learner. A didactic introduction to the methodology underlying NGBoost is available in this slide deck.
via pippip install --upgrade ngboostvia conda-forgeconda install -c conda-forge ngboost
Probabilistic regression example on the Boston housing dataset:
from ngboost import NGBRegressorfrom sklearn.model_selection import train_test_splitfrom sklearn.metrics import mean_squared_error#Load Boston housing datasetdata_url = "http://lib.stat.cmu.edu/datasets/boston"raw_df = pd.read_csv(data_url, sep=r"\s+", skiprows=22, header=None)X = np.hstack([raw_df.values[::2, :], raw_df.values[1::2, :2]])Y = raw_df.values[1::2, 2]X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2)ngb = NGBRegressor().fit(X_train, Y_train)Y_preds = ngb.predict(X_test)Y_dists = ngb.pred_dist(X_test)# test Mean Squared Errortest_MSE = mean_squared_error(Y_preds, Y_test)print('Test MSE', test_MSE)# test Negative Log Likelihoodtest_NLL = -Y_dists.logpdf(Y_test).mean()print('Test NLL', test_NLL)
Details on available distributions, scoring rules, learners, tuning, and model interpretation are available in our user guide, which also includes numerous usage examples and information on how to add new distributions or scores to NGBoost.
Tony Duan, Anand Avati, Daisy Yi Ding, Khanh K. Thai, Sanjay Basu, Andrew Y. Ng, Alejandro Schuler. 2019.
NGBoost: Natural Gradient Boosting for Probabilistic Prediction.
arXiv