XGBoost is an open-source gradient-boosting library begun by Tianqi Chen around 2014 and developed under the DMLC (Distributed Machine Learning Community) project. Its documentation describes it as “an optimized distributed gradient boosting library designed to be highly efficient, flexible and portable,” implementing machine learning algorithms under the gradient-boosting framework, also known as GBDT or GBM. For several years it was the single most common winning ingredient in Kaggle competitions on structured, tabular data.
The accompanying paper, Chen and Guestrin’s “XGBoost: A Scalable Tree Boosting System” (KDD 2016), frames the project explicitly as a systems contribution rather than a new learning theory. Gradient-boosted trees were already well understood; what XGBoost added was an engineering treatment that made them fast and scalable. The paper highlights a sparsity-aware algorithm for handling missing and sparse features, a weighted quantile sketch for approximate split finding, and attention to cache access patterns, data compression, and out-of-core computation so that training could scale beyond billions of examples on modest hardware.
As software, XGBoost is a C++ core wrapped in bindings for many languages, with Python and R being the most heavily used. The Python binding deliberately mimics the scikit-learn estimator interface, exposing classes with fit and predict methods, so an XGBoost model can be dropped into scikit-learn pipelines, cross-validation loops, and grid searches with little friction. This interoperability was a significant reason for its rapid adoption: practitioners did not have to abandon their existing tooling to use it.
The library also exposed performance-engineering choices directly to users. Parallel tree construction, the choice between exact and histogram-based split algorithms, and later GPU acceleration are configurable, letting the same model definition run on a laptop or a cluster. This combination of strong out-of-the-box accuracy on tabular data and tunable, well-optimized execution is what set it apart from earlier boosting implementations.
XGBoost is a clear case study in how careful systems work, rather than novel algorithms, can reshape a field’s tooling. It established gradient boosting as the default first choice for tabular prediction, influenced competing libraries that followed with similar performance-focused designs, and remains a standard component of the practical machine-learning toolkit.