MDA for random forests: inconsistency, and a practical solution via the Sobol-MDA - Centre de mathématiques appliquées (CMAP) Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2021

MDA for random forests: inconsistency, and a practical solution via the Sobol-MDA

Résumé

Variable importance measures are the main tools to analyze the black-box mechanism of random forests. Although the Mean Decrease Accuracy (MDA) is widely accepted as the most efficient variable importance measure for random forests, little is known about its theoretical properties. In fact, the exact MDA definition varies across the main random forest software. In this article, our objective is to rigorously analyze the behavior of the main MDA implementations. Consequently, we mathematically formalize the various implemented MDA algorithms, and then establish their limits when the sample size increases. In particular, we break down these limits in three components: the first two are related to Sobol indices, which are well-defined measures of a variable contribution to the output variance, widely used in the sensitivity analysis field, as opposed to the third term, whose value increases with dependence within input variables. Thus, we theoretically demonstrate that the MDA does not target the right quantity when inputs are dependent, a fact that has already been noticed experimentally. To address this issue, we define a new importance measure for random forests, the Sobol-MDA, which fixes the flaws of the original MDA. We prove the consistency of the Sobol-MDA and show its good empirical performance through experiments on both simulated and real data. An open source implementation in R and C++ is available online.
Fichier principal
Vignette du fichier
sobolMDA_arxiv.pdf (910.72 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03151611 , version 1 (25-02-2021)
hal-03151611 , version 2 (16-11-2021)
hal-03151611 , version 3 (28-02-2022)

Identifiants

Citer

Clément Bénard, Sébastien da Veiga, Erwan Scornet. MDA for random forests: inconsistency, and a practical solution via the Sobol-MDA. 2021. ⟨hal-03151611v1⟩
332 Consultations
490 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More