Explainable Hyperparameters Optimization using Hilbert-Schmidt Independence Criterion - Centre de mathématiques appliquées (CMAP) Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2021

Explainable Hyperparameters Optimization using Hilbert-Schmidt Independence Criterion

Résumé

Tackling new machine learning problems with neural networks always means optimizing numerous hyperparameters that define their structure and strongly impact their performances. In this work, we use a robust system conception approach to build explainable hyperparameters optimization. This approach is defined by the research of a parametrization of a system (a neural network) to optimize its output response (the prediction error) to an input stimulation (the test set). To that end, we study the use of Hilbert-Schmidt Independence Criterion (HSIC), a probability distribution dependence measure widely used for sensitivity analysis in robust system conception, in the context of Hyperparameters Optimization. Hyperparameters spaces can be complex and awkward, with different natures of hyperparameters (categorical, discrete, boolean, continuous), interactions and inter dependencies, which makes it non trivial to apply HSIC. We alleviate these difficulties to make HSIC applicable in that context and obtain an analysis tool that quantifies the relative impact of hyperparameters on a Neural Network's final error. Notably, we show how this knowledge allows obtaining competitive neural networks that are naturally much more cost effective.
Fichier principal
Vignette du fichier
ExplainableHSIC___preprint.pdf (1.39 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03128298 , version 1 (02-02-2021)
hal-03128298 , version 2 (25-02-2021)
hal-03128298 , version 3 (13-04-2021)
hal-03128298 , version 4 (22-06-2021)
hal-03128298 , version 5 (13-12-2021)

Identifiants

  • HAL Id : hal-03128298 , version 2

Citer

Paul Novello, Gaël Poëtte, David Lugato, Pietro M Congedo. Explainable Hyperparameters Optimization using Hilbert-Schmidt Independence Criterion. 2021. ⟨hal-03128298v2⟩
626 Consultations
811 Téléchargements

Partager

Gmail Facebook X LinkedIn More