Hyperparameter optimization of deep neural networks: combining Hperband with Bayesian model selection - Télécom Paris Accéder directement au contenu
Communication Dans Un Congrès Année : 2017

Hyperparameter optimization of deep neural networks: combining Hperband with Bayesian model selection

Résumé

One common problem in building deep learning architectures is the choice of the hyper-parameters. Among the various existing strategies, we propose to combine two complementary ones. On the one hand, the Hyperband method formalizes hyper-parameter optimization as a resource allocation problem, where the resource is the time to be distributed between many configurations to test. On the other hand, Bayesian optimization tries to model the hyper-parameter space as efficiently as possible to select the next model to train. Our approach is to model the space with a Gaussian process and sample the next group of models to evaluate with Hyperband. Preliminary results show a slight improvement over each method individually, suggesting the need and interest for further experiments.
Fichier non déposé

Dates et versions

hal-02412262 , version 1 (15-12-2019)

Identifiants

  • HAL Id : hal-02412262 , version 1

Citer

Hadrien Bertrand, Roberto Ardon, Matthieu Perrot, Isabelle Bloch. Hyperparameter optimization of deep neural networks: combining Hperband with Bayesian model selection. CAp, 2017, Grenoble, France. ⟨hal-02412262⟩
111 Consultations
0 Téléchargements

Partager

Gmail Facebook X LinkedIn More