The $f$-Divergence Expectation Iteration Scheme

Kamélia Daudel 1, 2 Randal Douc 3, 4 François Portier 1, 2 François Roueff 1, 2
1 S2A - Signal, Statistique et Apprentissage
LTCI - Laboratoire Traitement et Communication de l'Information
4 TIPIC-SAMOVAR - Traitement de l'Information Pour Images et Communications
SAMOVAR - Services répartis, Architectures, MOdélisation, Validation, Administration des Réseaux
Abstract : This paper introduces the $f$-EI$(\phi)$ algorithm, a novel iterative algorithm which operates on measures and performs $f$-divergence minimisation in a Bayesian framework. We prove that for a rich family of values of $(f,\phi)$ this algorithm leads at each step to a systematic decrease in the $f$-divergence and show that we achieve an optimum. In the particular case where we consider a weighted sum of Dirac measures and the $\alpha$-divergence, we obtain that the calculations involved in the $f$-EI$(\phi)$ algorithm simplify to gradient-based computations. Empirical results support the claim that the $f$-EI$(\phi)$ algorithm serves as a powerful tool to assist Variational methods.
Complete list of metadatas

Cited literature [33 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02298857
Contributor : Kamélia Daudel <>
Submitted on : Friday, September 27, 2019 - 11:39:29 AM
Last modification on : Thursday, October 17, 2019 - 12:37:03 PM

File

ddpr2019.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02298857, version 1
  • ARXIV : 1909.12239

Citation

Kamélia Daudel, Randal Douc, François Portier, François Roueff. The $f$-Divergence Expectation Iteration Scheme. 2019. ⟨hal-02298857⟩

Share

Metrics

Record views

73

Files downloads

23