On minimum entropy and Gaussian transport

Olivier Rioul 1, 2
1 COMNUM - Communications Numériques
LTCI - Laboratoire Traitement et Communication de l'Information
Abstract :

A nontrivial linear mixture of independent random variables of fixed entropies has minimum entropy only for Gaussian distributions. This "minimum entropy principle" was first stated for two variables by Shannon in 1948 in the form of the entropy-power inequality which has long been proven useful for deriving converse multiuser coding theorems. It was also also applied to deconvolution problems by Donoho and generalized to linear transformations by Zamir and Feder, and more recently to Rényi entropies with different formulations by Bobkov and Chistyakov and by Ram and Sason. Available proofs involve either the integration over a path of Gaussian perturbation of Fisher information or minimum mean-squared errors, or a limiting case of Young's convolutional inequality with sharp constants. In this work, we show that a natural transportation argument from the Gaussian distribution yields simple derivations of the minimum entropy principle in all these cases. The basic ingredient is a change of variable in differential entropies which dates back to Shannon’s 1948 paper. We discuss possible generalizations and perspectives of this method.

Complete list of metadatas

https://hal.telecom-paristech.fr/hal-02287932
Contributor : Telecomparis Hal <>
Submitted on : Friday, September 13, 2019 - 5:30:27 PM
Last modification on : Thursday, October 17, 2019 - 12:36:59 PM

Identifiers

  • HAL Id : hal-02287932, version 1

Citation

Olivier Rioul. On minimum entropy and Gaussian transport. Entropy 2018: From Physics to Information Sciences and Geometry, May 2018, Barcelona, Spain. ⟨hal-02287932⟩

Share

Metrics

Record views

5