How to Maximize the Renyi Entropy Rate and Why
Seeking an operational meaning to differential Renyi entropy, we formulate the "high-resolution task-encoding problem" and solve for its asymptotics. The solution motivates the study of the maximization of Renyi entropy subject to linear constraints. Particularly noteworthy is that the maximum of the Renyi entropy of a random vector subject to constraints on its components is not achieved by a random vector with independent components. This raises a question about the Renyi entropy rate of stochastic processes: Given a cost constraint on its components, which stationary stochastic process has the highest Renyi entropy rate? We shall see that the supremum is closely related to the analogous supremum of the Shannon rate. We'll conclude with a Burg-like theorem on the Renyi entropy rate and the spectral estimation problem.
Based on joint work with Christoph Bunte.