In this paper we present a result that relates merging of closed convex sets of discrete probability functions respectively by the squared Euclidean distance and the Kullback-Leibler divergence, using an inspiration from the Rényi entropy. While selecting the probability function with the highest Shannon entropy appears to be a convincingly justified way of representing a closed convex set of probability functions, the discussion on how to represent several closed convex sets of probability functions is still ongoing. The presented result provides a perspective on this discussion. Furthermore, for those who prefer the standard minimisation based on the squared Euclidean distance, it provides a connection to a probabilistic merging operator based on the Kullback-Leibler divergence, which is closely connected to the Shannon entropy.
We offer a new approach to the \emph{information decomposition} problem in information theory: given a `target' random variable co-distributed with multiple `source' variables, how can we decompose the mutual information into a sum of non-negative terms that quantify the contributions of each random variable, not only individually but also in combination? We define a new way to decompose the mutual information, which we call the \emph{Information Attribution} (IA), and derive a solution using cooperative game theory. It can be seen as assigning a "fair share'' of the mutual information to each combination of the source variables. Our decomposition is based on a different lattice from the usual `partial information decomposition' (PID) approach, and as a consequence {the IA} has a smaller number of terms {than PID}: it has analogs of the synergy and unique information terms, but lacks separate terms corresponding to redundancy, instead sharing redundant information between the unique information terms. Because of this, it is able to obey equivalents of the axioms known as `local positivity' and `identity', which cannot be simultaneously satisfied by a PID measure., Nihat Ay, Daniel Polani and Nathaniel Virgo., and Obsahuje bibliografické odkazy