Title
An algebraic, analytic, and algorithmic investigation on the capacity and capacity-achieving input probability distributions of finite-input-finite-output discrete memoryless channels
Document Type
Article
Publication Date
3-1-2008
Abstract
In this paper, we investigate the capacity and capacity-achieving input probability distributions (IPDs) of finite-input-finite-output discrete memoryless channels (DMCs). In the general respect, we establish a novel and simple characterization for the capacity-achieving IPDs of a DMC, which is equivalent to the conventional Kuhn-Tucker conditions. We then prove a conjecture of Majani and Rumsey, which claims that every probability component of each capacity-achieving IPD of a DMC with positive capacity is less than 1 - e-1, where e = 2.71828182⋯ is the base of natural logarithms. It remains an open problem whether there exists an explicit closed-form solution for the capacity and capacity-achieving IPDs of a general finite-input-finite-output DMC, except for the two-input-two-output DMC. In the algebraic respect, we demonstrate that there does not, in general, exist an algebraic solution for the capacity-achieving IPDs of an m-input-n-output DMC for any m ≥ 2 and any n ≥ 3. In the analytic respect, however, we can obtain an explicit closed-form analytic solution, represented as an infinite series, for the capacity-achieving IPD of a two-input-three-output DMC. We also provide a formula for the average capacity of weakly symmetric DMCs and show that the average capacity in nats per channel use of the n-input-n-output weakly symmetric DMCs increases for n ≥ 2 but has a finite limit of 1 - γ as n → ∞, where γ = 0.57721566⋯ is Euler's constant. In the algorithmic respect, the convergence of the Arimoto-Blahut algorithm is proved in a direct and elementary way. A new and simple iterative algorithm for calculating a capacity-achieving IPD is then proposed, which is provably convergent for all DMCs with positive transition probabilities. Finally, the characterization and determination of the set of all capacity-achieving IPDs of a DMC are addressed. © 2008 IEEE.
Publication Source (Journal or Book title)
IEEE Transactions on Information Theory
First Page
1003
Last Page
1023
Recommended Citation
Liang, X. (2008). An algebraic, analytic, and algorithmic investigation on the capacity and capacity-achieving input probability distributions of finite-input-finite-output discrete memoryless channels. IEEE Transactions on Information Theory, 54 (3), 1003-1023. https://doi.org/10.1109/TIT.2007.915703