Cross-entropy, feedback and variational inference
What the functional selects
Fix a density matrix . For an output supported on its support, the von Neumann entropy and the quantum relative entropy satisfy
The right side is linear in the output. Diagonalizing shows that its minimum is . Equality holds precisely when the output is supported on the maximal eigenspace. The bound is attainable over all CPTP maps: discard the input and prepare any state on that space. This optimization cannot single out a feedback channel; even fixing the output for one input generally leaves its action on others undetermined. To select a channel one needs a task with independently specified observations, allowed resources and a rule for scoring different channels. Resource cost of feedback
Two different optimization problems
For diagonal input and output , the functional becomes the cross-entropy . Its minimum lies on the most probable input symbol. Variational inference instead starts with a joint generative law , observed with , and a trial posterior . Its objective is
With an unrestricted posterior family it is minimized by the posterior itself. For the first objective is about at a point mass and at , in nats. Removing off-diagonal entries neither supplies a sensory observation nor selects the posterior. A bridge to active inference needs a generative law, interventions and a learning rule that can be compared on data.
Spectral entropy on the state domain
For a positive operator of unit trace, all eigenvalues are nonnegative, so the sum with equals the von Neumann entropy. For a general operator it is not a state entropy. This identity gives no extremal characterization of the self-model.