In 2024-09-12 9.360 - neurobiology of self, Fan states that it seems like the brain has more information about an object than is contained in the set of sensory neurons.
This is because it's generating information.
But... what does that mean? Formally, in the information-theory sense... I don't think this really makes any sense.
However, intuitively it makes sense, since the brain is able to fill in the gaps.
I can feel a leaf and be able to tell what its color is, even though I haven't opened my eyes
This is because there exists structure to the world.
Really, the color green isn't new information, since the conditional entropy of color given the feel (for this realization of feel) is close to 0.
However, being able to use this structure is itself not immediate. It's something learned. You could give blindfold me an give me a basketball, and I'd be able to predict its color. But presumably a baby wouldn't be able to do this.
So... what information is being added?
It almost feels related to inductive bias: it's a regularity in our model which helps us extrapolate to far out of distribution (we didn't get any visual data, yet we can predict it)