does how many foundation model layers are required for a particular competency correlate with evolution
#idea #question #ai #interp
- in the platonic representation hypothesis and what it says about the world, I argue that this layered similarity of representations between different large models is a result of the fact that evolution built up our model of the world layer by layer
- this hypothesis would imply that when a particular competency arose evolutionarily should be monotonically related with how many layers of these foundation models are required to perform this task
- e.g. one might expect something like "contains an animal" or "has something scary" to be predictable with fewer layers than something like number sense