In thinking about change vs continuity, I started wondering, what even is new?
Summary
Example motivating questions
Do LLMs create new content?
Is the idea of an LLM even new?
Are human ideas new?
My main thesis here was that the step at which something new was actually created, is where a change in/redraw of the randomness would have the biggest effect.
Levels at which you get self averaging (as I think physicists would call it) would be disqualified
I.e. if an idea would've still occurred around the same time, had you gone back a couple years and re-randomized the chance interactions and chance observations/encounters that people had (and perhaps re-randomized genetics), then I'd consider that idea pure computation rather than creation.
On the other hand, if such a re-randomization would've precluded said idea from occurring (or delayed it significantly) than I'd consider that to be creation.
For example, what was new in Newton's laws of motion was not the thought process that led to his idea, but the apple that fell on his head. More precisely, it was the particular lucky interactions with people, the environment, and perhaps his genetics, that at this particular scale of coarse graining had never arisen in the past.
Since under a counterfactual re-randomization of his neural firings, he probably would've still come up with his laws of motion.
But a under a re-randomization of those chance encounters throughout say the years leading up to his discoveries, things likely would've been different.
Similarly, what was new in the development of LLMs was not the particular interactions between the people involved, but rather the "chance institutions/structures" from which LLMs then followed: big tech companies, the creation of the particular institutes and conferences dedicated to AI, an environment in which people working on AI would be able to take inspiration from developments in neuroscience, etc...