what does new mean

#theme

Summary

Example motivating questions
- Do LLMs create new content?
- Is the idea of an LLM even new?
- Are human ideas new?
My main thesis here was that the step at which something new was actually created, is where a change in/redraw of the randomness would have the biggest effect.
- Levels at which you get self averaging (as I think physicists would call it) would be disqualified
- I.e. if an idea would've still occurred around the same time, had you gone back a couple years and re-randomized the chance interactions and chance observations/encounters that people had (and perhaps re-randomized genetics), then I'd consider that idea pure computation rather than creation.
- On the other hand, if such a re-randomization would've precluded said idea from occurring (or delayed it significantly) than I'd consider that to be creation.
- For example, what was new in Newton's laws of motion was not the thought process that led to his idea, but the apple that fell on his head. More precisely, it was the particular lucky interactions with people, the environment, and perhaps his genetics, that at this particular scale of coarse graining had never arisen in the past.
  - Since under a counterfactual re-randomization of his neural firings, he probably would've still come up with his laws of motion.
  - But a under a re-randomization of those chance encounters throughout say the years leading up to his discoveries, things likely would've been different.
- Similarly, what was new in the development of LLMs was not the particular interactions between the people involved, but rather the "chance institutions/structures" from which LLMs then followed: big tech companies, the creation of the particular institutes and conferences dedicated to AI, an environment in which people working on AI would be able to take inspiration from developments in neuroscience, etc...
what is new - examples
how to tell is something is new - an attempt at an operational definition

Implications

maybe the true impact is by the people who create the structures and technologies which make scientific discovery possible (even though the glory ends up with the Edisons and the )
the importance of mistakes and chance in science: maybe mistakes and chance are the only truly new things?
- Penicillin was highly unlikely to have been an instance of simultaneous discovery, as it was due to an accident in the lab

Things to follow up on

what is purely symmetry breaking vs actually fundamental and new?

Raw notes

2024-08-04

There's always this question of: what is new? E.g. when asking the question of how to balance between cultural transmission and cultural creation, what really is creation?

Perhaps our answer to this question has changed since LLMs have shot onto the scene. Whereas I might've considered before creation to be someone sitting in a room and thinking, I'm realizing that creation might instead be synonymous with chance and choice.

And emergence happens when the change and choice at some level becomes exhausted. E.g. at the level of cells, perhaps we've exhausted the extent to which different interactions between cells lead to unfamiliar/unpredictable behaviors.

What do I mean by chance and choice? I mean the chance encounters and interactions. And I mean that chance really is choice.

How does chance get exhausted?

When the variance in observables caused by a single person's path through life becomes negligible

One might argue also that innovation is still happening; perhaps at an even faster pace than ever before. We've got ChatGPT, a new Alzheimer's drug, etc...

But my argument would be that these things sort of had to be. They were just computations. The forward development of concepts at the societal level; sort of well formed entities which had an almost deterministic trajectory forward
- Now, that doesn't necessarily mean they were predictable. Just as one wouldn't necessarily say that my dreams are predictable.
- And it's not even individual level predictability that's changing.
- Instead, it's some more coarse grained notion of predictability.
- Or maybe something that only exists counterfactually: if I had rerun history with different randomness, would things be different?
- That's what something new is. Chance and choice. Which are really just the same thing. A path we chose as a result of random chance.

Hmm. I kind like where this is going. It's a counterintuitive definition of new. For instance, I'm saying that my thoughts sort of in isolation aren't really new. These are just the computation. Because adding some noise at the level of the neurons would not have made a significant change in the conclusions I reached.

This goes against what I like to think; I like to think of my ideas/thoughts as being the real content of innovation. And it sure does feel that way. It feels like I'm trying hard to generate good ideas.
But perhaps this is an instance where in resilience vs adaptability, resilience wins.

Analogously, at the society level, I'm arguing that, e.g. LLMs would've still arised if we injected more random noise at the individual level. Because the ideas floating around, necessary for this to occur, were there at a macroscopic level.

In contrast to, perhaps, something like the invention of Galois theory can be considered something truly new.
[?] what is purely symmetry breaking vs actually fundamental and new
[?] If I were to take GPT and have it just keep eating the internet and generating text to put on the internet, without any additional human input, would that be new/creation?
- It is true the GPT has randomness. But I guess the point is that, for macro level identifiable changes, is
[?] How important is genetic variation?
[?] Where does the environment enter?
[?] Also, the fact that progress itself creates new opportunities; it changes how interactions work
- Perhaps this relates to the environment, in the sense that progress gets solidified and enters as a component of the environment.

What does all of this mean, operationally?