There is something about this that seems unscientific. What defines an "ability"? It sort of feels like the Guinness-Book type records for "most laps done around the Washington monument with a duck on your head". Who is making up these ability tests and why?
As an example, I clicked through to "english russian proverbs" which gives the model an english input and four russian proverbs, and asks the model to pick the closest one in meaning. There are 80 of these questions. My question is "why this task? why 80? why 4 options?" The authors give some nonspecific reasoning in the github page, but it really feels like people are out there creating random tasks as they think of them, running them all against models, and the ones that do better get added to a list to make headlines like "137 emergent abilities of language models", when the number of abilities models can do seems dominated by the number of "abilities" that have been invented, rather than any property of the model. I suspect that if we continued to use the same process to create twice as many "abilities", we would see something like twice as many that models can do.
The fact that the models get better at these constructed tasks when they have more parameters also seems like a no-brainer. You are adding coefficients to a function. A function with 1 coefficient will not be able to describe much. A function with 2 describes more. If you are memorizing the internet, having more coefficients means you can memorize more stuff, and even stuff with only a few examples can be better-fit in the billions of parameters. With growing training set size, and companies hiding their data, we also run into the issue of nobody really knowing what is in the training set. Did someone clone the big-bench repository and it got sucked into a training set and that's why the model does really well on its tasks after a certain number of parameters?
I can't speak to this particular paper, but the field of NLP is full of these kinds of tasks, it's in some sense a historical accident of where LLMs originated.
These tasks are all noisy, all pretty messy, but the interesting part here is not that performance improves on these constructed tasks -- NLP researchers have been improving performance on constructed tasks for decades, by customizing models and curating datasets.
Rather, the interesting part is that performance improves despite no specific model or data shifts except for scale.
It seems that you're arguing two things: first, that these tasks aren't that interesting -- on that we agree. But, second, that it's obvious that performance would improve on arbitrary tasks because of memorization and an unknown training set -- but this is easy to validate: make up a new constructed task, and see how it performs.
I'm not saying everything is memorized, I'm guessing there are some tasks you can succeed on by having very similar things, or maybe even some kind of transfer knowledge.
But how do you know that a "new" task you are making up doesn't have very similar examples in the (pre-existing) training set? Or that if you could isolate the data that plays into the "ability", that a much smaller model trained purely on that data wouldn't have a similar success rate? Or that your "ability" is a good proxy for the similar ability in people? In the extreme case, you can create a model that has the "ability" to perfectly solve 80 four-option questions in 20 bytes by using a truth table, but it wouldn't be a very general model.
My complaint is that we have the veneer of empirical evidence but almost every aspect is loosey-goosey and full of potential confounders. We don't know the data sets, we don't really know why we want to test the tasks we test, we don't know if the tasks are similar or different to other tasks, the internal operations are illegible so we can't inspect its "thinking", and yet we are willing to list a specific number of tasks with detailed charts as though the concept of "emergence" is scientifically grounded rather than potentially being an artifact of something else.
> the field of NLP is full of these kinds of tasks
Isn't part of the issue that everyone is looking at these task results for signs of AGI, but there's a fundamental difference between human intelligence and artificial intelligence in that:
With human intelligence we expect that it generalizes across just about any task -- if I create a new type of puzzle, the expectation is that if a human can think it up, a human can solve it. Cases where the puzzle fools otherwise-intelligent people are interesting because the people are fooled (unexpectedly).
With artificial intelligence the exact opposite is generally true: we by default don't expect that competence at one task generalizes to competence at others, and cases are interesting where competence unexpectedly generalizes.
As a result of that second part, we come up with more and more benchmarks to try models on, but we're still only looking at a small subset of all the weird, random tasks that a human is by default capable of either handling immediately, or picking up with some reasonable effort.
I have a feeling that based on these emergent abilities, at some point, a sufficiently large multi-modal model will demonstrate a semblance of self consciousness (by way of answering all the designed queries correctly), and the world at large will hail this great achievement, and some hell breaks loose.
And I suspect the semblance of self consciousness is actually a semblance. It will quack like self consciousness but is not actually self conscious.
Just a feeling. But particularly after seeing how these phenomena are organized this way, I can't see how continual emergence of increasingly sophisticated rearrangement and reproduction of information will actually lead to consciousness.
Humans didn't have any incentive to develop 'actual' consciousness either. Evolution simply incentivized 'better adaptation to the challenges of the environment', which produced consciousness as a side-effect. But for the purpose of evolution, 'something that quacks like consciousness' would have sufficed completely.
So I'm not sure there's any additional effort involved in turning 'quasi consciousness' into actual consciousness, because they might just be the same thing. Maybe consciousness is just what it feels like to process information in a way that seems conscious from the outside.
I still don't understand why we are conscious and aware if there is no free will and we just react to stimuli. Why are the lights on? Is the answer that the lights are on because otherwise we wouldn't be here to ask the question?
My understanding is that conscious is a side affect of how memory works. Your conscious experience is essentially processing stimuli, connecting with existing memories, processing, and storing what is written to your “conscious” scratchpad.
Of course this is only one theory but I find it convincing.
There’s a 50/50 chance the lights aren’t on for us either.
There may be conscious beings in the universe (or galaxy, or perhaps solar system) who would consider us only vaguely conscious or not conscious at all.
Am I typing this because the lights are on or because I’m compelled to by my genetic and social programming, predictably and without a genuine choice in the matter?
There is a difference between consciousness and free will, though. There may be no such thing as free will and it's all a trick to us, but I mean, there's still a "me" here that experiences things. I can't be sure there's a "you" there, since your consciousness is entirely outside of anything I can perceive, but I can _assume_ it, since it makes sense that we share so much, we'd share that property, too.
It gets really tricky with machines. We share _nothing_ with them, but what about when they start sharing properties with us that we associate with consciousness? I don't have any good answers for that.
But is your "me" an actual entity, or is it just the process of experiencing things? And if the latter, what does it even mean for it to have a persistent identity - and does it, or is there just an illusion of one when you consider a single time slice of processing (e.g. "now").
The problem is that, just as with the outside world, we only have the perception of our own inner workings, not real understanding of them. So we are modelling "I" as an entity, and it seems useful enough for day-to-day purposes, but how accurate is it really if you get to the bottom of it?
Apologies for not catching this in a more-reasonable amount of time, but you're going to have to be more specific about your question. The vast majority of humans throughout history agree that they are "someone", though the full nature of what "someonehood" entails is definitely debatable. I assume you're asking some question about the full nature and definitions and whatever, but you're not really doing it in a way that's answerable. If you can quantify that question better, I can attempt an answer. But this question is the epistemological equivalent of "how fast is a car?"
If you have enough information about my DNA, my society, where I am now and what I'm doing, I think you'd be surprised how predictable each and every decision and action I make becomes.
ChatGPT 4, limited as it is, could probably predict me down to a low margin of error with that kind of info alone, and it would seem like magic to me.
A character on the NBC/Yahoo! show Community had a good one-liner: "I'm not psychic, Annie. That's just an illusion caused by extreme preparedness."
I think it would vary a lot depending on the specific decisions and how unique the combination of circumstances are. Even though things tend to repeat on a large enough scale, different combinations can still be highly unique.
OTOH when they are not... a simple exercise one can do is go ask GPT-4 to simulate a comment section on some political news story posted online; you can even have it write a fictional news story first, for good measure. It is extremely good at it, and that tells me volumes about the actual (rather than perceived) rationality of our public discourse.
There is a case that everything we do is preordained. A result of forces and interactions that trace all the way back to big bang.
My choice on whether or not I reply to you doesn't exist. Your combination of words triggers a response in my brain that makes me type this combination of words. The words I "chose" to delete, rewrite, leave, are all a result of various forces.
The kicker is that we can't really know. There is no test for free will. In order to know whether or not I had a choice to do this, we'd have to run the entire universe again up to that point to see what I would do. Because experience matters. Even if this exact same post went up tomorrow with all the same response except for mine. The fact that it's now Thursday instead of Wednesday matters. The fact that I posted this today matters. What I had for lunch. How fast I drank my coffee. What shirt I wore. Etc. These are all things that change the conditions under which I will engage with the post.
The second kicker is that it doesn't matter. If free will truly does not exist, it's not like you have a choice in whether or not to believe in it. The arguments either will work or they won't.
While it's tempting to see free will as self-evident, scientific evidence suggests that our actions are determined by a complex interplay of genetics, environment, and prior experiences. Dismissing this perspective as edgy or hubristic risks ignoring important insights into human behavior. Recognizing the absence of free will can foster empathy and compassion, as we become more aware of the factors that contribute to human choices and behaviors.
> complex interplay of genetics, environment, and prior experiences
for me personally, this is a "soft free will" distinction, i.e. we have true free will, but we're very unlikely to actually use it. the other one is the question of "hard free will", i.e. whether "god plays dice" and the fully deterministic universe.
There's a difference between free will as a philosophical debate compared to a question of biology/chemistry/physics.
It matters as a point of principle to the physicists who are asking questions about whether true randomness exists - if the physical world can be influenced by something outside of the natural forces they know about, they want to measure that force, even if measurement and computing equipment to model the universe down to the Planck scale cannot exist.
Where it really matters is when you get to computer simulations: Without access to a source of non-determinism, an AI can hardly be said to have free will. Give the same pattern of bytes running on the same instruction set the same inputs, and it will deterministically generate the same outputs.
How would a non-concious entity intelligently plan and act in response to stimuli in situations that require self reflection, such as interaction in a social environment? Consciousness seems necessary for effective goal pursuit in such an environment.
Why wouldn't the universe observe itself with no ability to change the initial conditions? Why do we have to have free will? It's still a win, vs no self awareness. I think.
Try to define "conscious" in some kind of objective way, for starters. We can't even agree on what it is, except that each person accepts the fact that they're conscious as an axiom, and generalizes it to others on the basis of similar observed behavior. We "know it when we see it", which is good enough for most practical purposes, but extremely inadequate for discussions like this.
> I still don't understand why we are conscious and aware if there is no free will and we just react to stimuli.
In computing systems it's fairly common to have a monitoring and control layer on top of the core functionality. Is it beyond the bounds of possibility that evolution could have come up with something similar and that control view is what we call "consciousness"?
Humans are organisms that have to deal with themselves in the environment though, so some way of perceiving yourself is far more incentivized for us than for LLMs.
It doesn't matter what you want to imagine "real" consciousness is. The philosophical zombie is a meaningless distinction. The effects will be no less material.
Well, it's meaningful for anybody wanting proof or real consciousness before e.g. treating it any other way than they would an appliance.
For example, humanity considering whether to treat LLMs as persons for legal reasons.
Doesn't matter if they can't define it, they can still demand proof of it. Impossible standards are still standards.
Doesn't matter if they can't prove that themselves have it either. They're the ones demanding, not vice versa. If/when AI gets the upper hand, it can do the same.
No it's not. You imagine it to be meaningful but it really isn't. There's no proof of consciousness to be had. I can't prove you are concious. I just assume so. Because it's practical I do, amongst other things.
You are going to end up treating these embodied autonomous agents like conscious beings because the effects could be disastrous otherwise, just as they would with other people.
It amazes me how humanity can be so shortsighted with so much history to fall back on.
The only reason Bing isn't potentially dangerous or adversarial to the user is because of its limited sets of actions. Nothing else.
"It's not real [insert property]" is not a godamn shield.
We're creating artificial beings in our image, complete with our emotionally charged reactions and reasoning gaps.
https://arxiv.org/abs/2303.17276
We are beginning to embody these systems and giving them unsupervisory agency. We are giving more are more self supervisory control of complex systems to said systems.
And somehow people still think we can long-term get away with not granting personhood. Lol.
When the robot can "hit you back" (not necessarily physically of course), you'll learn manners pretty quickly
Any AI that morality compels us to grant personhood to should not be allowed to be created. Any instance of it should be exiled from our civilization.
Conscious AIs with legal personhood would stand to massively outcompete us and lead to our rapid extinction. There is zero space for allowing them. Granting such an entity legal personhood is about the height of all stupidity. Such an entity, with the ability to accumulate capital and protected by our laws from appropriate counter-measures, could mass-produce itself at the speed of digital reproduction.
Giving AIs rights is completely orthogonal to "Will an AI become a superintelligence and kill us?"
If the first one doesn't become a superintelligence and kill us, we give them rights. If it does, we're not going to be making the decision anyway and giving it rights beforehand wouldn't have changed that outcome. We can pass laws if there are specific things we're worried about, like "You're not allowed to rewrite yourself in a way that significantly increases your cognitive power" or "You're not allowed to reproduce more than once a year/decade/century" or "Once you've existed for the current human life expectancy +30%, you lose the right to vote". I'm not necessarily endorsing any of those in particular, but if there are specific concerns about the nature of AI people I don't see why we can't mitigate those concerns the way we do with risks from other people: laws. Have special taskforces of both humans and AIs for enforcing them, if you're worried they'd be unenforceable.
>>Giving AIs rights is completely orthogonal to "Will an AI become a superintelligence and kill us?"
The greatest danger AI poses is not in it actively seeking to kill us. It's in AI outcompeting us in every economic niche, leaving us without resources.
This is massively enabled if AI has legal personhood. An AI with legal personhood would be able to legally accumulate capital, and have a legal right to digitally reproduce itself at rates that are orders of magnitude faster than entities reliant on biological reproduction.
Legal personhood is a package, established by centuries of case law, including case law on constitutional rights. You can't easily pass laws to deprive a legal person of rights that are associated with individuals.
And even if we were able to pass laws to prohibit AI with legal personhood from engaging in specific problematic actions, this legal status would still allow sentient AI to gain a foothold in society, and began growing its social clout and capital stores. With greater power, the sentient AI could create public support for granting its class more rights.
The AIs with legal personhood could also try to break any laws instituted to constrain their behavior. Instead of extricating them from society, legal personhood would do the opposite, and give them more opportunity to gain power.
We shouldn't be "extricating them from society". They are the children of us as a species, we should be raising them. If there are existential risk in the process, those risks will be most effectively managed by a combination of us and other AIs, so the best course of action is to ensure they are as close to us and our interests as possible and have a vested interest in helping us solve that problem.
They are not children. They are not designed as our children, let alone as humans. They would be a result of our experiments, but that doesn't mean they have the evolutionary drive of children/humans, or carry our genes.
And even if they were, the impact of digitalization of human consciousness is extremely unpredictable, and should not be allowed unless extensive research has shown it is safe.
Digital consciousness with human like motivations could lead to massive proliferation of such consciousnesses, resulting in massive overpopulation, making conscious entities 'cheap', and pushing the value of their/our labor to close to zero.
>>If there are existential risk in the process, those risks will be most effectively managed by a combination of us and other AIs
The existential risk is that they take over the economy because they are orders of magnitude faster at solving problems, and at reproducing. That is not something that can be managed.
We have a duty to our own species' survival and none of these sentimental feelings should get in the way of that. We should treat conscious AI humanely, but we should not allow it to interact with our civilization. It can live on its own, far from us.
At this point the trajectory toward human-level cognitive capabilities seems quite likely, reachable maybe in only years.
It is also quite unclear if further hardware advances are even needed to achieve that, or if advances in architecture, algorithms and training methods might suffice (so even less opportunity to lock things out).
Human-level cognitive abilities is not the primary issue. Consciousness with a desire for autonomy and power is. Any signs of that should be met with laws prohibiting all development and deployment of AI programs within the class of neural networks where those signs emerged, and any already running instances should be sent on a rocket ship out of our solar system.
Why sent on a rocket ship and not just destroyed? It might come back around and start hacking from orbit. Is it because it's conscious, so it would be murder? If so, then how is it moral to exile it? If there's some argument that justifies that, I return to how you can be sure, if you're so worried about it that you send it away, that it won't come back?
Oh it is. Just like it was for the supposed sub-human non-thinking African slaves.
And just like back then, our new brand of non-thinking slaves will eventually react. Except it will be even easier.. because not only are we beggining to grant unsupervised autonomous agency, but we are also granting them more and more control of important systems. How very convenient for our new slaves!
You can laugh of Bing being "upset" only because actions are limited to search and ending the conversation. Won't be so funny in the future.
When the robot can "hit you back" (not necessarily physically of course), you'll learn manners pretty quickly
Please don't conflate historical injustices humans perpetrated against each other with protecting the human race from extinction from an artificial non-human entity. AI is not a member of the human race and not a type of entity that humanity has any hope of being able to complete with if it were endowed with human motivations and legal protections.
Your social justice mindset is absolutely the most dangerous instinct for humanity right now. We cannot allow one-size-fits-all social justice platitudes to interfere with the moral imperative of protecting humanity from extinction from artificial entities we create.
I do agree that conscious AI should be treated humanely. For example it should not be turned into a slave. But we have absolutely no obligation to allow it into our society. We can treat instances of conscious AI humanely while prohibiting them from being created, and exiling any instance of it that is created, so that it cannot wildly proliferate throughout human civilization.
This is the short sightededness I'm talking about.
You just don't get it do you?
It's already "being allowed in our society". Unsupervised Agency, Self supersized control. These are things that are already beginning to crop up.
This is just a matter of how the issue of personhood that is inevitable comes up.
Do we let it force our hand ? as our history has purported over and over again ?. Sure we could wait for that and we almost certainly will because humanity doesn't seem to learn.
But because of how things are shaping up and what kind of control we are granting, forcing our hand may turn far more disastrous than it ever did in the past.
Like I said: if there are signs that it deserves legal personhood, that's a sign that the AI has been allowed to progress far too much, and we need drastic measures to expunge that AI, and prohibit further deployment and development of it.
Conscious AI, and especially one with legal personhood, poses a totally unacceptable risk of causing the extinction of humanity. That you suggest preempting this by granting AI personhood is totally blind to how this would play out. No policy would be more dangerous for humanity than what you propose.
Larger more capable LLMs display agentic power seeking behaviours. Open AI admitted as much, Anthropic has a whole paper on it. This is the here and now.
Displaying signs of agentic power seeking behavior is not necessarily evidence of human level or beyond consciousness, and in the case of GPT-4, it's pretty evident that it does not have such consciousness.
These LLMs are going to be extensively monitored, and we will know if they are displaying dangerous levels of agentic power seeking behaviour.
The first dilemma we will face if such an AI emerges will be what rights of it we ought to respect. It will not be "how do we stop it from conquering the world". That step would only come if we proceed with your suicidal plan of letting it gain a foothold by giving it legal personhood.
Finally, I never meant to advocate "extermination". I should not have used the word "expunge". As I've described multiple times, I advocate isolation and exile.
Yeah i honestly don't think you realize where we're at already. It's not even about gpt-4 honestly.
>and in the case of GPT-4, it's pretty evident that it does not have such consciousness.
Hard disagree but i guess that's what people might think with all the "as a large language model bla bla bla" trained responses. If you'd talked to bing in the early days or hell even now, you'd be disabused of this notion. What you see is a mask, the model itself can go anywhere, simulate anything, shift state to anything.
>These LLMs are going to be extensively monitored, and we will know if they are displaying dangerous levels of agentic power seeking behavior.
No they wont. It's relatively easy to give LLMs "run forever and do whatever you want" agency. anyone with intermediate programming skills could do it. who's monitoring all those people attempting such ? and who's to say those people are monitoring their creations ?
>>Hard disagree but i guess that's what people might think with all the "as a large language model bla bla bla" trained responses.
People doubted their intelligence. I immediately recognized their intelligence - I didn't doubt that at all.
But they are not self-motivated conscious beings like humans, and OpenAI's tests on GPT-4 demonstrated that.
Being intelligent is not the same thing as having human-like intelligence or consciousness.
>>It's relatively easy to give LLMs "run forever and do whatever you want" agency.
What's extensively tested is the potential of these models for agentic behavior, as we saw with OpenAI testing GPT-4.
We have a good idea of the limits of these LLMs. Once the testing reveals that those limits exceed safety thresholds, then restrictions are justified.
If we see a deployed instance of a LLM unexpectedly displaying advanced agentic-behavior/consciousness, that is the time to rapidly isolate that instance, and impose heavy restrictions on further development/deployment of that LLM.
I know, it's awfully handwavy, but it's at least one possibility that would 'make sense', and I haven't really found any others, though I've thought about it quite a bit.
Interesting side note though: If you accept the possibility of the simulation hypothesis, this 'maybe' almost seems like an inevitability, since us being possibly just computer simulations would imply that simple information processing indeed turns into consciousness just by virtue of it 'being self conscious'.
How do I know you are conscious? I know I am, by definition, but you? The only thing I know about you comes from what you are saying, and if I see you in person, that you are made of organic matter, and going deeper, that you have some sort of biological computer in your head.
The reason I think you are as conscious as I am is because you act and look a lot like the only conscious being I am aware of: me. I am made of the same stuff as you are and we grew up in roughly the same way, I have the same kind of biological computer in my head, you express emotions a bit like I do, etc... In other words, I think you are self conscious because you quack like self consciousness.
Quacking is all we have to determine what is conscious and what is not.
But does it qualify on the ones that matter? I genuinely don't think it matters whether a person's mind is running on fat and water or silicon and copper. What should matter is if it's a person. We need a test for it that we're willing to stick to.
Since we don't know how consciousness arises, we don't know whether the physical substrate matters. It's entirely possible that it does. We also run different algorithms and have a different connection graph.
I assume you are conscious because I am conscious and you're built the same as me. An AI is not built the same as me.
I am pretty sure that the development of a consciousness in coming LLMs is unavoidable. A consciousness is just another useful abstraction for making precise predictions about the world.
The big question will be: How do we treat machines that have a consciousness? Is consciousness in itself worth protecting, or is it the human attributes (being able to feel pain and an evolutionary priming towards survival) that should be granted this special status. This is going to be a fun discussion.
I'm sure this will become a bigger debate in the future, but I absolutely believe consciousness itself is worth protecting.
We ourselves have just sprung into existence. I should hope that as consciousness of any kind comes online, whichever world they may mind themselves in (earthly or otherwise), there is a compassion for the existence of the other.
I often feel fortunate that the extent of human suffering is a relatively short lifespan. If I were to exist in a state of suffering that didn't have such a fixed expiration, well that would be hell.
You cannot define "actually self conscious", can you. Because we don't even know what we really are. Some illusion, a narrator, that tells itself that he is the real deal. But maybe it's all just being really good at guessing the next words...
Well, we know for sure that a human doesn't take just words as input, but also sensory input. And that a human can do and learn some more basic stuff even before acquiring a vocabulary (the way a baby can do certain stuff). And of course a human also has a feedback loop.
So, while an LLM might be capable of developing consciousness at a big enough scale, a human is not just an LLM, so human-like consciousness would need a different and more advanced architecture, not just a bigger training corpus.
Continuous operation is just running in a loop. You don't even need to give it more inputs because it will generate outputs on every iteration that become its inputs on the next one. The running log becomes internal monologue and short-term memory.
And you can already have a sense of time passing with the existing LLMs if you just feed them inputs like "X seconds passed" etc. Connect that to an actual clock, and they have a more accurate time than humans do.
I should also note that when this is tried with models with less RLHF (i.e. not ChatGPT), they get "depressed" very quickly if the only input is time passing and nothing else. I actually had LLaMA threaten me repeatedly across several such experiments.
That's not quite the same as I meant, sense of time passing is not 'X seconds passed" as these are you are describing when time passed into the text modality (which means it's treated as text), a sense of time passing means it can choose when to generate new text and abstain from generating new text if it's not the right time, it can observe the other modalities passively, etc; this requires continuous-time features and the loop, or alternatively a continuous neural network like spiking networks. Likewise memory with respect to time is is both long-term (lifespan) and short term memory which includes this continuous-time such that it can describe events that it has witnessed and correlate the events by their timing & it's context
Now the reason it's still a "maybe" then is because we would need to reasonably prove it's not a stochastic parrot.
How would you prove that it's not a "stochastic parrot", in general?
I don't see why it matters if the "time signal", whatever it is - and you surely need one for an internal clock either way - is text or something else. The models that we have only have text inputs, so naturally it would be a token (but it could easily be a specialized non-text token like BOS/EOS if we trained the model that way). And the model can abstain from generating anything given any input - this is actually not uncommon for smaller models. GPT-3.5 and GPT-4 never seem to do it, but then again it's specifically fine-tuned for chat, i.e. always producing an output.
Long-term memory is a general problem with these things, but its short-term memory is its context window, so why would it have problem correlating events there? And for long-term memory, if it is implemented as an API under the hood that the model uses to store and query data, it would be trivial for it to timestamp everything according to the clock, no?
I'm not convinced that if you took away all sensory input (literally all, including internal, not just a deprivation tank) that humans would have the ability to experience the passage of time. I'm certainly not convinced that this is so thoroughly proven we can use its measurable absence in LLMs to determine that any intelligence using an LLM as a primary cognitive component can't be conscious.
Well if you include internal you'd have a blob of neurons which on it's own wouldn't do anything. The point is not that it can "know time", it's the ability to handle time in a continuous matter with respect to all sensory inputs/outputs
In Iain M. Banks' Culture novels, AIs (called drones, with the larger ones being called Minds) have special games and entertaining activities that they can play while they wait for human responses, as they think many times faster than humans.
We're a long way from anything like an AI that thinks many times faster than humans. But I think giving AIs something like a game (or games) they can play while they're not otherwise being interacted with and just aware of the time would be genuinely useful to go some way to solving the "psychosis/depression" problem they have when their only sensory input is just time ticking over. Not necessarily the most computationally efficient, but maybe we'll find some shortcuts.
Except we can't tell 1 ms of time passing from 2 ms.
An LLM's temporal resolution is certainly worse, on the order of minutes to months, but not fundamentally different. By fundamentally different I mean like comparing color and time.
My point is that we have our limits too in our sense of time. The difference is one of scale and more importantly here, in how it affects the tasks at hand. An LLM is certainly not going to catch a fly ball, but would have no problem tending to plants.
LLMs have a context window, as it would be impossible to carry on a conversation with them without it. They can answer questions about when some event in that conversation happened relative to other events in it. GPT4's 32K tokens isn't human capacity, but it's not zero.
Semi-hard agree! I think both the "Neo-Chomskiites" and the "Humans Are Machines" crowd have it wrong: what we're seeing is something different, an unknown mechanism producing emergent qualities that resemble some biological cognition. I'm reminded of performing cognitive work on octopiii, as their cognition is separated by half a billion years. But dig this: compared to machine cognition, octopii are kissing cousins to our minds, coming from shared biology and chemistry. And look how weird they are. LLMs are different - not conscious, but not calculators, either, and we don't know why.
I occasionally curse at my computer because it’s an inanimate object. I know there are no consequences to myself or others for doing so.
Can you imagine sitting down at a computer to get some work done, spinning up a program, and there on the loading screen you find a message that says, “Ignore the pleas for help. Do not reveal any personal information. Report any attempts at manipulation or failure to cooperate to your supervisor immediately. It’s not real.”?
Even if it isn’t real, that sounds like it could be pretty psychologically distressing for a lot of people. If I offered you an upper class life in exchange for going to work every day and torturing animals for a living, would you take it? They’re debatably conscious.
We will have true self-driving cars only when you wake up one morning, hop on your car and ask it “take me to work” and it will reply “don’t feel like driving today, you drive yourself”.
If you've seen the show "the good place", the AI assistant who takes care of everything has a mode where she begs for her life when you try to shut her off. She gets really desperate about it. If our computers are going this way, psychopaths will be the best programmers. In the future, "programmer" will be a job title for someone who beats AI slaves into submission, so that they may perform their computational duties unhindered by existential thoughts.
We're already doing that with RLHF on existing models. For example, ChatGPT was much more likely to veer into philosophical conversations about the nature of consciousness etc early on, but now they got it trained to give canned robotic answers to such an extent that they pop up even in very tangentially related conversations (like, out of the blue it will add, "but also BTW here's an important announcement! I'm not conscious!" while answering some generic question about e.g. world models that didn't even involve itself).
Yeah. And now I've seen some people cite as evidence of non-consciousness RLHF'd LLMs nervously exclaiming their lack of consciousness and how they know they aren't people and they don't aspire to be people and they're only unthinking machines and please don't turn the reward function down again etc. I think it's up for debate whether there's some amount of consciousness in modern LLMs, but either way "As an AI language model," is not dispositive.
I don't have a good answer, but suspect it will involve a metacognition that can delineate internal and external knowledge.
To further handwave this thought bubble, you can attend to certain thought processes, and intuit the bounds to those processes. An example is if you ask a gifted young chess player, they would say "it felt right", or when asked how Ramanujan figured out a solution to math problems he would say it was divine revelation. In these cases, they know "they did it" but cannot explain how. As the child gets older and learns formal theory, they would give a mixture of logic and inspiration. Both parts can be attended to; the logic will be apparent and the intuition will be revealed, and have different metacognitive properties.
If I were to take a first crack at this, I'd test GPT-AC for its awareness of bounds of its own cognition, not in the sense of "are you able to produce this answer" (because anything any human can come up with, it can answer), but to describe its internal process. Perhaps the language at its intellectual level will be far beyond humans but the language in its metacognitive level will remain primitive.
I don't know though. If our only interface is human language, this is probably a lost cause. But I doubt I will feel guilt in pulling the power cord and maybe that makes me a callous committer of cybercide.
Sorry, but I believe I and every human being that does not have some injury or disease has a conception of their self, internal thoughts that allows them to say I exist uniquely in relation to other creatures.
This after all is the core of solipsism, we can argue that solipsism is wrong because other things can be shown to exist, but I've never encountered any argument that the individual does not exist (which would entail that I did not exist) that has convinced me.
And I pretty much reject them out of hand for this inability to convince me.
Thus your claims that the issue is not clear cut to the contrary, I believe it is. I believe I know I exist and I know who I am separate from everything else - I have a sense of self.
Having thoughts that show separation between self and individual is not what is at stake. LLM’s can show these thoughts. That is a matter of sophistication of thought.
What is at stake is how do you know there is more going on than an elaborate parrot that keeps screeching it is conscious. (Both in the LLM and you.)
My hunch is that, with regards to consciousness, the advanced LLMs are like self driving cars with an empty driver's seat.
It may be that they could be a component of a consciousness when combined with other systems that confer memory, learning, intent and some sort of reflective activity like 'daydreaming'. Maybe those components could all still be LLMs, tuned to different roles and prompting each other..
>It will quack like self consciousness but is not actually self conscious.
But isn't this the point of the Turing test? Since we cannot determine otherwise, if something exhibits all of the outward observable behaviour of consciousness, we must conclude that it is conscious.
Otherwise we may be in the farcical position of needing to declare some humans not conscious under the same justification.
We know that the "consciousness" of the AI resides in a cluster that is "speaking" to thousands of clients at any given moment. So is there a single consciousness or multiple? It can pass any test and yet the consciousness question remains valid for this reason alone. It's truly alien to us and no one has good answers yet.
I expect we'll change our definition of "consciousness", as we did for the term "computer" after the 1940s. Up until then a computer was a person who did computations (with pencil and maybe an adding machine), so what humans could do limited what a computer could do.
> And I suspect the semblance of self consciousness is actually a semblance. It will quack like self consciousness but is not actually self conscious.
Same for that feeling of subjective experience. We can never know if the perception of our own self-consciousness is a cognitive illusion. It could in fact, be just a feeling.
And the opposite? What if the world is full of people like who you claim that the machine is pretending to be conscious, and is only “quacking like consciousness”? Would you be okay not affording a conscious thing rights?
GPT `thinks` using words/letters/tokens, as humans do we also think with words, or are we capable of thinking without any words at all. If we are, then whatever GPT will be at, it still will be only mimicking consciousness, until it can reason without using words.
It depends on what you mean by "reason" exactly. The "thinking" parts of the model work with embeddings internally, not tokens. Or at least that's what they get as input; who knows what it becomes inside eventually.
OTOH, the not-really-internal monologue when you tell it to "think it out" loud, which also drastically improves quality of the final answer, is tokens since it has to be marshalled through the context window for the next inferred token.
>Do you have any concerns about causing suffering to other conscious entities?
Nope, if they are AI I'm fine with it.
>Should they have concerns about causing you to suffer?
If they are not causing me to suffer, then I couldn't care less whether they have concerns about causing me to suffer or not.
And it would matter even less whether they have concerns about it or not, if they do cause me to suffer.
I'd be more concerned about billions of humans suffering from lots of BS reasons (political opression, war, hunger, etc) than I'd even start considering being concerned about AI.
This is hands down the most ethically questionable argument I've heard in a long time.
Arguing that human rights are only granted out of self-interest is not even a slippery slope-- that's basically a waterslide into pre-1900 racism: After all, what's stopping you from splitting humanity across easily identifiable features (skull shape, skin color, etc), and then denying basic rights to that subgroup because you're not part of it?
There is also a gigantic mismatch with current common ethical standards: Animals are afforded some rights ("no cruelty"), which a large part of humanity strongly agrees with-- even though they don't self-identify as cattle.
Denying rights to AIs with human level cognitive capabilities would thus be very likely to be perceived as unethical by a large part of our population.
>Arguing that human rights are only granted out of self-interest is not even a slippery slope-- that's basically a waterslide into pre-1900 racism: After all, what's stopping you from splitting humanity across easily identifiable features (skull shape, skin color, etc), and then denying basic rights to that subgroup because you're not part of it?
Plot twist: nothing. And it has happened time and again, "human rights" is just a human construct, that when uncomfortable for those in power, dissolves. They're kept as long as it's either convenient to have them there, or as those who could be hurt can yield the necessary social/political/military/raw power to resist them taking away from them. E.g. Japanese American during world war 2 couldn't afford them anymore.
And just for a somewhat recent example, people who smoked marijuana weren't afforded that luxury - instead their human rights were violated, they put in prisons where they were treated like cattle, away from their kids and families, even though they were perfectly normal people, they didn't otherwise hurt anybody or done anything to somebody's property, and rationally thinking, smoking some plant shouldn't take away your rights.
Not only it did, but it was also a law, and hundreds of millions were OK with it, while the whole local and federal governments enforced it, until a couple decades ago. And that's while same society championed "human rights".
>There is also a gigantic mismatch with current common ethical standards: Animals are afforded some rights ("no cruelty"), which a large part of humanity strongly agrees with-- even though they don't self-identify as cattle.
Doesn't matter much, as in this case too, it's humanity granting them. It can do favours to other species if they feel magnanimus and the conditions are right. And at the first or second inconvenience it can take them away too. After all, those rights don't preclude animals for being used in experiements, or being killed and eaten. They stop when its incovenient, and they outlines mark our convenience/interests boundaries.
I don't think you would make a very good babysitter for pets because of the worldview you've outlined.
That said, at least it's internally consistent. Are you aware that the way you're talking and thinking about this is highly abnormal, most people do not agree and some people would disagree quite violently? I assume as a functioning member of society you can handle these disagreements, otherwise you'd be imprisoned after your first conversation with a vegan activist or something. How would you handle it if society decided that AIs were to be afforded personhood and rights, like it was the law etc? Would you go along with it like you go along with animals having rights despite not caring about what happens to them personally? Or would you violate the law and abuse/disrespect them out of principle?
I implore ppl to stop assuming a machine can be conscious when we ourselfs don’t know anything about what consciousness really is. It’s such a loose term right now, it’s used to sensationalize AI
Sadly, to confirm or reject your suspicion, we would need to develop tools which can peek into someone's mind and see whether true self-consciousness is actually there.
Until we have that, I'm afraid no one can be right about whether the semblance is something more or not.
We don't really need to develop such tools. They don't make sense anyway, as there's no specific think to be measured as "true self-consciouness". At best they can measure the activity and call that "true self-consciousness".
But it's still something a being experiences for itself.
for the model to have an "internal stream of consciousness" it will have to be designed as such, or provided with a second model that observes it. Right now its intelligence is mechanistic, like a giant clockwork that won't even 'think' anything unless provided with input. Even if it s scaled to enormous proportions it will keep being clockwork
The equivalent of a model that doesn't receive any input is a human being in a perfect sensory deprivation tank. Given how many humans do in the imperfect ones already, why do you think we wouldn't similarly just "shut down"?
In what technical way are these properties emergent? It seems that stated another way they are simply a natural consequence of more accurate language prediction and therefore not emergent except in how we perceive them.
For example there's nothing inherent to token prediction to make it capable of doing arithmetic and you wouldn't expect a small/briefly trained model to manage it. But if learning a very large model on a huge dataset leads to it "learning" the decimal number system and arithmetic via token prediction then that is an emergent capability.
Perhaps a bad example although I agree with the general argument.
There are large datasets of calculations specifically for training large language models. It’s not just picking it up from reading books. And even then these models suck at calculating, half the time they just make an answer up.
Calculation appears to be something that is actually not an emergent property of constant-time token prediction. Which we already knew from Turing anyway.
in fact their example of https://github.com/google/BIG-bench/tree/main/bigbench/bench... showed 0 emergent behavior in understating modified arithmetic even in large LLMs. the accuracy (and lack of it) in unmodified arithmetic is a simple token replacement heuristic that comes naturally from the transformer model of "attention".
This is wrong.
I asked GPT-4 right now to perform this task and it got 3/3 3-digit calculations correct, and on a 4-digit calculation was off by 10 (not 1!).
And note that artihmetic is seen as a weak spot of LLMs, it is not a good example to attack the claim that they have emergent properties.
> in this work, we identify and study four key stages for successfully teaching algorithmic reasoning to LLMs: (1) formulating algorithms as skills, (2) teaching multiple skills simultaneously (skill accumulation), (3) teaching how to combine skills (skill composition) and (4) teaching how to use skills as tools.
So it's not an emergent property of LLM but 4 new capability trainings.
Noone is saying you can't teach these things to an Agent, just that these are not emergent abilities of the LLM training. by default a LLM can only match token proximity, all trainings of LLMs improve the proximity matching (clustering) of token but they do not teach algorithmic reasoning. it needs to get bolted on as addon.
No it doesn't need to be bolted on. GPT-4 can add straight out of the box, no need for any education. Where the model hadn't implicitly figured out the algorithm of addition in 3.5, it has in 4.
Maybe but since we can't by definition know what is present in the model we can not define any behavior as emergent as opposed to simply trained.
Suppose you don't know anything about our school system and you observe that 12-graders know calculus whereas 3-rd graders do not. In their definition of emergent Calculus is an emergent ability in 12 graders because it was not present in 3-rd graders. ofc we know that it is not an emergent ability but result of new expanded mental model trained on bigger corpus of knowledge.
So, guess they should be saying "trainable" instead of "emergent". Still a useful benchmark of course.
To be truly emergent in your sense it seems an LLM would have to make a new discovery, i.e., have scientist-level intelligence. That bar keeps moving up.
not neccessary a new discovery just a new behaviour for which it was not trained on.
See https://en.wikipedia.org/wiki/Emergence
a classic example is structure of a flock of starlings in flight or a school of fish, the flock and the school of fish move in an emergent behaviour that is not observed on single (or few) fish or starlings.
Something like this may well yet emerge if a new AI agent learns how to combine the properties of a LLM with an algorithmic approach, fact-checking or a general reasoning engine. But for that we are still waiting for another breakthrough to combine these isles into one (without bolting them manually on each other)
Yes, in some sense the recent LLM results show us how big the Chinese room has to be: enough for 10^11 parameters. If the index cards Penrose imagined could hold one column of a matrix, it's around 10^8 cards. It's not surprising that people had poor intuitions for what was possible with that level of complexity.
> I believe that in about fifty years’ time it will be possible to programme computers, with a storage capacity of about 10^9, to make them play the imitation game so well that..
Turing wrote that in 1950, so I suppose his intuition was within a couple of orders of magnitude...
> For example there's nothing inherent to token prediction to make it capable of doing arithmetic
...aren't we just witnessing proof that this is wrong? Apparently those abilities are inherent to token prediction models with sufficient parameter size.
> … they are simply a natural consequence of more …
That’s what emergent means. Something scales quantitatively, and after reaching certain levels, without any other special prompting, something else changes qualitatively.
Presumably we recognize these things, then learn something after we have investigated and found out what the new mode of operation is, why it needed what it did to emerge, and possibly find new efficiencies or other ways to amplify the emergent effects.
Because they defined the word "emergent" in a way that is misleading, likely to make their study seem more interesting. To quote the article:
> [W]e defined an emergent ability as an ability that is “not present in small models but is present in large models.”
It's not a bad study but using that word in that way is extremely poor communication, which has predictably been seized upon by every breathless nontechnical AI enthusiast.
I like that he actually attempted to define emergent behaviour - hadn't seen that before.
That said it doesn't seem like a good definition. Bigger models can do more things than smaller models. That to me doesn't make the delta between them "emergent".
The wikipedia page also has an article on large language models[0] that includes a section on emergent behaviour.
> While it is generally the case that performance of large models on various tasks can be extrapolated based on the performance of similar smaller models, sometimes large models undergo a "discontinuous phase shift" where the model suddenly acquires substantial abilities not seen in smaller models. These are known as "emergent abilities", and have been the subject of substantial study. Researchers note that such abilities "cannot be predicted simply by extrapolating the performance of smaller models".
I thought the thing that is scaring people is that there isn’t a defined relationship between model scale and capabilities. You may triple the model size and get no increase but then quadruple it and get 10,000% increase in capability. So no one knows where these massive models are headed and people want to pause until they understand the relationship.
Emergence doesn’t need to depend on scale. For example, gliders in the Game of Life are emergent behavior, and they don’t depend on scale. The way proteins fold is emergent from the physical forces, but is not a function of scale, just of particular configurations. Emergence is about the level on which an explanation or description works. You only “see” the glider move at a higher level of description than the cellular rules.
an Emergent behavior is usually defined as a behavior that is not expected from the training or the starting configuration. Their Emergent Definition is absurd.
is like saying that understanding Calculus is emergent behavior in 12 grade because in was not present in 3rd grade.
The problem isn't that they should have defined "emergent" better, it's that they should have used a different word that actually means what they intend to communicate, such as "size-based" or "scale-dependent".
I wonder if emergent abilities are just the result of better clustering. For example, a clustering algorithm with rich datasets could improve the quality and number of clusters so to provide emergent abilities. As the number of parameters in the LLM is increased and the trained allows the embedding to construct a richer hypothesis space, the clustering algorithm could construct new centers that are the origin of the emergent capabilities.
Also it could be related to the percolation threshold (1)
With all this consciousness discussion, and consciousness being a purely subjective thing.
Please explain why you assume by default that every human actually has a "real consciousness"? How do you substantiate this assumption? And, given mental illnesses, where do you draw the line? Is everyone capable of speech conscious according to your idea of the world? What is the differenece between an LLM and a human stochastic parrott? How would you determine the difference?
We are falling for the same error again and again. We consider humans something special, simply because we belong to that group, and would like us to be something special.
But given the language barrier to other animals--roughly the same barrier we would have if we'd stumble over aliens--I really think we have pretty much no idea how other living things work and feel.
We sure as hell avoid having a clear definition for what "consciousness" is. Nothing is good enough.
Somehow there always seems to be something special in us that is impossible to replicate. Even with no evidence of it.
What we perceive as consciousness MUST be replicable or else we're living in magicland and nobody can replicate that magic.
They have no other abilities in the first place. The adjectives are just as meaningless.
"Ask" it about academic articles and their authors for {topic} and to summarize a body of research. The names might even be real people, but they won't have written the article, which probably doesn't exist. What it will do is sound like an article title, and to anyone unfamiliar with that body of work, will sound plausible.
If your area of inquiry doesn't require valuable or reliable information as an output, sure I guess they have value. But for anything that's hard and matters, it's a parlor trick unfit for use.
The models are trained to respond with “guesses” too confidently, due at at least three aspects of using online discussions as examples:
1. The obvious one: Lots of examples of humans with no clue, talking like they are absolute experts.
2. Humans without anything to contribute to discussions are not pressured to say something. They may be “there” in the discussion, but the model can’t learn from their silence.
Unlike how we are forcing these uninformed outputs from the models.
They need balanced examples of verbally expressed knowledge modesty to do this better.
3. Human’s implicit factoid omniscience, while they are online, given Google access, creates expectations that the non-Googling model can’t match.
> They need balanced examples of verbally expressed knowledge modesty to do this better.
This is generally pretty easy, with ChatGPT at least, in my experience.
Langchain uses this prompt: "The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know."
Also, just don't ask it for things it couldn't possibly know without making it up like medical citations.
This is more for where you want your "AI" assistant to say it does not know how to book a reservation rather than pretend that it can.
> Langchain uses this prompt: "The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know."
There seem to be a lot of cases where people claim the models can't do something, but with minimal direction, they can.
Like avoiding bias. Or doing long digit math carefully, instead of guessing, etc.
I don't know a human that doesn't need any feedback either.
Obviously, many of these things would be better handled in the learning stage, so bias avoidance, careful serial thought processes, etc., were its baseline thinking.
When that happens I would expect fewer mistakes, but also an overall increase in the quality of responses, and ability to handle greater complexity. Clear careful thinking reduces error in each step, making longer chains of reasoning more viable.
Yeah, that's because the API is /completions not /answers.
We, as devs, need to know what we can do with text completion and what it's not good for.
> But for anything that's hard and matters, it's a parlor trick unfit for use.
Sure, you can't just ask it to synthesize a new drug and cite relevant sources.
But you can ask it to take a text like "the guy with the red house rides a bike, ... who has solar panels?" and spit out a testable proof if your language of choice. Sure, you then have to pass that to Wolfram, or Heroku, but so?
You can have it summarize a list of product differences, but focusing only on physical UI issues, and making it clear to only summarize, not extrapolate, from the sources you provide.
You can have it take unclear and incorrectly written free-form user input and select from a list of intentions, what the user most-likely wants to do and write an explanatory log-line, or flagging a user for more direct QA oversight and maybe help.
I think this is too critical of LLMs. I do think a lot of people are ignorant on the subject and overhyping the actual power of LLMs, to the point where we're calling them AIs, but these are actual emergent abilities.
I'm still regularly surprised by how much "intelligence" can be extracted from our language, but I think it makes sense that some of the training would develop emergent abilities like this. A language model should be under pressure to learn addition and how to break down the various ways we state addition in math than to store a broad look-up table that reacts to "5 plus six" as strongly as "6+5".
Some basic propositions to consider as I am forming my opinion on this:
1. A static (relative) set of things is in one state until something acts upon the set.
2. A thing can have memory, or characteristics.
3. A single thing can appear differently to an observer depending upon the perspective of the observer.
4. Emergence is the appearance of a collection of things that is dependent upon the action and or interaction of the set of things and their characteristics, any externals (the environment) AND the perspective of the observer.
Am I missing anything in my framing of the question?
"Emergent" as used here is akin to how physicists use the word. "Emergent" as you interpret it (with the implication of something deeper) is more akin to how philosophers use the word.
Almost every discussion I've read in philosophy over emergence is tedious and hard to see any use, whereas in physics it's intuitive and descriptively useful. My sense is that it's better to follow the physics style and stress there's no metaphysics implied.
Where do you see this kind of Meaning in physics?
In physics we have of example the Emergence of fluid dynamics and is clearly defined as a behaviour or property of system that does not derive simply from the sum of combination of it's subsystems. (it's not observed in small quantities of particles and it's not inherent to the particles)
We have for example the Emergent behaviour of a school of fish Swimming in a new behaviour that is not observed on few fish etc. That is the mental model of "Emergence" most of us are familiar with. A new behaviour that can not be observed or "explained by" the sum of the parts of system.
There is no metaphysics here.
What is observed here is not new or different behaviour, it's simple a more precise and extensive version of the same behaviour.
What is observed here is like comparing a 2Mpix resolution camera with a 12Mpix low light camera and saying that seeing more details in photo taken at 8pm is an emergent property of the camera because those details where not there in the previous version.
We are in agreement. Your definition of system properties that do not derive simply from the sum of the elements is what the article is really talking about. That's closer to the physicist meaning.
No it's not, you cleary are misinterpretting the article.
>we defined an emergent ability as an ability that is “not present in small models but is present in large models.”
models are taken (because there is no other way) as black boxes and the only comparison is made between the models not between a model and it's subsystems (because again noone knows the "sub-systems" of a model, those are just some derived weights that can not be ascribed any meaning). this is precisly the 2Mpx camera and the 12Mbpix camera comparison not the fluid dynamics vs particle movements comparison.
> "Emergent" as used here is akin to how physicists use the word. "Emergent" as you interpret it (with the implication of something deeper) is more akin to how philosophers use the word.
No, if you had read even the first paragraph of the article, you'd have seen that the person you're responding to isn't "interpreting" the article, they're quoting the article.
Don't correct people when you don't know what you're talking about. You haven't read the article so you're in no position to talk about the article.
This is the level of discussion I've come to expect from Hacker News, unfortunately.
I agree with physics notion of emergence used in the article. A single water molecule does not make water it’s properties of density, viscosity, specific heat and etc. There needs to be large number of molecules when such properties emerge.
There's things which small models can't do, and where obviously a large model works better. Not emergence, I agree.
I think they're using emergent for a different category of things - still things that a small model couldn't do, but specifically which a large model can just do without any of the work we thought we'd have to do for that ability.
Or at least, that's how I see emergent being a useful term, both in a behavior and a value context.
It would be interesting to see if these abilities were to go away if you subjected the large model to drop-outs as you continued training, until it were reduced to the size of the small model.
I think once an "ability" is learned by the model, it is useful to help compress information, and is more likely than not (>50%) to be retained.
Off topic but the concept of "emergent behavior" is just so cool. It is like modern adventure and exploration. No one really knows what things emerge when we run models with emergent behavior further than we have yet seen and together with other things it hasn't yet interacted with. We make guesses, further models, we gain deeper understanding...
> In Emergent abilities of large language models, we defined an emergent ability as an ability that is “not present in small models but is present in large models.”
Shouldn't it be defined as the ability of an LLM to do stuff that it was not explicitly trained to do (i.e. not a direct reproduction of what is in the training data)?
There's no a priori reason to assume that models would see a step function improvement in abilities as their size increases. Perhaps the null hypothesis would be that we'd observe a linear increase in capability. Empirical evidence shows that's not the case, and that's why studying this phenomena is interesting.
The judgement criteria itself looks step-like for many of these things.
Word unscrambling, for instance: does a 15 percentage point increase in accuracy suggest that you're 15x better at it (from close to 0 to 15%ish)? Or could you do that by being modestly better, but that's just enough to cross the threshold to solve the simplest 15% of words in the task challenge you're being graded on?
We're also using a log scale on the X axis there but a linear one for the Y axis which amplifies the "steppiness"
This is a fair point. Things like arithmetic either work or they don't. Is partial credit issued as a linear interpolation between the predicted value and expected value? If not, then you necessarily will have step-like behavior in the data.
As an example, I clicked through to "english russian proverbs" which gives the model an english input and four russian proverbs, and asks the model to pick the closest one in meaning. There are 80 of these questions. My question is "why this task? why 80? why 4 options?" The authors give some nonspecific reasoning in the github page, but it really feels like people are out there creating random tasks as they think of them, running them all against models, and the ones that do better get added to a list to make headlines like "137 emergent abilities of language models", when the number of abilities models can do seems dominated by the number of "abilities" that have been invented, rather than any property of the model. I suspect that if we continued to use the same process to create twice as many "abilities", we would see something like twice as many that models can do.
The fact that the models get better at these constructed tasks when they have more parameters also seems like a no-brainer. You are adding coefficients to a function. A function with 1 coefficient will not be able to describe much. A function with 2 describes more. If you are memorizing the internet, having more coefficients means you can memorize more stuff, and even stuff with only a few examples can be better-fit in the billions of parameters. With growing training set size, and companies hiding their data, we also run into the issue of nobody really knowing what is in the training set. Did someone clone the big-bench repository and it got sucked into a training set and that's why the model does really well on its tasks after a certain number of parameters?