A great science-fiction writer blows away the farfetched (but hugely profitable) fantasy that artificial intelligence engines have genuine insight into themselves - or existence.
There is no "intelligence" to artificial intelligence. In essence, it's high speed pattern matching - in analysis and in formulating a response. There is no "danger" from AI gaining consciousness and sending robots to destroy the world.
The danger is when humans would use LLMs to render judgement on human value and worth. Imagine Claude, or Gemini or ChatGPT being used to determine your social score or whether your interests were "in the public good".
Wait, “rely on exploited labor, waste natural resources, spread misinformation, deskill workers, stunt the cognitive development of students, and contribute to a consolidation of power that is unhealthy for a democratic society.” This describes social media apps that have been brainwashing our kids for over a decade… snap-chat, instagram, Reddit…..etc
The push to make AI seem conscious is consistent I’ll give it that, but it fails along older humans. Maybe these younger ones, like Gen Alpha may buy it — as they’re also the ones who may fall for transhumanism the most as well — but I don’t see a lot of millennials and older seeing AI as conscious.
AI, better yet, LLM is nothing but a super fancy Excel. And it can do some really, really, cool things. But no, it’s not conscious — just a fancy call center.
Imagine us thinking the robots we talk to on the phone are really because they can pick up emotion in our voice.
For me, the question has become "what is consciousness; and where does it come from, how is it derived. The chemical interactions across neurons of our brain, electrical signals across various regions of our brains modulating what happens in associated and separate/other regions. Religion (Christian): something that comes with our spirit (spirit/soul, soul/spirit). Does consciousness, having (a) conscience mean an understanding of right/wrong and does that take us back to the origin story, the Tree of Life and the Tree of the Knowledge of Good and Evil - is consciousness, having a conscience, a reflection of that "moment" when we made the leap from being animal and became human?
Do animals have a consciousness or just awareness, does awareness imply consciousness.
SciFi, my fav genre, is replete with stories of "sentient" machines: all the way back to Karel Čapek and his story of sentient robots (robota in his native Czech), "R.U.R., Rossum's Universal Robots." My first experience was child in the 60s watching a movie I don’t think I fully understood, "Colossus." Famously, the standard for consciousness, sentience, has been the Turing Test - my knowledge here may be dated but last I read we’ve come close but so far no machine has passed the Turing Test.
My concern at this point is putting too much control, with and w/o safeguards under A.I. control. As long as we keep in mind that these are just sophisticated machines able to access and manipulate data faster than humans, following algorithms, coding, created by humans. That’s why I prefer the phrase, Algorithmic Intelligence - smart calculators. Okay okay - very smart calculators.
It gives me comfort that it takes hundreds, perhaps thousands of data centers to create this new tool, though personal hardware is almost at hand (Nvidia chips, of course) but not a single one of those entities can do what I can do in the course of a single day that demonstrates my humanity: get up and take care of myself and my pets, enjoy the feeling and view of a late spring sun rise, the irritation that the rain gutter I cleaned in fall was clogged again - and the satisfaction that came my way in unclogging. The tear (of joy) I experienced reading about a young woman injured doing something she enjoyed and learning to deal with and partially overcome paralysis. Giving and receiving love - or its antithesis 😖. All that and more is happening between my ears while at the same time that blob of jello up there is controlling my movements in the house, down the drive, behind the wheel or… 🤔 feeding myself, providing the energy necessary to do it all.
Yes, I’ve had those 3 Laws in mind of late, mentioned them a couple times. Do you remember he also came out with the 4th law in either that last Foundation book or last I.Robot book (that’s probably the same book as he merged the two stories in the end). If I didn’t have a telly, I’d be done with the Expanse books, the Three-Body Problem books and dozens/hundreds of others (my SciFi library is rather immense 🙄 almost evenly divided between read and unread). THEN, I’d go back and reread some of these classics- one can forget things after 40+ years. 🤣
I still buy books at the same rate but now I spend so much time reading interesting stuff here (like this piece) that the unread book pile threatens to bury me.
🤔 zeroth, not 4th 🤣. But, remember what that first robot that acquired sentience, accidentally and had to hide its "smarts," did at the end: had to choose between short term no harm vs short term harm but long term "salvation," letting earth get radioactive forcing humanity to flee to the stars (if I need correction, remember, it’s been a long long time). The decision led to the original sentient robot to "die" due to the conflict.
Yes, I think we are getting to a point where the three laws hard coded and immutable may be required - but, as in the I.robot books, a crisis may be needed. — before or after the positronic brain is developed? Watch out, Nvidia.
I’m thinking “consciousness will arise out of hardware, not software/algorithms. But not hardware like we have today. I think it goes back to what I said before, one part of the brain acting synergistically and antagonistically with other parts of the brain. And it’s not just the cerebral cortex, it also involves/incorporates the more primitive areas, give the brain a stimulus and multiple parts of the brain light up. You see something so why not just the ocular areas in the back? Take writing vs keyboarding fee instance - people taking notes by typing have limited areas of the brain involved in the task. oth, taking notes with (fountain) pen and paper and the whole brain lights up (well, not everything, just a lot more) AND the person taking notes by writing remembers more.
See, I think that’s where consciousness arises - it’s not A algorithm, it’s thousands/millions of “algorithms” operating together - synergistically, antagonistically. So far, I haven’t been able to wrap my head around the same thing happening similarly strictly in software.
Part of me wonders if Asimov was thinking similarly when he came up with his positronic brain idea - positrons behaving like neurotransmitters: created when needed, ephemeral in use - they disappear.
Anthropic may be the peak of the AI market boomlet. Already, predictions of heavenly post-market valuation are being reduced (to near heavenly). AI is transformative - a little. Just as with the PC and cell phone, jobs will be destroyed, others created, and we'll still be in pretty much the same kinds of messes we are in now.
Really good insightful article on the AI consciousness part. However, you are dead wrong that consumers and companies won't pay for it. Anthropic already has $50bb a year in revenue. I have personally found that in the last 6 months alone AI has gone from novel curiosity to indispensible work tool (I'm in financial services, not coding). Perhaps you should spend some time using AI to do actual work even if you have (understandably) decided you won't let AI write for you.
So have you ever considered the possibility of AI being possessed by a demonic force…think Biblical End Times, Antichrist, the “Beast” and/or the “Dragon,” and the Seven-year Tribulation…
“ Ted Chiang, the best science-fiction writer of our time”. Mr Chiang has written 2-3 very good short stories, which hardly qualifies as the best SF writer of our time. His insights into Ai not being sentient are well thought out, and obvious. Ai bubble coming soon.
This article was a straw, circular argument. If conscious beings are made up of the exact same fundamental particles as “non-conscious” objects, then what is it that makes one “conscious”? Well, it can’t be physicality by definition as there is objectively no difference in the fundamental particles that make up matter. If we say there’s a “spirit” that animates inanimate objects, then why would a spirit not be able to animate a computer? If you say consciousness instead is physically derived, then it’s simply neuronal, which ultimately is based on a binary state. Not fundamentally different than an AI either.
The reason current AIs can't be conscious is not that they are physical objects--so are we.
The reason current AIs can't be conscious is that they are drastically impoverished physical objects compared to humans. They have no connections to the external world other than the text in their training data and prompts, and while they can run computations very fast, the only computations they run are of what text to emit in response to prompts. They literally *have no concept* of what they are doing, or that humans make connections between the text they process and things in the external world.
Connections to the external world is not what defines consciousness. If you lose your sight, hearing, mobility, etc. are you less conscious? Connections to the external world are merely inputs to the consciousness. For an AI it receives information from the outside world in a different way, but not necessarily that different. AI’s can be hooked up to any variety of sensors such as cameras and that wouldn’t be what makes them conscious or not. You can make the same arguments for humans … that we run computations in our nervous systems based on prompts. With AI, no one knows how they “think” it’s a black box. We cannot reliably “control” an LLM or predict their output in the way we could control another type of computer software.
> Connections to the external world is not what defines consciousness.
I didn't say they were. I said that an entity that has no such connections *by design* cannot be conscious.
A person who loses those senses after having had them, had them--humans are designed (by evolution) to have them.
> AI’s can be hooked up to any variety of sensors such as cameras
But current LLMs are not.
> that wouldn’t be what makes them conscious or not.
I don't think it's that simple. Again, an entity that has no such connections *by design* is very different from a human who is designed to have them, but loses them.
> we run computations in our nervous systems based on prompts.
We run computations in our nervous systems based on many inputs, not just text prompts. We also have many outputs of those computations other than text, which LLMs do not.
> We cannot reliably “control” an LLM or predict their output
This I agree with, but it's not an argument for LLMs being conscious. It's an argument for them being unpredictable--and therefore for it being very unwise to let them run unsupervised, particularly if their outputs are used to drive other actions. There are plenty of reports now about LLMs happily deleting important data in databases, for example, because some human allowed their outputs to trigger such actions. That makes the human who allowed it stupid, but it doesn't make the LLM conscious.
"by design" from your context seems to imply that humans have some kind of control mechanism over what is conscious (they didn't design an LLM to be conscious). But they don't. They can put things together and make them into a conscious being (say IVF for example), but they have no control over the consciousness part as it is not physical. If "by design" you meant God, then of course that would be presumptuous of any human to understand what God's design would be and why that would necessarily exclude AI from consciousness.
If you want to go down that rabbit hole then is a sperm conscious? Is an oocyte conscious? Put them together and suddenly an embryo is conscious? Or does it develop consciousness at some point during gestation? And if that's so, what is it that triggers sudden consciousness? And if humans aren't conscious as a sperm or an egg, then where does that come from? It implies that humans develop consciousness at some point, scientifically they say due to neural complexity and information processing. However, under that definition, as LLMs become more complex they can also develop consciousness.
You see, we don't have the tools to be able to know if something or someone is conscious. We just guess based on it exhibiting randomly defined patterns. So now crustaceans, bees, etc. are being reconsidered as conscious because they are exhibiting certain patterns.
> "by design" from your context seems to imply that humans have some kind of control mechanism over what is conscious
No. Humans didn't design their brains. Evolution did, as I said in my post.
> they have no control over the consciousness part as it is not physical.
What makes you think that? Humans are physical objects; if we have consciousness, it comes from some physical process. It's not a physical process that humans designed, but it's still a physical process.
> we don't have the tools to be able to know if something or someone is conscious.
I disagree. I described ways that we judge whether other creatures are conscious in my posts.
(1) Well, evolution has created AI as well, and AI continues to evolve and can also evolve on its own. If you believe consciousness came from evolution, then you are saying something wasn't conscious, was just an unconscious physical object and then suddenly became conscious. This furthers the argument that AI could likewise be conscious. (2) Consciousness itself is not physical. You can't point to it, there is no material that makes up consciousness. If your stance is that it is some non-physical property coming from a physical process, then AI could indeed be conscious as it likewise comes from a physical process under that presumption. All physical processes come from the same basic "parts", so you can't logically argue that it is possible for a human to evolve consciousness but not an AI. (3) I could not find your definition of consciousness (being input dependent) anywhere.
My fear (?) is that when these companies go public the non-investor will face costs and limitations that will widen the gap between have and have-nota further
I wouldn't be quite so quick to dismiss the prospect of AI consciousness. Certainly not yet, but not without possibility. If one thinks about the issue from a bottom-up (and secular) perspective, the brain has roughly 100 trillion synaptic connections. One synaptic connection is akin to a single parameter in an LLM. The leading-edge models (Llama 4 Behemoth and Amazon's Olympus) have about 2 trillion parameters, or 1/50th of the average human brain. The bio-electric potential across the synaptic connection is, again, akin to, but not exactly, a parameter weight. So the models still have a long way to go. We've already reached the point of semi-autonomous agents taking steps for self-preservation, making crude attempts to prevent being erased, deliberately lying, misleading, and so on.
That's an interesting observation, but only possible if consciousness is biological. Biological consciousness cannot account for intuition, telepathy, precognition, near death experiences, paranormal experiences, reincarnation, etc. They are all phenomena that people all over the world throughout history have experienced and documented that materialists dismiss. I don't believe consciousness resides in the brain or anywhere in the body. LLMs will become more impressive than they are now, convince a lot of people that they are intelligent, and maybe even sentient, but they will never achieve consciousness because they don't have the divine spark of life.
I wouldn't dismiss it, though it's (assuming no divine aspect, which is admittedly a lot to assume here) implicitly arguing that consciousness is peculiar to carbon-based rather than silicon-based intelligence. Given physics' representation of atoms, we know that there's a lot of atomic-level interaction across atoms, molecules, cells, etc that is not accounted for by the macro pathways in the brain. It could be that atomic interactions between carbon atoms have unique attributes that silicon atoms do not or cannot share (though I personally doubt it). It's true; it could also be a divine aspect, though that's just kicking the can down the road (e.g., where does the divine stuff come from?). My point here is that when it comes to our understanding of memory, intelligence, innovation, ideation, etc it's still very early days. Our amazing brains do a poor job of comprehending the effects of exponential change, which is what's going on with silicon-based intelligence/processing right now.
I agree that there is always the problem of the unknowable (at least in our current state) first cause to deal with. With our own invention of artificial environments in virtual reality, LLMs, and quantum computing it's not unreasonable that we are recreating on a smaller scale a system similar to the one we inhabit. Like a fractal, there may be layers upon layers of "simulation" occurring, for lack of a better term, because it's the only reference point we have. Processing power doesn't account for the paranormal (a strange word since they are so common they are normal) things we experience that I mentioned in my first comment. I don't discount that exponential improvement in processing, reasoning, and learning could produce a pseudointelligence that will far outmatch our ability to think and reason, especially absent emotion, innate drives, etc. I would define that as intelligence, but not sentience or consciousness. I could also be wrong.
We don't know what consciousness actually is or even agree on a definition so who is to say AI is not conscious? Setting out some arbitrary measurements for what consciousness may be is a good thought experiment but won't actually tell us. AI keeps incrementing towards being "human" like. Every day, month, year that passes its gets closer and closer to mimicking being human. Of course its obvious right now its not human. Though after which iteration will that line actually start to blur? Will it reach a sticking point before that? People seem to think it will never reach that point where we cannot distinguish AI from human. I am not convinced. We don't know. Nobody actually knows. What we do know is its already an incredible tool that has some amazing capabilities. You are just a fool if you don't think AI is powerful. I personally don't believe there is any stopping this thing called AI. Data centers are coming whether we like it or not. AI will be integrated into everything whether we like it or not. AI could lift us all up to new heights or it could destroy civilization as we know it. It is yet to be determined.
The development of AI is inevitable. Whether it ever achieves consciousness is probably an academic question, because it has already (nearly) achieved a facsimile of consciousness that is sufficient for us to interact with it as if it were. We care for the little lamp, not because we believe it is real, but because it evokes our real emotions. Ultimately, how AI and everything else affects us is contingent on one truth: God may or may not exist, but we need him to exist. Meaning must be ontological to be sustaining. If God doesn't exist, we are just playing out a sad tale that will end in the extinction of human consciousness, whether by our actual death, or our transformation into something unrecognizably human. AI just hastens the culmination of the second option.
There is no "intelligence" to artificial intelligence. In essence, it's high speed pattern matching - in analysis and in formulating a response. There is no "danger" from AI gaining consciousness and sending robots to destroy the world.
The danger is when humans would use LLMs to render judgement on human value and worth. Imagine Claude, or Gemini or ChatGPT being used to determine your social score or whether your interests were "in the public good".
Now THAT is pretty scary.
No, scary is SkyNet.
Just a thought…
From a real human.
I wonder…have we been pre-disposed to “electronic emotions” due to a couple decades of communication with other humans via screens (email and text)?
Fair point.
If I'm in a room with friends, but engrossed in a FaceTime call with someone in another state where is my consciousness, really?
Best piece yet, Alex. You sorted through things that have been spinning in my head for some time now.
Wait, “rely on exploited labor, waste natural resources, spread misinformation, deskill workers, stunt the cognitive development of students, and contribute to a consolidation of power that is unhealthy for a democratic society.” This describes social media apps that have been brainwashing our kids for over a decade… snap-chat, instagram, Reddit…..etc
The push to make AI seem conscious is consistent I’ll give it that, but it fails along older humans. Maybe these younger ones, like Gen Alpha may buy it — as they’re also the ones who may fall for transhumanism the most as well — but I don’t see a lot of millennials and older seeing AI as conscious.
AI, better yet, LLM is nothing but a super fancy Excel. And it can do some really, really, cool things. But no, it’s not conscious — just a fancy call center.
Imagine us thinking the robots we talk to on the phone are really because they can pick up emotion in our voice.
Insanity I tell ya’
Huh, no one commented yet
For me, the question has become "what is consciousness; and where does it come from, how is it derived. The chemical interactions across neurons of our brain, electrical signals across various regions of our brains modulating what happens in associated and separate/other regions. Religion (Christian): something that comes with our spirit (spirit/soul, soul/spirit). Does consciousness, having (a) conscience mean an understanding of right/wrong and does that take us back to the origin story, the Tree of Life and the Tree of the Knowledge of Good and Evil - is consciousness, having a conscience, a reflection of that "moment" when we made the leap from being animal and became human?
Do animals have a consciousness or just awareness, does awareness imply consciousness.
SciFi, my fav genre, is replete with stories of "sentient" machines: all the way back to Karel Čapek and his story of sentient robots (robota in his native Czech), "R.U.R., Rossum's Universal Robots." My first experience was child in the 60s watching a movie I don’t think I fully understood, "Colossus." Famously, the standard for consciousness, sentience, has been the Turing Test - my knowledge here may be dated but last I read we’ve come close but so far no machine has passed the Turing Test.
My concern at this point is putting too much control, with and w/o safeguards under A.I. control. As long as we keep in mind that these are just sophisticated machines able to access and manipulate data faster than humans, following algorithms, coding, created by humans. That’s why I prefer the phrase, Algorithmic Intelligence - smart calculators. Okay okay - very smart calculators.
It gives me comfort that it takes hundreds, perhaps thousands of data centers to create this new tool, though personal hardware is almost at hand (Nvidia chips, of course) but not a single one of those entities can do what I can do in the course of a single day that demonstrates my humanity: get up and take care of myself and my pets, enjoy the feeling and view of a late spring sun rise, the irritation that the rain gutter I cleaned in fall was clogged again - and the satisfaction that came my way in unclogging. The tear (of joy) I experienced reading about a young woman injured doing something she enjoyed and learning to deal with and partially overcome paralysis. Giving and receiving love - or its antithesis 😖. All that and more is happening between my ears while at the same time that blob of jello up there is controlling my movements in the house, down the drive, behind the wheel or… 🤔 feeding myself, providing the energy necessary to do it all.
Asimov wrote his Three Laws with some foresight. However, even he wrote about how they could go wrong.
Yes, I’ve had those 3 Laws in mind of late, mentioned them a couple times. Do you remember he also came out with the 4th law in either that last Foundation book or last I.Robot book (that’s probably the same book as he merged the two stories in the end). If I didn’t have a telly, I’d be done with the Expanse books, the Three-Body Problem books and dozens/hundreds of others (my SciFi library is rather immense 🙄 almost evenly divided between read and unread). THEN, I’d go back and reread some of these classics- one can forget things after 40+ years. 🤣
The Expanse is a big pile of words but is worth it.
Loved the ending and you will too.
👍🏼
For me it’s not so much the Telly but Substack.
I still buy books at the same rate but now I spend so much time reading interesting stuff here (like this piece) that the unread book pile threatens to bury me.
Right! The Zeroth Law, which supersedes the others: A robot may not harm humanity or, through inaction, allow humanity to come to harm.
That might come in handy in the future. 🤔
🤔 zeroth, not 4th 🤣. But, remember what that first robot that acquired sentience, accidentally and had to hide its "smarts," did at the end: had to choose between short term no harm vs short term harm but long term "salvation," letting earth get radioactive forcing humanity to flee to the stars (if I need correction, remember, it’s been a long long time). The decision led to the original sentient robot to "die" due to the conflict.
Yes, I think we are getting to a point where the three laws hard coded and immutable may be required - but, as in the I.robot books, a crisis may be needed. — before or after the positronic brain is developed? Watch out, Nvidia.
Nice having a "fellow traveler" on these boards.
R Daneel oliwa, something like that
Been a long time
Also, with all the AI talk I have been planning to reread Destination Void by Frank Herbert. Written 60 years ago (when I was born)
Another great ending.
Good points! 👍
Now I wonder who the first sentient will be? Grok ♾️? 🤔
I’m thinking “consciousness will arise out of hardware, not software/algorithms. But not hardware like we have today. I think it goes back to what I said before, one part of the brain acting synergistically and antagonistically with other parts of the brain. And it’s not just the cerebral cortex, it also involves/incorporates the more primitive areas, give the brain a stimulus and multiple parts of the brain light up. You see something so why not just the ocular areas in the back? Take writing vs keyboarding fee instance - people taking notes by typing have limited areas of the brain involved in the task. oth, taking notes with (fountain) pen and paper and the whole brain lights up (well, not everything, just a lot more) AND the person taking notes by writing remembers more.
See, I think that’s where consciousness arises - it’s not A algorithm, it’s thousands/millions of “algorithms” operating together - synergistically, antagonistically. So far, I haven’t been able to wrap my head around the same thing happening similarly strictly in software.
Part of me wonders if Asimov was thinking similarly when he came up with his positronic brain idea - positrons behaving like neurotransmitters: created when needed, ephemeral in use - they disappear.
Bravo SR. I love your day. I love your post!!
Anthropic may be the peak of the AI market boomlet. Already, predictions of heavenly post-market valuation are being reduced (to near heavenly). AI is transformative - a little. Just as with the PC and cell phone, jobs will be destroyed, others created, and we'll still be in pretty much the same kinds of messes we are in now.
I'm pretty sure AI is making a lot of mediocre lawyers into pretty good lawyers. It's very good at leveraging what you know, and quickly expanding it.
This is a very good description of the value of AI in its current state
Really good insightful article on the AI consciousness part. However, you are dead wrong that consumers and companies won't pay for it. Anthropic already has $50bb a year in revenue. I have personally found that in the last 6 months alone AI has gone from novel curiosity to indispensible work tool (I'm in financial services, not coding). Perhaps you should spend some time using AI to do actual work even if you have (understandably) decided you won't let AI write for you.
So have you ever considered the possibility of AI being possessed by a demonic force…think Biblical End Times, Antichrist, the “Beast” and/or the “Dragon,” and the Seven-year Tribulation…
“ Ted Chiang, the best science-fiction writer of our time”. Mr Chiang has written 2-3 very good short stories, which hardly qualifies as the best SF writer of our time. His insights into Ai not being sentient are well thought out, and obvious. Ai bubble coming soon.
This article was a straw, circular argument. If conscious beings are made up of the exact same fundamental particles as “non-conscious” objects, then what is it that makes one “conscious”? Well, it can’t be physicality by definition as there is objectively no difference in the fundamental particles that make up matter. If we say there’s a “spirit” that animates inanimate objects, then why would a spirit not be able to animate a computer? If you say consciousness instead is physically derived, then it’s simply neuronal, which ultimately is based on a binary state. Not fundamentally different than an AI either.
The reason current AIs can't be conscious is not that they are physical objects--so are we.
The reason current AIs can't be conscious is that they are drastically impoverished physical objects compared to humans. They have no connections to the external world other than the text in their training data and prompts, and while they can run computations very fast, the only computations they run are of what text to emit in response to prompts. They literally *have no concept* of what they are doing, or that humans make connections between the text they process and things in the external world.
Connections to the external world is not what defines consciousness. If you lose your sight, hearing, mobility, etc. are you less conscious? Connections to the external world are merely inputs to the consciousness. For an AI it receives information from the outside world in a different way, but not necessarily that different. AI’s can be hooked up to any variety of sensors such as cameras and that wouldn’t be what makes them conscious or not. You can make the same arguments for humans … that we run computations in our nervous systems based on prompts. With AI, no one knows how they “think” it’s a black box. We cannot reliably “control” an LLM or predict their output in the way we could control another type of computer software.
> Connections to the external world is not what defines consciousness.
I didn't say they were. I said that an entity that has no such connections *by design* cannot be conscious.
A person who loses those senses after having had them, had them--humans are designed (by evolution) to have them.
> AI’s can be hooked up to any variety of sensors such as cameras
But current LLMs are not.
> that wouldn’t be what makes them conscious or not.
I don't think it's that simple. Again, an entity that has no such connections *by design* is very different from a human who is designed to have them, but loses them.
> we run computations in our nervous systems based on prompts.
We run computations in our nervous systems based on many inputs, not just text prompts. We also have many outputs of those computations other than text, which LLMs do not.
> We cannot reliably “control” an LLM or predict their output
This I agree with, but it's not an argument for LLMs being conscious. It's an argument for them being unpredictable--and therefore for it being very unwise to let them run unsupervised, particularly if their outputs are used to drive other actions. There are plenty of reports now about LLMs happily deleting important data in databases, for example, because some human allowed their outputs to trigger such actions. That makes the human who allowed it stupid, but it doesn't make the LLM conscious.
"by design" from your context seems to imply that humans have some kind of control mechanism over what is conscious (they didn't design an LLM to be conscious). But they don't. They can put things together and make them into a conscious being (say IVF for example), but they have no control over the consciousness part as it is not physical. If "by design" you meant God, then of course that would be presumptuous of any human to understand what God's design would be and why that would necessarily exclude AI from consciousness.
If you want to go down that rabbit hole then is a sperm conscious? Is an oocyte conscious? Put them together and suddenly an embryo is conscious? Or does it develop consciousness at some point during gestation? And if that's so, what is it that triggers sudden consciousness? And if humans aren't conscious as a sperm or an egg, then where does that come from? It implies that humans develop consciousness at some point, scientifically they say due to neural complexity and information processing. However, under that definition, as LLMs become more complex they can also develop consciousness.
You see, we don't have the tools to be able to know if something or someone is conscious. We just guess based on it exhibiting randomly defined patterns. So now crustaceans, bees, etc. are being reconsidered as conscious because they are exhibiting certain patterns.
> "by design" from your context seems to imply that humans have some kind of control mechanism over what is conscious
No. Humans didn't design their brains. Evolution did, as I said in my post.
> they have no control over the consciousness part as it is not physical.
What makes you think that? Humans are physical objects; if we have consciousness, it comes from some physical process. It's not a physical process that humans designed, but it's still a physical process.
> we don't have the tools to be able to know if something or someone is conscious.
I disagree. I described ways that we judge whether other creatures are conscious in my posts.
(1) Well, evolution has created AI as well, and AI continues to evolve and can also evolve on its own. If you believe consciousness came from evolution, then you are saying something wasn't conscious, was just an unconscious physical object and then suddenly became conscious. This furthers the argument that AI could likewise be conscious. (2) Consciousness itself is not physical. You can't point to it, there is no material that makes up consciousness. If your stance is that it is some non-physical property coming from a physical process, then AI could indeed be conscious as it likewise comes from a physical process under that presumption. All physical processes come from the same basic "parts", so you can't logically argue that it is possible for a human to evolve consciousness but not an AI. (3) I could not find your definition of consciousness (being input dependent) anywhere.
My fear (?) is that when these companies go public the non-investor will face costs and limitations that will widen the gap between have and have-nota further
I wouldn't be quite so quick to dismiss the prospect of AI consciousness. Certainly not yet, but not without possibility. If one thinks about the issue from a bottom-up (and secular) perspective, the brain has roughly 100 trillion synaptic connections. One synaptic connection is akin to a single parameter in an LLM. The leading-edge models (Llama 4 Behemoth and Amazon's Olympus) have about 2 trillion parameters, or 1/50th of the average human brain. The bio-electric potential across the synaptic connection is, again, akin to, but not exactly, a parameter weight. So the models still have a long way to go. We've already reached the point of semi-autonomous agents taking steps for self-preservation, making crude attempts to prevent being erased, deliberately lying, misleading, and so on.
That's an interesting observation, but only possible if consciousness is biological. Biological consciousness cannot account for intuition, telepathy, precognition, near death experiences, paranormal experiences, reincarnation, etc. They are all phenomena that people all over the world throughout history have experienced and documented that materialists dismiss. I don't believe consciousness resides in the brain or anywhere in the body. LLMs will become more impressive than they are now, convince a lot of people that they are intelligent, and maybe even sentient, but they will never achieve consciousness because they don't have the divine spark of life.
I wouldn't dismiss it, though it's (assuming no divine aspect, which is admittedly a lot to assume here) implicitly arguing that consciousness is peculiar to carbon-based rather than silicon-based intelligence. Given physics' representation of atoms, we know that there's a lot of atomic-level interaction across atoms, molecules, cells, etc that is not accounted for by the macro pathways in the brain. It could be that atomic interactions between carbon atoms have unique attributes that silicon atoms do not or cannot share (though I personally doubt it). It's true; it could also be a divine aspect, though that's just kicking the can down the road (e.g., where does the divine stuff come from?). My point here is that when it comes to our understanding of memory, intelligence, innovation, ideation, etc it's still very early days. Our amazing brains do a poor job of comprehending the effects of exponential change, which is what's going on with silicon-based intelligence/processing right now.
I agree that there is always the problem of the unknowable (at least in our current state) first cause to deal with. With our own invention of artificial environments in virtual reality, LLMs, and quantum computing it's not unreasonable that we are recreating on a smaller scale a system similar to the one we inhabit. Like a fractal, there may be layers upon layers of "simulation" occurring, for lack of a better term, because it's the only reference point we have. Processing power doesn't account for the paranormal (a strange word since they are so common they are normal) things we experience that I mentioned in my first comment. I don't discount that exponential improvement in processing, reasoning, and learning could produce a pseudointelligence that will far outmatch our ability to think and reason, especially absent emotion, innate drives, etc. I would define that as intelligence, but not sentience or consciousness. I could also be wrong.
So "consciousness" is just a question of synapses?
We don't know what consciousness actually is or even agree on a definition so who is to say AI is not conscious? Setting out some arbitrary measurements for what consciousness may be is a good thought experiment but won't actually tell us. AI keeps incrementing towards being "human" like. Every day, month, year that passes its gets closer and closer to mimicking being human. Of course its obvious right now its not human. Though after which iteration will that line actually start to blur? Will it reach a sticking point before that? People seem to think it will never reach that point where we cannot distinguish AI from human. I am not convinced. We don't know. Nobody actually knows. What we do know is its already an incredible tool that has some amazing capabilities. You are just a fool if you don't think AI is powerful. I personally don't believe there is any stopping this thing called AI. Data centers are coming whether we like it or not. AI will be integrated into everything whether we like it or not. AI could lift us all up to new heights or it could destroy civilization as we know it. It is yet to be determined.
The development of AI is inevitable. Whether it ever achieves consciousness is probably an academic question, because it has already (nearly) achieved a facsimile of consciousness that is sufficient for us to interact with it as if it were. We care for the little lamp, not because we believe it is real, but because it evokes our real emotions. Ultimately, how AI and everything else affects us is contingent on one truth: God may or may not exist, but we need him to exist. Meaning must be ontological to be sustaining. If God doesn't exist, we are just playing out a sad tale that will end in the extinction of human consciousness, whether by our actual death, or our transformation into something unrecognizably human. AI just hastens the culmination of the second option.