The issue isn’t whether we can imagine a smarter entity - obviously we can, as we do in sci-fi. But what we imagine are just results of human intelligence. They’re always bounded by our own cognitive limits. We picture a smarter person, not something categorically beyond us.
The real concept behind Artificial Superintelligence is that it wouldn’t just be smarter in the way Einstein was smarter than average - it would be to us what we are to ants. Or less generously, what we are to bacteria. We can observe bacteria under a microscope, study their behavior, even manipulate them - and they have no concept of what we are, or that we even exist. That’s the kind of intelligence gap we’re talking about.
Imagine trying to argue against a perfect proof. Take something as basic as 1 + 1 = 2. Now imagine an argument for something much more complex - like a definitive answer to climate change, or consciousness, or free will - delivered with the same kind of clarity and irrefutability. That’s the kind of persuasive power we’re dealing with. Not charisma. Not rhetoric. Not “debating skills.” But precision of thought orders of magnitude beyond our own.
The fact that we think we can comprehend what this would be like is part of the limitation. Just like a five-year-old thinks they understand what it means to be an adult - until they grow up and realize they had no idea.
Logic is logic. There is no “advanced” logic that somehow allows you to decipher aspects of reality you otherwise could not. Humanity has yet to encounter anything that cannot be consistently explained in more and more detail, as we investigate it further.
We can and do answer complex questions. That human society is too disorganized to disseminate the answers we do have, and act on them at scale, isn’t going to be changed by explaining the same thing slightly better.
Imagine trying to argue against a perfect proof. Take something as basic as 1 + 1 = 2. Now imagine an argument for something much more complex - like a definitive answer to climate change, or consciousness, or free will - delivered with the same kind of clarity and irrefutability.
Absolutely nothing about humans makes me think we are incapable of finding such answers on our own. And if we are genuinely incapable of developing a definitive answer on something, I’m more inclined to believe there isn’t one, than assume that we are simply too “small-minded” to find an answer that is obvious to the hypothetical superintelligence.
But precision of thought orders of magnitude beyond our own.
This is just the “god doesn’t need to make sense to us, his thoughts are beyond our comprehension” -argument, again.
Just like a five-year-old thinks they understand what it means to be an adult - until they grow up and realize they had no idea.
They don’t know, because we don’t tell them. Children in adverse conditions are perfectly capable of understanding the realities of survival.
You are using the fact that there are things we don’t understand, yet, as if it were proof that there are things we can’t understand, ever. Or eventually figure out on our own.
That non-sentients cannot comprehend sentience (ants and humans) has absolutely no relevance on whether sentients are able to comprehend other sentients (humans and machine intelligences).
I think machine thinking, in contrast to the human mind, will just be a faster processor of logic.
There is absolutely nothing stopping the weakest modern CPU from running the exact same code as the fastest modern CPU. The only difference will be the rate at which the work is completed.
I’m not saying ASI would think in some magical new way. I’m saying it could process so much more data with such precision that it would detect patterns or connections we physically can’t. Like how an AI can tell biological sex from a retina scan, but no human doctor can do even knowing it’s possible. That’s not just “faster logic.” It’s a cognitive scale we simply don’t have. I see no reason to assume that we’re anywhere near the far end of the intelligence spectrum.
My comment about it’s potenttial persuation capabilities was more of the dangers of such system. That an ASI might be so good at persuasion, threat construction, and lying that it could influence us in ways we don’t even fully realize. Not because it’s “divine” - but because it’s just far more competent at manipulating human behavior than any human is.
Superpowered lying is already a thing, and all we needed was demographic data and context control.
Today, it is possible to get a population to believe almost anything. Show them the right argument, at the right time, in the right context, and they believe it. Facebook and google have scaled up exactly that into their main sources of revenue.
Same goes for attention hacking. AI generated content designed to hook viewers functions in entirely predictable, and fairly well understood ways. And the same goes for the algorithms which “recommend” additional content based on what someone is watching.
As for why doctors can’t do things AIs are pulling off, I’d suggest that’s because current systems are using indicators we don’t know about, which they aren’t sentient enough to explain. If they could, I have no doubt a human doctor, given enough time, could learn about, and detect, such indicators.
There is no evidence that what these models are doing, is “beyond our scale of thinking”.
But again, I do think the machine will be faster.
Current models display “emergent capabilities”, as in abilities we don’t know about before the model is created and tested. But once it is created, we can and have figured out what it is doing and how.
The issue isn’t whether we can imagine a smarter entity - obviously we can, as we do in sci-fi. But what we imagine are just results of human intelligence. They’re always bounded by our own cognitive limits. We picture a smarter person, not something categorically beyond us.
The real concept behind Artificial Superintelligence is that it wouldn’t just be smarter in the way Einstein was smarter than average - it would be to us what we are to ants. Or less generously, what we are to bacteria. We can observe bacteria under a microscope, study their behavior, even manipulate them - and they have no concept of what we are, or that we even exist. That’s the kind of intelligence gap we’re talking about.
Imagine trying to argue against a perfect proof. Take something as basic as 1 + 1 = 2. Now imagine an argument for something much more complex - like a definitive answer to climate change, or consciousness, or free will - delivered with the same kind of clarity and irrefutability. That’s the kind of persuasive power we’re dealing with. Not charisma. Not rhetoric. Not “debating skills.” But precision of thought orders of magnitude beyond our own.
The fact that we think we can comprehend what this would be like is part of the limitation. Just like a five-year-old thinks they understand what it means to be an adult - until they grow up and realize they had no idea.
Logic is logic. There is no “advanced” logic that somehow allows you to decipher aspects of reality you otherwise could not. Humanity has yet to encounter anything that cannot be consistently explained in more and more detail, as we investigate it further.
We can and do answer complex questions. That human society is too disorganized to disseminate the answers we do have, and act on them at scale, isn’t going to be changed by explaining the same thing slightly better.
Absolutely nothing about humans makes me think we are incapable of finding such answers on our own. And if we are genuinely incapable of developing a definitive answer on something, I’m more inclined to believe there isn’t one, than assume that we are simply too “small-minded” to find an answer that is obvious to the hypothetical superintelligence.
This is just the “god doesn’t need to make sense to us, his thoughts are beyond our comprehension” -argument, again.
They don’t know, because we don’t tell them. Children in adverse conditions are perfectly capable of understanding the realities of survival.
You are using the fact that there are things we don’t understand, yet, as if it were proof that there are things we can’t understand, ever. Or eventually figure out on our own.
That non-sentients cannot comprehend sentience (ants and humans) has absolutely no relevance on whether sentients are able to comprehend other sentients (humans and machine intelligences).
I think machine thinking, in contrast to the human mind, will just be a faster processor of logic.
There is absolutely nothing stopping the weakest modern CPU from running the exact same code as the fastest modern CPU. The only difference will be the rate at which the work is completed.
Beginning by insulting your opponent isn’t exactly the best way to ensure they’ll finish reading your message.
You have a great day.
Fair.
I’ve removed it, and I’m sorry.
I’m not saying ASI would think in some magical new way. I’m saying it could process so much more data with such precision that it would detect patterns or connections we physically can’t. Like how an AI can tell biological sex from a retina scan, but no human doctor can do even knowing it’s possible. That’s not just “faster logic.” It’s a cognitive scale we simply don’t have. I see no reason to assume that we’re anywhere near the far end of the intelligence spectrum.
My comment about it’s potenttial persuation capabilities was more of the dangers of such system. That an ASI might be so good at persuasion, threat construction, and lying that it could influence us in ways we don’t even fully realize. Not because it’s “divine” - but because it’s just far more competent at manipulating human behavior than any human is.
Superpowered lying is already a thing, and all we needed was demographic data and context control.
Today, it is possible to get a population to believe almost anything. Show them the right argument, at the right time, in the right context, and they believe it. Facebook and google have scaled up exactly that into their main sources of revenue.
Same goes for attention hacking. AI generated content designed to hook viewers functions in entirely predictable, and fairly well understood ways. And the same goes for the algorithms which “recommend” additional content based on what someone is watching.
As for why doctors can’t do things AIs are pulling off, I’d suggest that’s because current systems are using indicators we don’t know about, which they aren’t sentient enough to explain. If they could, I have no doubt a human doctor, given enough time, could learn about, and detect, such indicators.
There is no evidence that what these models are doing, is “beyond our scale of thinking”.
But again, I do think the machine will be faster.
Current models display “emergent capabilities”, as in abilities we don’t know about before the model is created and tested. But once it is created, we can and have figured out what it is doing and how.