>It will quack like self consciousness but is not actually self conscious.
But isn't this the point of the Turing test? Since we cannot determine otherwise, if something exhibits all of the outward observable behaviour of consciousness, we must conclude that it is conscious.
Otherwise we may be in the farcical position of needing to declare some humans not conscious under the same justification.
We know that the "consciousness" of the AI resides in a cluster that is "speaking" to thousands of clients at any given moment. So is there a single consciousness or multiple? It can pass any test and yet the consciousness question remains valid for this reason alone. It's truly alien to us and no one has good answers yet.
I expect we'll change our definition of "consciousness", as we did for the term "computer" after the 1940s. Up until then a computer was a person who did computations (with pencil and maybe an adding machine), so what humans could do limited what a computer could do.
But isn't this the point of the Turing test? Since we cannot determine otherwise, if something exhibits all of the outward observable behaviour of consciousness, we must conclude that it is conscious.
Otherwise we may be in the farcical position of needing to declare some humans not conscious under the same justification.