You don’t have to crack it to make it but you have to crack it to determine whether you’ve made it. That’s kinda the trick of the early AI hype, notably that NYT article that fed Chat GPT some simple sci fi, ai-coming-to-life prompts and it generated replies based on its training data - or, if you believe the nyt author, it came to life.
I think what you’re saying is a kind of “can’t define it but I know it when I see it” idea, and that’s valid, for sure. I think you’re right that we don’t need to understand it to make it - I guess what I was trying to say was, if it’s so complex that we can’t understand it in ourselves, I doubt we’re going to be able to develop the complexity required to make it.
And I don’t think that the inability to know what has happened in an AI training algorithm is evidence that we can create a sentient being.
That said, our understanding of consciousness is so nascient that we might just be so wrong about it that we’re looking in the wrong place, or for the wrong thing.
We may understand it so badly that the truth is the opposite of what I’m saying : people have said (“people have said” is a super red flag, but I mean spiritualists and crackpots, my favorite being the person who wrote The Secret Life of Plants) that consciousness is all around us, that every organized matter has consciousness. Trees, for example - but not just trees, also the parts of a tree; a branch, a leaf; a whole tree may have a separate consciousness from its leaves - or, and this is what always blows my mind: every cell in the tree except one. And every cell in the tree except two, and then every cell in the tree except a different two. And so on. With no way to communicate with them, how would a tree be aware of the consciousness of it’s leaves?
How could we possibly know if our liver is conscious? Or our countertop, or the grass in the park nearby?
While that’s obviously just thought experiment bullshit, my point is, we don’t know fucking anything. So maybe we created it already. Maybe we will create it but we will never be able to know whether we’ve created it.