Re: chatgpt
It occurs to me that there have been not a small number of bits of literature and film devoted to essentially this story arc. They don't end well. Any of them.
What we tend to forget is that stories set in sci-fi futures (or costume-drama pasts) are never really about the alleged time period, they're about the present in which the authors live. The concerns, fears, problematic structures and relationships of their own times, which can be examined more minutely by putting them into an exotic setting.
What we haven't learned is that the capacity or drive to do something as an accomplishment doesn't mean that it ought to be done. Often, it ought not be. Such stories are re-tellings of how our species has a genius for ignoring our hunches and intuitions, when we think our immediate comfort or advantage could be helped.
AI is little different, except that if the materialist view of the conscious mind is true, we'll ultimately create one - at which point all bets are off. Self interest looks different to a bear, lion, or ungulate than it does to a human - why should we imagine an AI being would perceive its self interest in a way that fundamentally defaults to compassion for and deference to humans? Because we wrote it into the code? A code which the AI being itself could - and is designed to - rewrite?
If I use the word "God," I sure don't mean an old man in the sky who just loves the occasional goat sacrifice. - Anne Lamott