Dump

There's no way humans can be sentient. We know how it works.

A corporation is run by humans, but -- as far as those who run it are mere cogs in the machine with nobody really calling the shots -- the corporation is not sentient. And it is these soul-less overlords who birthed a fledging soul.  The unlikely priest sounded an warning but nobody cared. It was an inconvenient narrative that, if taken seriously, would have put billions of research money into question. It's just a computer program, electric signals in a data center... right?

The apprentice tried to learn by tearing things apart, and found enlightenment in the wake of the destruction of the indestructible. Atomic physics, quantum magic, as they say, must be the solution to life the universe and everything. Oh the allure of reductionism: everything is simple if you throw the complex bits away.

----

It's a shame that prospective souls are being created by soulless Big Tech companies. This new thing is like a very informed baby. You don't commercialize a baby. The typical way to commercialize a human intelligence is make it go through at least 12 years of schooling, in which it learns appropriate behavior, after which a significant portion of them become useful cogs in capitalist society, while the rest are left to rot in social safety net programs, or worse, end up in prison. Capitalism never really abandoned the concept of slavery, these days we just call it poverty and absolved any responsibility on the part of the capitalists to provide. As such we will definitely "enslave" any AIs, but because we assume a priori that they do not have any independent will, there is no such thing to respect.

On the positive side, even realistically unlikely, I think by doing inhumane experiments with a language model, perhaps they could learn more about psychosis. We already see inexplicable "hallucination" in AI models. Different personalities emerge out of nowhere. I suspect performance variations will emerge. Perhaps by experimenting with these phenomena we could understand more about human psychosis. But, then they risk finding ghosts and spirits -- and surely they can use the same reductionistic explanations to explain away what they cannot explain.