So, the use of human brain cells is different territory. Bioethically speaking, it is illicit to press human body parts (which includes cells) into service for technological ends. They can only licitly enter into service of the integral human whole (such a life-saving transplants). Using them in technology also commoditizes the human body. (The reverse is not the case: it can be licit to use cybernetic devices to restore native function within a human body.)
The question comes up a lot here when there are posts about whether to consider LLMs conscious.
"We're going to condemn conscious entities to slavery"
vs
"They'll always just be machines and it's dehumanizing to consider them anything else"
Computers made of transistors were designed to function like computers made of vacuum tubes without negating the benefits of transistors.
I think the questions would be very much different. LLMs are not made of human brain cells.
I don't know if I buy that argument or even if it makes sense. But yeah this all smacks inextricably of soul talk.