I am honestly amazed by how people are not interested in the obvious counterfactuals. Are we seriously that disinterested in how many people were helped by AI, that we don't even feel the need to ask that question when looking at potential harm?
Concerned that AI companies, like social media companies, exist in an unregulated state that is likely to be detrimental to most minors and many adults? Absolutely.
We simply don’t know the scale of either side of the equation at this point.
Like all tools we regulate heroin and we should regulate AI in a way they attempts to maximize the utility that it provides to society.
Additionally with things like state lottery systems we decide that we should regulate certain things in such a way that the profits are distributed to society, rather then letting a few rent seekers take advantage of intrinsic addictive nature of the human mind to the detriment of all of society.
We should consider these things when developing regulations around technology like AI.
From what I can gather, a lot of ML people are utilitarians, for better or worse. Which is why you're seeing this discussed at all, if we all agreed on the ethics it would have been a no-brainer.
And I don't think any utilitarian would be against "something with some harm but mostly good can be made to do even less harm".
But the person I initially replied to, obviously doesn't see it as "some harm VS much good" (which we could argue if it's true or not), and would say that any harm + any good is still worth considering if the harm is worth it, besides the good it could do.
That's certainly the impression you gave with your response. You didn't engage with the clear example of harmful behavior or debate what various numbers on either side of the equation would mean. Your approach was to circumvent OP's direct question to talk philosophy. That suggests you think there are some philosophical outlooks that could similarly sidestep the question of a therapist pushing people to kill themselves, which is a rather simple and unambiguous example of harm that could be reduced.
Sounds more like the plot line of a 60s sci-fi novel or old Star Trek episode. What would the prime directive say? Haha
I'm not even a utilitarian, but if there are many many people with stories like her, at some point you have to consider it. Some people use cars to kill themselves, but cars help and enrich the lives of 99.99% of people who use them.
They are mostly useful, but occasionally can kill someone who indulges in them too much.
Look at how much is being invested in this garbage then look at the excuses when they tell us we can't have universal medicare, free school lunches, or universal childcare.
Then the question becomes more if we're fine with some people dying because some other people aren't.
But AFAIK, we don't know (and probably can never know) the exact ratio of people AI has helped still be alive today VS helped contribute to that these people aren't alive today, which makes the whole thing a bit moot.
In fact, targeted research with this data could help do more research on how to convince more people to stay alive, right?