TikTok’s algorithm keeps pushing suicide to vulnerable kids
bloomberg.com
bloomberg.com
Let's suppose there is no law on the books that explicitly fleshes out liability if you train a model and it does not do as you expect, we'll need to figure out some nearest neighbor matching.
So you will have to shoehorn this novel issue into some preexisting framework of precedent. Maybe something about negligence liability and a common public use item, like a bridge or something. But a bridge isn't as complicated as an ML model and a bridge failure is way more auditable. Bridges don't hallucinate. Also there are unintended consequences to doing this sort of legal massaging, before you know it, campaign finance = speech.
"Hold them accountable" is so broad that verges on meaninglessness. Part of the whole living in a rapidly evolving tech landscape is living with laws that are always a few steps behind the times. Which bleeping sucks, but until we press for a better large scale solution, it is going to keep sucking.
In case of alcohol, sure I can see it but that is not exactly comparable. Enough alcohol will kill you 100% of the time, this aint even close.
that being said some content warning or a bit of push to seek help wouldn't be a bad idea when someone gets stuck in some topic like that
We can deflect saying "Anyone who knows a bit about recommendation systems", blame my use of the platform or play these off as a technical problem but imo it seems like Google and the other tech companies know exactly what they are doing in these use cases and are just cynically deploying this content with the culture war filter on because they know it makes the most money. I think when people ascribe political leanings to tech companies that maybe the mean belief of their employee's but as an org it is not the standard opined ultra left havens.
Negligently letting it be pushed onto people is much worse.
Imagine if facebook were doing the same thing now, there would me massive uproar.
by all means put a content warning or maybe a suicide prevention hotline or other help but I'm tired of clowns trying to solve any problem with "bad content" with censorship. Like that didn't misfire enough times in history...
If you have a problem with it, you should instead be demanding an open index of all content on these platforms so that you can be assured that all information submitted is available and free from real censorship.
No not "random" but systematic, targeted and effective "censorship"
When a news corp chooses to not run a news story for the third day in a row, is that censorship?
When I am pissed off at a customer services agent, and instead of shouting "GET FUCKED YOU USELESS SHIT" is that censorship?
When I am walking along the road and see a very sexy person, and I _don't_ shout out "HEY THERE SEXY, I'D LIKE TO FUCK/GET FUCKED BY YOU!" is that censorship?
A well ordered society requires rules. Those rules need to be clear, and generated by consensus.
When you negligently and repeatedly show footage to a vulnerable person designed to deliberately illicit harm, that is not what a well ordered and productive society allows.
But then the US is not a well ordered society, so do what you want kid.
It can be. If it were a news story that made a company or a government look bad and the news corp was bribed or pressured to kill the story and stop talking about it, that's absolutely censorship.
> When I am pissed off at a customer services agent, and instead of shouting "GET FUCKED YOU USELESS SHIT" is that censorship? When I am walking along the road and see a very sexy person, and I _don't_ shout out "HEY THERE SEXY, I'D LIKE TO FUCK/GET FUCKED BY YOU!" is that censorship?
That first one's not a full sentence, but if both cases are about self-censorship then you should know that's a very real thing and often it's a good thing. Especially when the decision on what gets censored is based on your own thoughts and feelings more than external factors. Being able to choose for yourself what to say or not say is a freedom, not a problem. Self-censorship isn't always ideal though (Self-censorship as a result of chilling effects for example).
> When you negligently and repeatedly show footage to a vulnerable person designed to deliberately illicit harm, that is not what a well ordered and productive society allows.
When something has already been judged as negligent it's automatically something a well ordered and productive society wouldn't allow. I'm not sure that's the case here though. If they just didn't care enough to make sure their algorithm wasn't doing more harm than good then maybe it would be negligence. If they are intentionally trying to harm people that would be something else.
However the content is being used/misused I'm not sure that I agree that the content was designed to deliberately illicit harm. I'd bet that most of these types of videos are created by suicidal people expressing their own feelings and looking to share them with others who are going through similar trials. I think there should be freedom for people who are struggling with mental health to participate in a community of others and to express themselves in memes and tiny videos.
I also think people should have the freedom to end their own lives if they choose to. Having places online where people can talk and share information about that subject is perfectly reasonable. Although I think there are children for whom suicide might be a legitimate option, even if most others would disagree not everything online needs to be appropriate for children, let alone suicidal children.
The problem with tiktoc isn't that the content exists, it's that it's being heavily promoted to children. This is the same problem with youtube. The algorithms push more narrow and extreme views without considering anything except their profits (unless you're one of the people who think tiktoc is intentionally trying to kill off Americans one suicidal teenager at a time in which case it'd not just be monetary considerations driving their choices). It would be so much better if algorithms were designed to present a more balanced view of things so that even if someone were thinking about suicide and interested in hearing perspectives on the subject they didn't only get the most extreme videos saying it's a great solution.
So why even bring it up? It's completely disingenuous imho.
A bit like the production of fentanyl precursor chemicals and their export to Mexico...
And Chinese filters can't be applied to the world wide version because they won't be able to compete with other addictive social media.
They have an algorithm that chooses what videos to show you based on types of videos you view. I'm not sure how that doesn't qualify as intentional.
I forget the details of it, but the point was that this could be extended to other behavior as well, possibly all behavior. The underlying connotation is that humans take on pack roles as pack animals, and are rewarded by connective activities, even suicide, as crazy as it sounds.
Social media is a conduit of behavior modeling. It is best for everyone to realize this.
I recall a story someone told (can't remember if it was a co-worker or a comment on this site) about someone in an office who stood up from their desk at 4:00 on a Friday, said, "I guess I'll take one for the team," and walked out the door. Sounds a bit like that.
Horrific.
Almost nothing I search for is on my FYP. My FYP is almost all pranks, basketball, dancing in public, and Harry Mack.
I have noticed that if I watch a video all the way through I will get more of those videos pretty quickly. If this kid watched a depression video all the way through it will throw another at him shortly. If he watches that one then it can quickly change the algorithm to begin to favor those. The algorithm has a short memory.
(didn't downvote you though, I understand your point)
The key is education, you have to explain to your kids how these services works, how they are optimized for content engagement to show ads, how they mine your data, and what kinds of things they should look out for and avoid (like misinformation, catfishing, grooming, viral stupidity etc)
We also hitch-hiked to save bus money parents gave us to get some ice cream
I have an ongoing conversation with mine about the things he finds online. We talk about how people try to benefit from appearing "rich" online. We talk about how people have problems in their lives and find things to blame and rant about online. We talk about how people you don't know can try to get things from you or make you think a certain way.
He even started using the term "clickbait" before I even taught it to him. These kids are savy. I think the biggest thing we can do for them is to help them understand how to contextualize what they're engaging with because they don't get to grow up in the "before" and "after" worlds that elder millennials did.
And frankly, I see that my kid is just as saturated in propaganda from his public school as he is from online media. So far, in parallel to his education, school seems to be running one giant ad for the US armed forces.
Sit down with them and go through tiktok or insta. Point out dark patterns, bad content.
Answer their questions truthfully. Sometimes the truthful answer is "I don't know".
The field you want is "family dynamics". If things are already off the rails, find a mediator (doesn't have to be an actual legal mediator).
1. Understand their viewpoint and give them the tools to understand yours
2. Find common ground and define the areas of conflict
3. Reassure and reaffirm relationship
4. Repeat
I had the internet for a while when I turned 11.
My parents were pretty clear about what was good/bad. "you can see all sorts of stuff on the internet. But first things first, never use your real name or address. Second, we can totally see what youre looking at."
For me that worked.
For my older sister, she still tried to run off with a bloke she met off a chat room.
so you cant win them all.
While you cannot keep kids away from it forever, it makes a lot of difference when they are exposed to it and how well-equipped they are to deal with it.
This is also a true statement, but happens far less than it does with social media (or other violent or otherwise undesired media).
That's not currently what people want, but I think it could change it the future. But at the same time, there is a particular American quality of stubborness/pride that will probably push back "we have the right to keep hurting ourselves even though we don't really want to, but are being manipulated by algorithmic attention-fixation tricks!"
>This is also a true statement, but happens far less than it does with social media (or other violent or otherwise undesired media).
There is a difference between parent giving kid a lesson about alcohol (which the poster above is proposing), and just starting to drink with your underage kid (which you are comparing to)
It still happens, but it's a lot less socially acceptable, and it happens a lot less.
The real national security issue is if a major power can weaken a rival power to a significant degree without even raising a weapon. That indicates an enormous lack of resilience for a culture that prizes free expression, warts and all.
While I have concerns about the above, I am MUCH more concerned that through this and Twitter's recent changes, we're slowly losing extremely valuable "alternative news sources." and that these are mostly a result of quiet socio-politcal interests, and not for many (or any) genuine concerns for safety (i.e. Facebook and Instagram will happily continue to push the exact same stuff if Tiktok and Twitter go down)
Good stuff.
[thisisfine.jpg]
This is not a novel insight. It's why advertising works. AI-based Skinner-box algorithms are like traditional advertising techniques on steroids. They inevitably profilerate the kind of content that optimizes for the most engagement, and due to the way our lizard brains work, this happens to be content that generates emotional manipulation and outrage. The consequence? A population-wide mental health catastrophe.
what do people expect? if it learns what makes you stay on the platform it gives you more of the stuff that makes you stay
you are what you eat.
I don’t have kids, but surely you see the design flaw at play here:
I’d want to give my future kid a phone so I may connect with them at will, but at the same time a phone allows others to connect with them (giving them addictive content to serve ads for example)
An iPhone has pretty good parental controls, but when it’s time for the child to connect with their peers, the same place they connect with peers is the same place designed to exploit them, with little to no alternative.
It’s like we’ve transported our 11 year olds into digital pool halls where parent supervision is banned and the pool hall owner keeps serving them drinks
I won’t be giving my future child a phone before 13 but I have no illusions that they’ll be normal peer after not having the same access to such things.
It’s like the McDonald’s play pens all over again, but worse!
try to use it and you will see how bad it actually is, reinstalling an app removes its timeout, any kid almost trivially goes around it, you have to not forget to block the web versions of the apps, there is +1 minute, there is issue with whatsapp links etc
my daughter(12) also just asks for 15 mins extra youtube, then she just looks at my fingerprints on the phone to guess the screentime code, so i have to wipe my fingerprints like i am in a james bond movie (changed the screentime code like 3 times until i figured out what is happening)
i am trying to teach her self control and also to monitor her feed like a hawk and be super critical of what she watches
also once a week we go over all the feeds and try to teach the algorithms to show better things (like/follow some science/code youtubers, block toxic ones, watch few videos etc)
i also pay for everything i can so she doesnt see ads (including having pi-hole and adblockers)
For example, someone posted about a group of girls in Japan jumping to their deaths. What makes something like that possible? How does it happen?
Please do not add comments if you don't have the requisite training or experience. Questions would be OK.
This sub-thread could turn into a conversation between people with relevant credentials answering questions. We could all learn something from reading through such a thread.
Please state your credentials if you comment or answer questions.
No need for credentials if you post a question.
Perhaps they should be fined in the millions like the rest of the other social network companies like Facebook, Instagram, etc. Repeat offences should be increased into the billions if possible.
TikTok does actively take down content that violates their guidelines. Children are much more likely to come across nudity or porn on Twitter/Instagram.
The point was that they're not doing anything that the other platforms aren't doing.
They seem to do a fair bit to identify content that breaks their guidelines. To the point that users often make up alternative terms for sensitive topics e.g. Nazis become Yahtzees/N@zi and suicide becomes unalive. The belief is that TikTok flags videos with these terms and may remove them outright or downrank them in the recommendation algorithm.
There's a lot more outright pornography on Twitter and Instagram than TikTok.
all it does is offer you choices of things that people generally react to. Its a window to your desires/hates.
Now, if you are taught that you are allowed to "long press/right click" and say "I don't like this" then your life gets much better.
Does that mean that they shouldn't have a duty of care to their customers? no.
Does that mean that they don't have a responsibility to keep a high standard of safe/good/clean content on their platform? no.
Doesn't take a huge leap to think it could be intentional..
[1] https://www.opindia.com/2022/07/tiktok-china-engineering-oth...
On TikTok you largely just get more of what you've already engaged with. If you're using the platform for porn you're going to get a lot of it. If you like and engage with scientific/political content you'll rarely, if ever, see a video of someone dancing.
edit:
It's anecdotal but having taken a look at TikTok's web feed: https://www.tiktok.com/foryou. The content there was less sexual than the purportedly sanitised Douyin.
For what it's worth. Four out of the first five videos I saw on Douyin were sexualised in some way, the first was of a train driver. I scrolled through 20 or 30 of the videos on TikTok's website and didn't find anything close.
It's not proof of what is being claimed, but they are variables that would need to be controlled before being confident that Douyin is or is not adjusting content to foreign or domestic users.
The original claim is that Douyin promotes "wholesome" educational content, whereas TikTok is manipulating the west with pornography and other depravities.
Anecdotal observations gathered through the web UI of each app, as an anonymous user, were that Douyin presented "pornographic" content, whereas TikTok did not.
The original claim only stands if we accept that ByteDance is:
* Allowing a vast quantity of sexually explicit content to be posted to Douyin.
* Not showing that content to their users in China but only to western users of Douyin, despite being presented entirely in Chinese.
While also:
* Targeting western users of TikTok with pornography and explicit videos.
* Not showing any of that content in their web UI, despite it quickly becoming the entirety of the Douyin one.
I find it hard to believe that this would be the case. A sanitised Douyin and depraved TikTok, or both being depraved, would have supported the idea. The behaviour that was observed runs entirely counter to the claim.
Frankly, the opinion does not appear to be grounded in reality but comes instead from a place of insecurity regarding our social values. Consider whether any western platform promotes science and engineering over pornography, Twitter and Instagram allow much more explicit content than TikTok, and then ask why TikTok seems to have been singled out.
By checking douyin.com from my device, my ip and my advertising fingerprint, I cannot have any clue how douyin behaves if I download it from a chinese phone, in china.
The base claim was that ByteDance was pushing sexualised content on TikTok while policing it on Douyin. This makes zero sense given that TikTok's web content was sanitised. Douyin being awash with sexualised content implies an audience for that content in China, further weakening the argument that this is some sort of targeted attack.
The alternative is that Douyin thirst traps are some sort of co-ordinated attack on the west. Despite not being available in the western app or translated in any way.
Not that OP's claims are proof - but your anecdata does not disprove it.
Is it inequality and injustice. where pointing it out. would actually threaten the power of the CCP?
When I started reading Chinese language media I was surprised how much apparent criticism there is -- but if you read carefully, anything approved in the PRC is always criticism of individuals, or maybe corporations, more rarely a local government, doing something that's already supposed to be officially prohibited, or which is not too sensitive, and never criticizes core official policy or high-ranking CCP members.
Which is how you can have anti-corruption, anti-sex scandal, anti-pollution news commentary, or even protests and marches, in China. But should official permission to disagree be withdrawn, the topic can very quickly become taboo. A couple years ago there was ongoing discussion in Chinese media about same-sex marriage and a member of the Yuan even introduced a bill to make same-sex marriage legal in China. Voted down. But an acceptable topic to discuss, not officially taboo. I suspect that has changed recently, with the campaign against the apparent feminization of Chinese men and now explicitly linking LGBT rights with the Western agenda. That shift, as I understand it, comes directly from Xi.
Isn't the narrative "that's what the western world consumers more of/wants" versus "China values achievements and education more than debauchery and stupidity"?
With parents working insane hours in the US and outsourcing the socialisation and education of kids to electronics and corporations with perverse incentives, with local communities practically gone in an urban lifestyle, we are raising generations of porn addicted dopamine junkies. The west is fucked.
It's the parents responsibility to curate the world the their kids are exposed to and build up their defences for all the crap that they're going to finally encounter.
Agreed on the first point, disagree on the second. I think it’s a fixable problem. Addicts can recover and further damage can be avoided.
I mean, I can speak from my own personal use case (I think I average 30-60 minutes of TikTok screen time a day per week... yeah... I know, not great). I would agree with you. The only reason somebody's feed would show "low grade" content is if they curated it to be that way for themselves.
Now, is it easier to curate a feed of garbage West vs East? Not sure...
The possibility of China abusing content moderation might be enough to throw out the app altogether. But the only "evidence" we currently have is guessing at what China might do and people in media amplifying that discomfort.
Compared to other social media it doesn't seem that "debauchery and stupidity" is more prominent there.
I'm leftist liberal and I have no particular paranoia about China, but I'm not going to pretend that we are besties
Edit: I won't remove the classification of myself as "leftist liberal" because it would make some of the replies to my comment confusing. I will apologize for it because it has derailed the discussion of China and TikTok. In the future I'll be more specific and avoid labels that no two people can agree on.
Suffice to say that I don't believe China deliberately created and released Covid.
I'm confused. Do you think the CPP is liberal? It's very authoritarian.
Liberal != Leftist
And neither a leftist nor a liberal has to answer for Communist China. Nor do you gain any special "objectivity" about China by being either one.
There is an interesting case made[1] that while this used to be true, it is actually not the case any more and that things are starting to become rather anti-liberal on both sides of the political spectrum.
[1] https://www.goodreads.com/book/show/102146148-what-s-our-pro...
This narrative dominated TV, print and to a lesser extent radio. It really created the notion in people's heads that this narrow band was really what people thought about. What people are reacting to in large part is the newly visible presence of alternative perspectives. They were always there. They were just strategically locked out by people who decided what was "fringe". Basically anything that didn't support institutions was ignored or disparaged.
Waco and Cuba are two prime examples. Waco was written off in the national media because the people in the compound were so "weird". And for years the national media has underwritten the American foreign policy blob's almost neurotic fixation with Cuba.
A quote from the page I was just reading:
> “When Speech First brought a lawsuit against Michigan, alleging that their bias response teams were unconstitutional and “flagrantly violate the First Amendment,” the court agreed, ruling against the school. As of 2022, 456 U.S. colleges⬥ have bias response teams (almost double the 2017 total of 232),49 and very few have been subject to investigations like the one at Michigan.”
Excerpt From What's Our Problem? Tim Urban https://books.apple.com/se/book/whats-our-problem/id64456422... This material may be protected by copyright.
Wrong. They are neoliberal, not classically liberal.
Classical liberalism (belief in naturalistic free market economics) failed in the 20th century and neoliberalism took its place, characterized by top-down micromanagement of economic ecosystem.
In other words, classical liberalism insists markets will thrive only when left to their own devices. Neoliberalism insists markets will thrive only when managed by competent administrators (see Federal Reserve).
The US is plenty authoritarian and China becoming a democracy wouldn't actually change any of the things that people are paranoid about. The state mucking around in private industry to achieve political and national security goals and wage information warefare is as American as hamburgers.
Taken as a single political entity on the national stage China is just USA but in italics. It makes total sense that we would be uneasy in the US but that unease is how everyone not in the US feels about us. The unwavering belief that we're the good guys leads us to project our military might much more freely which doesn't exactly make us popular.
It is crazy that TikTok being a Chinese based company is obviously some ploy to undermine the US while in reality the CCP just forced TikTok to change their algorithm to be educational in China where such a thing wouldn't fly in the US because of the first amendment. China's video game ban for under-18s operates on the same principal.
Ludicrous false equivalency.
The USA doesn’t have laws preventing its citizens from criticizing the government, nor are its corporations required by law to participate in policing speech.
This is the reason why criticism of China is so muted.
The two-party system makes for all sorts of weird alliances of distinct groups getting lumped together- there's really no reason that one's opinions on gun rights should be related to one's opinions on energy policy, or why one's opinion on abortion should be related to one's opinion on foreign affairs policy, but with two parties all these separate issues get grouped into two camps associated with either "Republican" or "Democrat" and then in turn everything "Republican" can also be called "Conservative" or "Right-Wing" and everything "Democrat" can also be called "Liberal" or "Left-Wing".
Using some academic definition of what "liberal" or "right-wing" or whatever meant in Europe in the 1890s or 1930s doesn't really work in popular English-language media today.
Winner-takes-all popular elections have proven to be naive, but I wouldn't say it's a quirk. It's probably the first election format you'd try.
My point wasn't that you need a coalition as much as that people will change their views to align with their coalition.
That's an oxymoron.
I don't really care what you mean by leftist, but you definitely cannot claim to be a liberal and have no "paranoia" about China.
I do think that through malice or neglegence, that tiktok's algorithm is harmful to children.
It may be true on the margin, but Douyin is full of 'Key Opinion Leaders' who at best pay lip service to education and attainment. It's even worse on Kuaishou (Douyin for more rural areas). [0]
I always kind of bristle at the argument that this is somehow impacting what kids want too - PBS exists and spams their content as far and wide as possible, but US kids still end up preferring Cocomelon, Gabby's Doll House, etc.
There's no reason to believe that Chinese youth just take what's given to them when there are literally millions of random livestreams (hard to regulate livestreamed content at that level) going on at any given time.
As usual with China, what's likely going on is that someone is the CCP wants it to be true, so they 'make it true' and US media without local context takes it at face value.
[0]https://jingdaily.com/2022-top-six-chinese-kols-you-need-to-...
To me, it seems much more likely that China is just censoring TikTok like they censor every other platform, while the rest of the world sees a standard recommendation algorithm. "Debauchery and stupidity" is a subjective, sanctimonious phrase that doesn't really mean anything.
As such, apps like TikTok are just following the target population’s specs.
Ironic hotbed of so much technological innovation (but then $$$ is a strong motivator).
This conspiracy theory always gets a good chuckle out of me and my Chinese friends whenever it comes up. I have a good chunk of my family living in China and I can assure you, Chinese kids watch just as much if not more junk over there than they do in the US.
This is similar to trying to argue that America kids only play wholesome games on the Xbox or Youtube because they have parent controls.
But to my first point: surely working in tech right now you have a few Chinese coworkers. I recommend you grab lunch with them sometime (and let them choose the place to eat!)
Maybe Tiktok is just trying to give the market what it wants? Idiotic stunts were going viral years before TikTok existed.
The opIndia article says "The flood of such content on TikTok, an immensely popular social media platform among the youth, is a serious problem that needs to be addressed at the earliest."
There's a lot of dumb stuff on TikTok, but trans people not being suppressed isn't really one of them. And there might be _toxic_ stuff on TikTok, but in the context of an article where youth are pushed to material about suicide, linking to an article calling for the suppression of trans perspectives as if that was part of the same problem is super shady.
As for Douyin having a different algorithm, I'd point out that within China it was still viewed as being harmful enough to kids that they put in a 40m daily time limit, and restricted it to specific hours. I'm guessing it's also not great for kids there. https://www.bbc.com/news/technology-58625934
Dang, tiktok is cruising for a bruising
It seems we've discovered that there's a good amount of value in looking for content rather than mindlessly consuming what's being fed to us.
It's the old (tongue-in-cheek) problem of telling your family you like cows... only to later dread the fact that every gift they ever give you from then on will be cow related.
It's not pedantry without purpose. The logical conclusion to algorithms are bad is the outlawing of all social media and news sites.
We don't allow 12 year olds to buy cigarettes because they're harmful and addictive. These apps are no different, they just don't cause obvious physical diseases like lung cancer.
It's clearly designed to entrap users in to addiction, but as a society we're still cool with it and it blows my mind.
My point is that if an industry has enough money and influence it doesn't actually matter if it's bad for people. Businesses will sell anything to anyone as long as it's profitable. We've known tobacco can cause cancer for almost as long as cigarettes have existed.
source: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4902755/
Here are the first five videos that were presented as an anonymous user:
https://www.douyin.com/video/7211500689002876164
https://www.douyin.com/video/7223312448332303668
https://www.douyin.com/video/7216732859879410982
.. where is the disinfo again??
But to think that this is just chaos theory is as naive as to think these behaviours are fully planned. That would mean that tiktok has no business strategy nor preferred outcomes to optimize towards.
This answers your question: the harm banning it does is the laws that will be pushed to make it happen. Meanwhile, as you said, Instagram stands ready to show the same kind of content to teenagers if TikTok goes away. I suspect the actual harm reduction will be minimal.
Who wants to bet e2ee capable apps hit the ban list next? All tech companies are gonna be hit with a "backdoor or ban" ultimatum from the US.
The CCP is an adversarial government with a strong incentive to weaken the US.
Talking about the CCP as if it were just some people with different ideas is not a good faith argument.
IT would be better to have an enforceable set of licensing. But that wont happen because 1st amendment noises (rightly or wrongly. I'm not sure that corporations should really be treated as humans in this instance, but US case law thinks it should since the 70s.)