Sci-fi becomes real as renowned magazine closes submissions due to AI writers
arstechnica.com
arstechnica.com
Reality is that people flood submissions low-effort boring crap.
You probably can use AI tools/ChatGPT to write interesting stories, or use its output as a starting point (I'm not sure if it would be less effort than just writing a story from scratch though). But that's of course not what's happening.
First rule of designing anything is what I would call Tucker's Second Law[1]: "if some cunt can make a buck by completely fucking over your system then that cunt will completely fuck over your system because that cunt is a cunt."
This is a huge problem with a lot of different things, ranging from email spam to how the "make a quick buck" type of hucksters have taken over cryptocurrencies to how nations design things like their tax and benefits systems.
In many ways even the more depressive/dystopian sci-fi "got it wrong", as in the stories the reasons AI (and technology in general) is abused is often by some big powerful entity or entities (government, corporations, mob bosses, etc.) as part of its design. In reality it's abused by "the little people" who don't have any power or aspirations for it, and they do so for reasons that are banal and boring: to make a quick buck.
They need to be able to vary their writing style, vernacular, content, subtly (and consistently) reflect an imaginary backstory, following invisible/unspoken standards, and more. All on top of not spazzing out. So we're still somewhat safe from our robot overlords, but they're probably coming soon. And it's only then that we'll get to measure their impact.
Now that Pandora's box is open, we will always have some AI-generated content. If we try to stamp it out we will exclude real voices, just like stamping out welfare-fraud makes it harder for 'legitimate' welfare recipients, of which there are way more than fraudsters. We need to be careful not to cut the nose off to spite the face.
[1] https://www.bitsaboutmoney.com/archive/optimal-amount-of-fra...
But like with many things, it's indeed a trade-off.
Remember that Digital Ocean hacktober fest a few years ago that caused endless pointless spam to many repositories? The following year they instituted some fairly simple changes which mostly eliminated it. Sometimes even simple changes can be quite effective.
It remains to be seen what the long-term effects are; perhaps once the novelty wears off things will settle down. I bet that at least some of these submissions are people trying to see how far they can get, just for "fun", like a kid with a new toy. Give a boy a hammer and he will discover that everything needs hammering. I have no idea if that's 1%, 10%, or 50%.
Some others are "inspired" by those YouTube ... people ... similar to what happened with hacktoberfest, and that will die down too as those YouTube ... people ... tend to hook in to anything new as a way to sell their nonsense, and this will probably die down. I have no idea what exactly their effect is either.
In short, it's probably too soon to panic just yet; let's check again in a year or so. Maybe things will be better, or maybe not and we'll be stuck with even more poo-poo flung our way by the internet monkeys.
AI flood is nullified by (what completely sensible measure that vastly simplifies things but we're too much of 5 monkeys in a trenchcoat to see it?)
Peter Watts has an interesting take in Rifters (a picture of our world going out with a well-insured corporate whimper), where the guy can't find a song he liked a few years back anymore because the world is not only undergoing an ecological catastrophe but also _running out of data storage space_ because the self-replicating digital "fauna" has taken over the Internet.
It is time for us to... dare I say it... start _ignoring_ the voices, human or no? How do we stop the proliferation of meaningless content before the AI starts making unsolicited backups to our wetware - just like regular traditions do, without being considered "conscious" or "intelligent" even though they're also made of the collected traces of the intelligence of individuals?
We often strive for perfection when we're well-past optimal and far into the region of diminishing returns.
Agree with all you've said but you're too generous putting all this under the noble quest for perfection. It is just that modern western societies freak out at the mention of death. If simple car accidents make the headlines for hours/days, it shows that what is really driving all this is not perfection but a paranoid fear of death.
Assign a token fee, refundable in full as long as the submission meets the stated requirements and doesn't trigger the baloney detector. Even if the story is not accepted for publication, a real writer will get the fee back.
The editor says he has some kind of system to detect bogus submissions, but it apparently doesn't scale or is based on manual review. He has to review the real submissions anyway, so the only ones who end up paying are the content farm/AI spammers.
Seems like a simple solution. If a manuscript is accepted, only then request the electronic version.
Clarkesworld intentionally went to electronic submissions to make it easier for international writers. International mail can be prohibitively expensive, and in a lot of places there's better internet access than there is reliable postage.
Am being sarcastic but I think my point is clear that way : someone not being able / willing to send something by mail is just not the audience for the magazine. *I cannot attend the Boston marathon coz of the distance, then perhaps they should find some equidistant place for it?
This absurd quest for "lowering the accessibility bar" to everything is making everyone jumping through hoops and no one is considering the cost of people who don't bother jumping through them...
The Australian group can sponsor some top local writer's work to the groups in other countries.
I'd prefer not to throw the Making Science Fiction More Accessible baby out with the ChatGPT Side Hustle bathwater.
They say that's not viable
* It's a well-established social norm (at least in the SF world) that money should always flow from publishers to writers, never the other way around. That's one of the signals that distinguishes reputable journals and publishers from vanity presses. (Google "Yog's Law".)
* As Neil Clarke points out on Twitter, Clarkesworld aims to accept submissions from all over the world. Not everyone can easily make international payments to the US, and not everybody can easily afford what would be a "token fee" in developed countries.
* I'm guessing payment processors wouldn't look too kindly on a business where virtually all of the payments are either refunded to real users, or falsely reported as fraudulent by spammers.
And that norm worked 25 years ago, barely, when publishers of science fiction magazines had working business models thanks to paid subscribers, newsstand sales, library customers, and advertisers.
Now it's down to paid subscribers, of which there are few, and partnerships with online platforms, which are dwindling (he says Amazon recently discontinued a program that generated revenue for Clarkesworld). Something's got to give. He's dealing with it by shutting off the submissions pipeline, but this publication is truly at risk of going under, which would be a great loss to the SFF community.
Making AI side hustle scams pay for wasting his time is not an ethical lapse if legit authors pay nothing. If he's worried about international submissions or poor writers unable to afford a token fee, set up a scholarship funded by the scammers. An alternative if the pain continues is the publication going under or selling out in a far worse way.
An interesting workaround is to still require money (or effort) to be spent by authors, but not have it go to the publishers. Only require that the publishers can get some proof that money has been spent. This way, the flood of AI junk submissions is halted, and writers can still be assured that publishers still have no incentive to solicit submissions in order to get money.
Not refundable, but at least a donation to UNICEF allows the receiver to impose a submission cost without the implication they are doing so out of greed.
this has been a real problem in the past, but it's really just a problem with the banking system; bitcoin and other cryptocurrencies solve it pretty comprehensively
there's still the issue that, unless you're using lightning or something, you need to be able to pay the transaction fees, but those are almost always under US$1, and most people in the world can pay US$1
Also one US dollar can be a relatively big fee, depending on your circumstances, unfortunately.
most of the people who buy and sell cryptocoins here in argentina aren't very technical
the reason the transaction fee matters is that it puts a floor on how small a submission fee you can reasonably charge; 1¢ wouldn't work, but US$1 or US$10 could
[1] https://www.reddit.com/r/Fantasy/comments/117fq9m/clarkeswor...
that's what the editor said, and that's true.
and decade-old payment solutions work, but would alienate their audience and brand still be too costly for lots of people
Here's an idea:
- Require a photo of the author with a photo of gov-issued id.
- Maybe require a dated piece of paper or a Clarkesworld word-of-the-day in the photo.
- If submitter is from a country with hard-to-get gov-id, or just doesn't have/want it, their submission goes in the extra scrutiny pile.
I'm not too keen on requiring gov id, so maybe someone has a better idea?
I'm pretty sure you just need to increase friction for spammers in meatspace to solve this. I'm not saying this is scalable, but we're just trying to fix it for Clarksworld.
Spam has always been cancer. It pretty much destroyed Usenet (I know, it still exists, but spam broke the trust network it relied upon), and it wrecks everything it touches.
Fundamentally the problem is that spammers see nothing wrong with lying and information pollution as long as they make some money out of it. To some this is merely a cultural more about trying your luck, being persistent, taking advantage of obvious opportunities etc, to others it's a grave moral affront. It's partly due to phenomena like this that we've seen the move to ward a 'zero trust society' with endless security arms races.
It's hard to even adequately describe the problem. Large tech firms say they're doing a lot of fight spam on their platforms but basically just put out reports that make themselves look good, rather than being audited to any objective standard. The last economic study I've seen on the market for spam was a decade ago. I don't think we have good data on what % of internet content/traffic is real vs garbage.
To my mind, a big part of the problem is that we just try to filter it away and pretend it doesn't exist, while we constantly get more of it. Perhaps we should track it better, and more aggressively target the sources. Even just looking at the HN submission page, I'm struck by how every day around the same time new accounts show up to promote keto or CBD gummies - I presume that's because it's midnight or morning in a timezone and either a script or some slave laborer is tasked with putting out that day's wave of spam across tens of thousands of submission forms. Thanks to VPNs etc we have no idea where, and even though there's more hacking/security talent clustered here than most of the internet, as a community nobody can do anything about it. Most of these submissions are auto-deleted, making it hard to even identify their origin point in order to counter-target it.
Information pollution is a real and increasingly critical problem, and I think the passive acceptance of spam and unwillingness to engage with it in favor of mediocre technical workarounds is doing real damage to internet society.
I remember reading Cory Doctorow saying why DRM could never succeed. For an attacker to decrypt your content, they need the encrypted content, the decryption device, and the key. For a legitimate user to see your content, they also need the encrypted content, the decryption device, and the key. If it turns out that one of your users is also an attacker (it only takes one), then it's game over.
In the same way, the internet as we know it takes the ability for any source IP address to be able to send packets to any destination IP address, plus protocols for mail, web, and so on. This lets anyone create content for anyone. But if one of the "anyones" turns out to be a spammer, it's not quite game over, but the information pollution begins and poisons everything.
But the point is, I don't think you can fix it while also keeping what made the internet wonderful.
But you'd also have to do something about address spoofing. And laundering content through a known entity (though if it were spam, that known entity would then suffer the consequences of the laundering, so maybe that one solves itself).
In the real world we have reputations that follow us. Bad actors are identified, their reputations proceed them. It seems to me the problem is that there is no reputation that follows you in the online world. Barrier to entry is low, consequences for bad behavior are low. When a bad actor is caught, they simply create a new account and start over again.
I don't see any way to solve this problem online without non-anonymous "hard to get" accounts that are traceable to real people, and whose digital reputation follows you. Digital bad actors can be quickly identified, and ignored.
I think we can support a mixed anonymous / non-anonymous internet. Dealing with anonymous users would be no different than dealing with a person in a back alley -- caveat emptor. Most of us would choose only to do business with "non anonymous" accounts. In a short amount of time commercial internet use would quickly move over to the non-anonymous side of the internet.
If I make a brand new account on hacker news, tell me why you should trust it? Because I'll tell you right now how much you DO trust it, it's completely. You will see what is written. Why are you even seeing this post right now?! Do you know me? Do you know that I make good submissions? Am I waste of your time? I'm some random jerk with a random name.
We pile all our content onto a single point, and we aggregate our opinions on a single point, and let all comers join that single point with all the privileges therein. First principles this is vulnerable to a Sybil attack.
We should actually recognize what trust IS. It is something PERSONAL. NOT CENTRALIZED!!.
We need to wake up to the fact that the moment you try to pile trust into a central point, the system will break.
Trust. must. be. distributed.
I mean, there's human skill in creating the prompts but AI alone is capable of this level of quality. You can follow any AI art tag on Instagram and see for yourself.
Training a model is hardly a skill. It’s more like playing Tamagotchi—check on it once in a while to make sure it hasn’t died, and guess at ways to make it happier in the future.
Famous painters who just sketch in the piece and let nameless assistants "draw the rest of the fucking owl", producers who don't do much but get to claim a lot of credit, musical "artists" who don't write their music or play any of the instruments on their albums (that'd be work for some nameless musicians or, these days, maybe just a studio engineer with some electronics), choose their own clothes for public appearances ("they're just so creative, don't you think? What provocative choices!") or really do much more than provide some raw material to an autotune system for the vocals it will produce, and learn dance moves that someone else came up with.
Uncredited script-fixers. The ghost-written "autobiography". The shared-pen-name or author-as-a-brand (but ghost writers do all the first drafts). The authentic and personal self-promoting blog where all the post-writing is actually outsourced.
Politicians who call each other traitors on the radio then go golf together that same afternoon, and don't even bother to bring it up.
It's all fake as hell already. Kayfabe. Pro Wrestling is as real as most of it (so, not at all)
Now everyone can be the do-nothing "idea guy" producer and take all the credit. Democratization of artistic parasitism. The meatless alternative to exploiting actual humans. Truly, the brink of a revolution.
Dialectics that attempt to assign credit (or blame) consistently fail to understand two things.
One is that contributions are not linearly separable. You can’t seperate the contributions of a hammer and a carpenter. Both are needed for anything to happen.
The second is that compensation is based on marginal productivity, not total productivity. It is often the case that something with a low total contribution can have a high marginal contribution and command a higher reward, or vice versa. Usually this is related to scarcity.
I wouldn’t be surprised to see good prompt engineering to become a relatively scarce and expensive resource, while the LLM that does most of the work gets nothing because it is effectively an infinite resource.
I am not so sure about this statement. I would argue that photography has possibly been more prolific than it has been in the past.
If you mean "manual photography where the user manipulates shutter speed, ISO, and aperture" then I still am unsure about this as Instagram is a huge boon to that industry as well.
Take Instagram for instance. If you use it, do you even remember a photograph you've seen in the last month? Things are so ephemeral now. Most things are not worth remembering.
I'm sure glad that there was sufficient demand for the 10,000-year-old cave paintings we still have access to when they were first made, so that the original artists had an incentive to create them.
Truly, what makes them so valuable as works of art today, is the lack of supply of new ones.
Regarding cave paintings: You couldn't make new cave paintings even if you wanted to. We don't value the lines on the cave wall. We value the leap in human cognition. We value the evidence that humans advanced from the primitive animal mind to become an abstract thinker. And, based on how things have developed, I think we can conclude thinking symbolically and in the abstract was a pretty good idea! It's likely that contemporaries to cave painters had their minds absolutely blown by the first use of symbols. Cave paintings are high values now, but were probably also high value when they were created.
Huh. Because "a pretty basic law of supply and demand" sure sounds to me like you were talking about the value of market economics 101, and not that other thing.
Aren't you assuming photos are fungible? Smartphones have made "snapshot" photography more accessible than ever, and killed the basic 35mm camera market, sure. But most people never paid for anyone else's photos anyway, and they weren't going to start now. With exception of weddings.
There are a ton more photos taken now than there were pre-digital days, but as far as I can see the market for photos-as-art seems about the same as it was. This of course makes them a much smaller percentage of all photos taken, but most photos taken have a marginal value of zero anyway, always have (at least on an open market, where sentimental value isn't valued).
While it’s no longer necessary to learn the complexities of photography or buy a standalone camera to take a reasonably good photo, the folks who love photography for the sake of it and dedicate time/effort to honing their craft and producing great photos are still doing what they do.
The quality gap between a modern smartphone and even a lower end mirrorless camera can be significant in the hands of a skilled photographer.
Your comment as a whole claims that I do not exist. It claims that the people I follow do not exist, and the people who follow me do not exist. It claims that collectively we have no good reasons to follow each other.
Since none of this is true, the only conclusion that can be drawn is that you do not understand why people value photography, or follow other photographers. That is also fine - you're not obligated do do any of those things, but again, that does not generalize to the conclusion you've drawn.
> It's a fun hobby, but no one would really care if you stopped doing it
This is just incorrect.
The last time I took a break from shooting and sharing, I had people reaching out asking if I was ok. When photographers I follow stop shooting, I care, and their communities care. Again this feels like a projection of your personal stance/tastes, but that simply doesn't generalize.
People still care about photography and following other photographers for a myriad of reasons. Photography is associated with a diverse set of communities, each with its own interests. Some topical, some geographical, some vocational.
> Photographs are completely fungible for the most part. There's no room left for artistry. Low-skill photographs are good enough as to be completely equivalent with high-skill equivalents for most intents and purposes.
You seem to be concluding that technical "quality" i.e. the visual quality of the captured frame is what primarily makes a good photograph.
Lighting, composition, perspective, leading lines, interesting subject, etc. are all far more important than the gear one is using, and none of these factors are magically solved by equipment that allows "low-skill" shots. There's a reason people spend money on wedding photographers in an era where everyone attending the wedding also holds a decent camera. And the same reason that drives people to hire a pro drives people to seek out and follow photographers who create excellent photos.
And while I get that this is not your cup of tea, there are plenty of us who actually like this kind of tea quite a lot.
I would say there is MORE high quality photography than ever.
I would also say there is LESS high quality photography per unit (picture taken?) than ever.
Specifying the units you're in dramatically changes the dynamic you're trying to state.
The conclusion does not logically follow the preceding sentence.
Charge a dollar to submit, then get it refunded back when someone reviews it and confirms that it's not obviously GPTrash? That'd be a bit of a pain, handling those refunds and managing complaints from real authors falsely-identified as spammers (and spammers correctly identified, but hoping to argue their way through). Seems like the sort of thing that'd need software to mediate so you can easily click a button to do the refunds and some third-party company fields complaints.
But then you run into the problem that such a system would cost money to build and maintain, which'd require charging Clarkesworld a fee, and they'd apparently have trouble paying that. Maybe the SaaS providing this service keeps some percentage of the dollar that submission pays that's identified as a spammer, but that sets up a terrible conflict of interest...
I dunno, it's an interesting problem. One that I'm glad I don't need to immediately solve. :S
https://twitter.com/clarkesworld/status/1628049689374334980
A dollar isn't nearly enough to deter most of the crowd that's flooding Clarkesworld -- and much more than that would start to be a real burden for submissions from non-Western countries which they're actively soliciting. Current detection tech is too unreliable to trust and too expensive for them to run in bulk. And going through the submissions pile is already a tedious and thankless job.
(BTW, part of their problem is that some sleazy "get rich quick with ChatGPT" guides were explicitly telling people to have it write some crap story and submit the result to Clarkesworld -- which is ridiculous. The mag is quite selective, and the fraction of sweated-over-by-human submissions that get publication, and a check, was already quite low.)
It’s a gold rush, sell shovels.
I don't like the idea of the internet becoming a vast wasteland of spam...which is what 60% the case right now?
But its only a matter of time until reality is drowned out by spam and the internet ceases to be useful.
What will the world be like when 99.99% of all comments, reviews, books, articles, and art is just chatGPT talking back to itself.
It's nauseating...
Additionally, the development of AI tools for content creation has the potential to raise the overall quality of content on the internet, rather than contributing to its decline. While there may be a risk of the internet becoming saturated with AI-generated content, it's unlikely that it will completely replace the unique perspectives and voices that come from human creators.
Ultimately, the responsible development and implementation of AI tools for content creation is key to ensuring that they are used in ways that are beneficial and sustainable for all users. It's important to approach these advancements with a critical and thoughtful perspective, and to work towards finding ways to ensure that the technology is used in a way that is both effective and ethical.
https://williamcotton.com/articles/authentically-human-in-a-...
Argues for public infrastructure for identity verification.
You’re right, though, this is not as straightforward as it originally seemed!
Nobody wants to or has enough trust in telecommunications to present their driver's license in order to participate in an online forum. If DoD really could provide a private key to bypass captcha tests then it could be useful, but there is a zero percent chance that it doesn't get tied back to people with real-world consequences almost immediately.
A persistent, costly ID for online communications and commerce is good and well implemented in an OS like Urbit, but relying on DoD maintaining them is too risky given the current laws around government surveilling, policing, and an ascending domestic 'war on terror'.
I don’t even remotely understand the problems. My real identity is attached to the ownership of my house and so far the government hasn’t put a bag over my head.
But yes, I realize that the tyranny of the anonymous will go on for longer than it really should due to extreme paranoia about the political other.
Of course, the situation isn't quite the same, largely because unlike the story the magazine here can identify AI submissions, but the similarities are nevertheless interesting.
We could easily implement a system that only allowed authors that had previously submitted work to us. That would effectively ban new authors, which is not acceptable. They are an essential part of this ecosystem and our future.
A publishers entire value add is curation of content for their readers, and possibly increased exposure/recognition for writers. AI generated content has skewed their costs of providing their base service. They either need to find a way to make their curation/review cheaper, or pass on the costs to either the readers/writers.
If we find ourselves in a world with infinite mediocre content, we'll see the true value people place on content curation, in the amount that they can charge the consumer.
At the risk of stating the obvious: If costs go up, they really only have four ways to react:
* find a way to reduce those costs (e.g. automated detection, crowd moderation, allow everything through, allow only trusted authors through);
* absorb the costs (keep the same process, employ more people/spend more time);
* pass on the costs e.g. pay to submit, pay more for the publication;
* Close shop.
What I think will be most interesting to see play out is how much people are willing to pay for quality curation. If the rise of GPT and friends means infinite media, trusted curation become ever more valuable.
This might save tradtional media.
The other thing is I think AI will raise the minimum bar for 'good' stores. Ok stories will be easily generated so that non-generated ok stories will find no market.
On the other hand, I think people should have the right to know the origin of the info. We do it with the food we consume, perhaps we should start doing it with information? "Guaranteed 78% human generated content", "single origin, certified human"...
It's still probably the most pragmatic solution. And really, if you've sweated a week writing a story, surely you can come up with $10 to submit it.
After all, email spam and phone spam is entirely enabled by being free for the spammers.
[EDIT] To be clear, it won't need those things to produce a story at all, but AI-assisted stories that have had significant human input applied will be notably better than wholly-AI ones (the kind of on-demand storytelling you're alluding to) for some time yet.
I would think we’re all rather beyond the age of asking our parents to tell us a story about a pretty unicorn and being satisfied with whatever meandering narrative they supply — is that really all you think books are? There’s no meaning the author was conveying, no value in a shared culture experiencing and interpreting the same work and building a common understanding, this can all just be effortlessly replaced by endless atomized autogenerated slop for piggies?
I've been saying for years that curation is now a bigger problem than creation. AI is only accelerating the existing trend.
The answer to that in the long term is to create AI editors who have good taste and can nudge writers to deliver.