Slopsquatting
en.wikipedia.org
en.wikipedia.org
It would be a good idea to disallow registering packages which only differ by '-'/'_'. Rust's crates.io does this, so if you register `foo-bar` you cannot register `foo_bar` anymore.
Underscore is just capital hyphen.
At the risk of perhaps misunderstanding or committing a category error, I wonder if there's such a thing as a category of "correct" hallucinating, distinct from things that are, in some sense, "known" via training (e.g. I read about prompting of one model showing it was able to accurately recreate most of the text of Harry Potter, so clearly it's "in there" somewhere).
An interesting upshot of that could be that models "grow" their own knowledge in an evolutionary way via hallucinations that are retained rather than pruned as part of routine filtering and training.
Though I'm sure some might suggest "hallucinating correctly" is just one of the same with ordinary b function. I wouldn't agree with that but I could at least see the argument.
> Slopsquatting is a type of cybersquatting.
I feel like this is going to fall under notability eventually
I'm all for having lots of small Wikipedia articles, but the past few years they've been tending toward combining small articles together. And this is more like a dictionary entry than an encyclopedia entry.
If hairdressers have to take time to learn how to not cut people's ears off, people publishing applications should have to learn basic security practices. I think you will find that no one finds this controversial. And yet, we are moving to a world where AI is making it easier than ever for an army of vibe coders to make apps without knowing the literal first thing about security.
I guess my biggest concern, with parallels to that time I sold life insurance, is that they test for one thing and then in practice you do a different thing. I hear the same is true for realtors. So.. it becomes an exercise in memorizing some BS that you won't use again after the test. If we do this, the software engineering test would need to be updated at least annually, and better be written by some well respected security researchers.
I think like most hypothetical discussions, the commenters proposing these ideas aren’t interested in practical versions of the idea with tradeoffs. They imagine a perfect version of it in their minds with no downsides that accomplishes everything they want.
The demand for professional licensure doesn’t even make sense in this context. Is professional licensing supposed to stop developers from naming their packages names that LLMs produce? Is it going to force the package repos to check that everyone has a professional license before submitting packages from the United States (or other countries with licensure)? Can it be worked around by changing your country in the drop-down box to a country that doesn’t have licensing?
The calls for software licensure never seem to take into account the global nature of the Internet and software development.
Adding a link to your verified license in your package.json or personal website so that installers can check that the author of the package they are using does have a license sounds perfectly fine.
Proving you reside or are licensed in some country before you can publish to that countries repository sounds very doable too.
We don't even have to do this perfectly. It's not about preventing people from skirting the system, it's about giving users and developers the option to install from only verified sources.
Would you rather get heart surgery from a licensed doctor or an unlicensed one? What if both existed where you live? I'd probably ask to see their license before going through with it.
Optimally, you'd probably have seniors do some "Security Compliance Certification" and the company do it, then the product has to be approved by the certified, and if an issue arises, the certified get to be reprimanded, especially the company certification in some exponentially scaling manner so that it doesn't become the cost of doing business.
2. Computing was new and mysterious and developed faster than lawmakers could understand it, and by now it's given so much power to the top 1% that they're for all intents and purposes above the law. Cosmetology licensure is from a time when legislation still helped us.
Protectionism by a de facto trade guild was always my assumption.
There are a lot of activities where bad practitioners present significant danger to society and licensure makes sense. I never understood how cutting hair rises to that level. I'd love to know how licensure in the barber profession is anything other than a bald-faced attempt at building a moat. It seems like the market could correct for a bad practitioner in the barber space pretty easily, and with little risk to society.
Why do you assume that? I bet most people don't know their barber personally, and just go to the shop to get a cut. Should getting a haircut be fraught with having to go online and read a bunch of reviews, followed by the inevitable bickering between fake reviews and fake responses on top of that? No, I just want to get a decent cut for a decent price. We can nitpick over how much training is reasonable, and sure there's an element of protectionism there, but if the Internet had taught us anything, it's that online reviews are bullshit. I would hate to have to rely on them to correct for a bad practitioner when they aren't really able to do anything about bad doctors, which has a much higher bar to practice.
I've had bad haircuts too. And I have the simplest hair cut ever. Just buzz it off. But noooo.. on multiple occasions they've missed way too many strands of hair.
My wife and her friends make personal recommendations about stylists frequently. My male friends either go to chain barber shops or they have a personally-known barber (and, in one case when an older barber retired he gave personal recommendations for a new barber). At least in semi-rural Western Ohio I see a lot of word-of-mouth for barbers. I'm an oldster (48), though, so I don't know how the youth handle this. (I also haven't participated personally because I haven't had a haircut in 30+ years...)
sooner or later command line interfaces will require background checks and be limited to a close select group of government approved individuals, e.g. like guns in japan.
This was just after the Optus leak. Some hundreds of thousands of customers' data, down to the passport and DOB level, leaked. Again. I was going to ask him whether we, the collected IT consultants in the room, simply couldn't be trusted any more.
We've proven that we can't. I firmly believe that independent companies should no longer, by law, be able to collect my identifying information. If you must identify me, the state should provide a service. You hand off to them, they validate me, they send you a token back, I'm validated.
Sadly the microphone never made it to my corner of the room.
is how to "manually" (semi-manually) tweak the LLMs parameters so we can alter what it 'knows for sure'
is this doable yet??? or is this one of those questions whose answer is best kept behind NDAs and other such practices?
They don't 'know' anything. They are a many-dimensional matrix of the next most likely syllable given all syllables that have come before (roughly speaking).
To ask what it 'knows' is to ask why a chicken crossed the road.
Inbefore people telling me "akshually we know all about bla bla bla..." no we dont.
Put differently: GPT-4 isn’t a knowledge base, it’s a *Bayesian autocomplete* over dense vectors. That’s why it can draft Python faster than many juniors, yet fail a trivial chain-of-thought step if the token path diverges.
The trick in production is to sandwich it: retrieval (facts) LLM (fluency) rule checker (logic). Without that third guardrail, you’re betting on probability mass, not truth.