Giving an audience something they never asked for is very easy when you don't have much experience interacting with them. That's where a lot of would be entrepreneurs and creators alike stumble. They forget (or don't know to) quantify the size of their obtainable market before taking action and building something.
Seeing as how LLMs just tell you what you want to hear and not what they think you need to hear I don't see this problem changing anytime soon. They might need to develop a different type of model to have it reason that way.
The problem with bouncing ideas off of AI is that you still need to know enough to know when something is likely a hallucination. Because unless you're double-checking it with some kind of regular cadence you're probably accepting fiction as fact. Its really easy to just trust everything these chatbots output because of the style of communication. I'll be the first admit I fall for this trap all the time.
You do know we're hemorrhaging and lot of finite resources to play these games badly, right? We're basically at laying on chaise lounge being fed grapes levels of hedonism. Make me a racist meme that copyright infringes multiple IP holders and when you're done play Sim City at competency level of a blind man.
So if I write a honey pot that includes my bank account and routing number and requests a modest some of $500 be wired to me in exchange for scraping my linkedin, github, website, etc. profile is it a crime if the agent does it?
That's still writing new code. Also, its kind of an extremely bad idea to do that because how are you going to test it? If you have to rewrite anything (hint: you probably don't) its best to do it incrementally over time because of the QA and stakeholder alignment overhead. You cannot push things into production unless it works as its users are expecting and it does exactly what stakeholders expect as well.
Hopefully they have a test suite written by QA otherwise they're for sure going to have a buggy mess on their hands. People need to learn that if you must rewrite something (often you don't actually need to) then an incremental approach best.
When I had an interview at Apple I asked them why they were using PHP, and what it boiled down to was momentum. They team was able rapidly satisfy requests and could navigate the codebase easily so they had no reason to rewrite it. Which amongst other insights helped shape my viewpoint that the best tool for the job is the one that you are (or the team is) most proficient in. What business is going to wait 3x as long because of technical merits of using a different tool? It doesn't really make sense.
If you actually cared so much about victims of fraud you wouldn't be mad about a fraudster getting caught. You'd care about all the victims, not just the ones who were victimized by Somalis.
The benefit of having a team of QA engineers create tests is their differing perspectives, so with LLMs being trained to act like affirmation engines you have to wonder how that impacts the test cases it creates. Its the problem of LLMs being miserable at critiques manifesting itself in a different way.
However, in saying that, I am by no means an AI hater, but rather I just want models to be better than they currently are. I am tired of the tech demos and benchmark stats that don't really mean much aside from impressing someone who's not in a critical thinking mindset.
Sounds like an antipattern being rebranded as a solution. I shouldn't have to precisely instruct AI on how to solve every problem. I should be able to give it requirements and with its vast knowledge it should be able to understand various design elements within a system like design patterns and make the appropriate change without me needing to tell it to look for those things.
Honestly, I am inclined to think a lot of the people who are wowed by benchmarks and simple tech demos probably aren't doing very much at their day job and if they're either working on simple codebases or ones that don't have very many users(more users == more bugs found). When you throw these models at complex software projects like SOAs, big object-oriented codebases, etc. their output can be totally unusable.
I don't think its that surprising. Bash is old as dirt and scripts by definition are meant to be simple. Where AI struggles is when you add complexity like object-oriented design. That's when the effect of it trying to solve every problem in a way unique to it just takes shit off the rails. LLMs known design patterns exist but they don't know how to use them because that's not how deep learning approaches problem solving.
Generally speaking, humans are more often than not the weakest link the chain when it comes to cyber security, so the fact that most of their access comes from social engineering isn't the least bit surprising.
They themselves are likely to some extent the victims of social engineering as well. After all who benefits from creating exploits for online games and getting children to become script kiddies? Its easier (and probably safer) to make money off of cyber crime if your role isn't committing the crimes yourself. It isn't illegal to create premium software that could in theory be use for crime if you don't market it that way.
Israel stole US nuclear secrets to create their own nuclear weapons program, they killed American navy men, they destroyed 90% of the buildings in Gaza and very blatantly committed genocide in the process of doing so, Palestinians prisoners are commonly held without trials or charges i.e. they're hostages. Zionism literally cannot exist without them committing ethnic cleansing because everywhere Israelis live used to be Palestinian properties.
Honestly, what is your point? What are you seeing that the rest of us aren't getting? For the record, my mother's family is mostly Sephardic.
So how do you catch the errors that AI made in the pull request? Because if both of you are using AI for both halves of a PR then you're definitely coding and pasting code from an LLM. Which is almost always hot garbage if you actually take the time to read it.
Do you also teach kids about jail time and/or being blackballed by the industry? Because no matter how well-intentioned you are I can see a 13 year old me doing the naughty thing.
A few years back I was working at a software company that provided on-site sensor sensor networks to hospitals, pharmacies, etc. Our product required them to physically install a server on-site, but we were starting to get disrupted by cloud-based solutions. Essentially what we did was alert medical staff when blood, organs, etc. refrigeration temperatures went out of range. If the right people involved did not get notifications on time for these issues people will die. Its not hyperbole, you have to wait years for liver transplant. Their aren't just new livers available for everyone if a handful of them spoil.
With that being said, the problem here isn't that it took 75 minutes to find the root cause, but rather that the fix took hours to propagate through the us-east-1 data center network. Which is completely unacceptable for industries like healthcare where even small disruptions are a matter of life and death.
This article is really only useful if LLMs are actually able to close the gap from where they are now to where they want to be in a reasonable amount of time. There are plenty of historical examples of technologies where the last few milestones are nearly impossible to achieve: hypersonic/supersonic travel, nuclear waste disposal, curing cancer, error-free language translation, etc. All of which have had periods of great immediate success, but development/research always gets stuck in the mud (sometimes for decades) because the level complexity to complete the race is exponentially higher than it was at the start.
Not saying you should disregard today's AI advancements, I think some level of preparedness is a necessity, but to go all in on the idea that deep learning will power us to true AGI is a gamble. We've dumped billions of dollars and countless hours of research into developing a cancer cure for decades but we still don't have a cure.
I've got a hard time believing most folks are clicking on the sources that LLM chatbots sometimes provide with their answers. Especially because they've been hard sold on the idea that AI chatbots are as smart as the smartest people on the planet. They're likely just going to do what normal folks do when smart people answer questions i.e. assume they're right and move on.
Bro, you're not a museum who's invested thousands or more into a single piece. Paying $2000 for the framing service to be done right is worth it when you're protecting a big investment.
The reason LLMs suck in plenty of brownfield projects is because those codebases likely either implemented frameworks in a proprietary way, maybe did not rely on any public framework at all, or were in general done in an esoteric way and therefor few (if any) similar codebases exist within the LLMs training data. Which is problematic because LLMs aren't capable of reasoning or learning they're literally just predicting the next most likely token in a chain similarly to how autocomplete works. Without you supplying additional context and explicitly defining guardrails for preforming common tasks the LLM has no frame of reference for working with your codebase.
Like 2 years ago I did a similar thing except my goal was to sell instructions on how to see a Djinn in your dreams. Which is a pretty common thing for middle-aged Arab women to claim they can do, and they'll usually tell their husband their genie saw some awful thing they did and them blame them for it. Where most Arab men saw a problem I saw an opportunity. I tricked an LLM into writing a lucid dreaming guide which would have you see a genie in your dreams, and then combined those instructions with common hand-drawn talisman techniques used for Arab Magic to make a complete, believable product.
Not only did that product idea work but I had repeat customers. However, eventually I ran into someone who's intensity scared the absolute shit out of me, and I realized I could very easily enable an unhinged person who's capable of committing acts of violence. I talked him down and immediately thereafter faked my death so I could close the store.
You do know that amphetamines have a different effect on the people who need them and the people who use the recreationally, right? For those of us with ADHD their effects are soothing and calming. I literally took 20mg after having to wait 2 days for prescriptions to fill and went straight to bed for 12 hours. Stop spreading misinformation about the medications people like me need to function the way you take for granted.
Honestly, why would anyone find this information useful? Creating a brand new greenfield project is a terrible test. Because literally anything it outputs as long as it looks good as long as it works following the happy path. Coding with LLMs falls apart in situations where complex reasoning is required. Situations such as having debugging issues in a service where there's either no framework in use or they've significantly modified a framework to make it better suit the authors needs.