rubendev

I think the solution is leaning on static types and hard to abuse abstractions to make programs correct by construction. It does mean that you’ll still have to use your brain to design the internal code architecture, and just use LLMs to fill in the boring bits between the safe abstractions.

It would still be running on that original laptop, no? So what did it replicate then? It can only possibly run on multiple thousands of dollars of compute and memory, and certainly not on an IoT device. The idea that AI is going to spread by itself is science fiction until compute and memory are so cheap and power efficient that my fridge can run it.

Yes, you need to apply output encoding or sanitization at the place where it is being combined with another string. Otherwise you don’t know the encoding which is needed. Even for HTML you cannot do it on the backend, because you don’t know if it will be injected into HTML PCDATA context (tags) or in HTML attribute context.

It is genuinely exhausting to rebuke the content of walls of LLM slop though.

A lot of this frustration comes from the fact that it takes a lot less effort to generate a wall of text than to actively engage with and rebuke the points that are made in them.

Don’t expect respect from your readers if you don’t respect them enough to formulate your own words.

That's not what the LLM hacking accidents have been like at all though. To improve the analogy, it would be like summoning a totally passive demon, giving it weapons and placing it next to a bank, place a small fence around it, and then command it to perform a totally safe "exercise" that is exactly like robbing a real bank.

LLMs do not have agency, they are just producing tokens based on a prompt that a person entered, and some of these tokens can trigger the tools that a person gave them access to.

I don’t think we should just accept that the average developer doesn’t know about these very basic hardening methods.

This is your chance to set yourself apart from LLM coding agents. If you want to call yourself a software “engineer” you need to start actually engineering, which includes knowing when and how to apply security principles and these hardening methods.

I reject the notion that “doing cybersecurity” is somehow a different job than software development.

I agree wholeheartedly. The solution is not to stop developing these so called “dangerous” AI models. The solution is to start properly engineering software.

I think the model was able to escape the sandbox and hack huggingface because they were incompetent or not giving enough priority to implementing basic cybersecurity principles.

If they would have done so, there wouldn’t have been an escape or a hack. The reason we don’t get much details is because the details are embarrassing for them.

No we just need developers to do the bare minimum of effort to write secure software. Most hacks are not super complicated vulnerabilities chained together, but just utter failures where authentication and authorization was simply forgotten or untested, or where nobody bothered to validate the data they receive.

The bar for software is so low that it is embarrassing for the entire profession.

I think the main problem with this is that you can be paid/coerced to vote for someone and be able to easily prove that you voted for the person they wanted. It’s easier to vote for the person you actually wanted to vote for, and maintain plausible deniability, if you go in person to a voting booth.

I can’t tell if the model card is written by AI or a parody of AI writing…

I think at minimum you would need to understand which theorems you want to prove about your code, and how to express those in Lean. Otherwise you won’t be able to verify the output. It may have proven some statement that is machine checked to be correct, but it’s pointless if you don’t understand what that statement means and if it covers what you want to verify about your code.

This is really a self inflicted problem. If you host your backend on the same origin as your frontend (using a reverse proxy) you don’t need CORS at all and you can use the vanilla SOP, and strengthen it further with a strict CSP.

What is your definition of faster to implement? Is it producing a plausible implementation, or is it faster at producing a correct and high quality implementation? Are you including time spent refactoring and fixing bugs in your metrics? If not, I think you are tracking a gut feeling rather than cold hard facts. I’m not saying this is easy to track, just saying that it’s hard to know for sure that you are really more productive with AI.

With a capable static analyzer that is not true. In many common cases they can deduce the possible ranges of values based on branching checks along the data flow path, and if that range falls within the buffer then it does not report it.

If you say you need the data for security reasons that’s all well and good, but then you can only use the data for that specific purpose. So you cannot suddenly start using it for targeted advertising just because you already have the data.

Please elaborate why you believe that? The ability to easily rotate encryption keys is considered an anti pattern?

Also, I gave the link to the appendix because there was a specific question about Argon2 parameters. For general developer audiences, they need to look at the standard itself which is a lot more high level about how to properly implement cryptography in software: https://github.com/OWASP/ASVS/blob/master/5.0/en/0x20-V11-Cr...

For the most common use-cases of cryptography like authentication and secure communication there is more specific, but still high level guidance that is useful for developers as well:

- https://github.com/OWASP/ASVS/blob/master/5.0/en/0x21-V12-Se...

- https://github.com/OWASP/ASVS/blob/master/5.0/en/0x18-V9-Sel...

- https://github.com/OWASP/ASVS/blob/master/5.0/en/0x15-V6-Aut...

Which one would you recommend instead? Referring dev teams to NIST standards or the like doesn’t work well in my experience.

Yes it’s an audit checklist for when you need to know specifically what to use and with which parameters.

It’s unfortunate if there are mistakes in there. The people at OWASP would be very happy to receive feedback on their GitHub I’m sure.

Can you give some examples of such commonly used libraries for languages like Java / C# / C++?

In my experience there are not many libraries like Google Tink around, and they are not in widespread use at all. Most applications doing encryption manually for specific purposes still have the words AES, CBC, GCM, IV etc hardcoded in their source code.

If you review such code, it’s still useful to have resources that show industry best practices, but I agree that the gold standard is to not have these details in your own code at all.

Yes I fully agree. I’m a big fan of libraries like Google Tink that make you pick a use case and use the best implementation for that use case with built in crypto agility.

Most crypto libraries are not built like that however. They just give you a big pile of primitives/algorithms to choose from. Then frameworks get built on top of that, not always taking into account best practices, and leave people that are serious about security the job of making sure the implementation is secure. This is the point where you need something like ASVS.

The OWASP ASVS appendix on Cryptography is one of the best and concise resources I know for this kind of thing: https://github.com/OWASP/ASVS/blob/master/5.0/en/0x92-Append...

If you only secure the login you will be sending your session cookies unencrypted for the other pages and they can be intercepted and used to impersonate you.

If the program has access to the credential, and the program is running on your computer, you also have access to the credential no matter how they try to obfuscate it.

What the game dev is supposed to do is have an account system on their backend, and ask the player to enter their credentials in the game. The game can then identify itself as this player to the backend servers. That way any actions on the backend can be attributed to a particular player and you have a good basis to make security decisions on.

You can set a CSP in the HTML head section using a meta http-equiv tag. It has similar functionality to X-frame-options IIRC.

A dark mode would be nice.