Yeah their work thus far has been an incredible public service to the Python community.
3,222 karma · joined February 9, 2010
When I do maintainer work that involves skimming a list of dozens of commits for information, it's very helpful to have concise commit titles.
So many tools go the direction of asking AI to generate a commit message for you, which will spend most of its space repeating in long form information available in the Git metadata, like the list of files changed.
And one can't really expect it to do a great job, if the information of "why did you change this?" wasn't available to it.
Meanwhile, AI coding seems likely to have the impact of more security bugs being introduced in systems.
Maybe there's some story where everyone finds the security bugs with AI tools before the bad guys, but I'm not very optimistic about how this will work out...
So many companies buy Microsoft regardless of the quality of the actual products. Given that context, why would directors invest in an expensive, invisible effort like controlling quality, when they could spend those resources on product launches that are naturally legible to upper management?
But I've seen fewer stories of that sort of thing with Android apps. Maybe the app store review process is able to catch it? But just as likely to me is that it's harder to discover that a mobile app is now maliciously sending data somewhere.
- The class of threat is interesting and worth taking seriously. I don't regret spending a few minutes thinking about it.
- The idea of specifically targeting people looking for Crypto jobs from sketchy companies for your crypto theft malware seems clever.
- The text is written by AI. The whole story is a bit weird, so it's plausible this is a made up story written by someone paid to market Cursor.
- The core claim, that using LLMs protect you from this class of threat seems flat wrong. For one thing, in the story, the person had to specifically ask the LLM about this specific risk. For another, a well-done attack of this form would (1) be tested against popular LLMs, (2) perhaps work by tricking Cursor and similar tools into installing the malware, without the user running anything themselves, or (3) Hide the shellcode in an `npm` dependency, so that the attack isn't even in the code available to the LLM until it's been installed, the payload delivered, and presumably the tracks of the attack hidden.
I think there's ways to manage the communication with users around which cases it is surprising/suspicious for the app to require that functionality. Personally, I don't love the model that apps ask for certain permissions but aren't required to explain in a way that can be verified by app store reviewers what they need those permissions for.
And even if one doesn't want every consumer to have to explicitly consent to the permission, it seems to me like you could still have an opt-out mechanism, so that the paranoid among us can implement a more restrictive policy, rather than giving up on the idea of having such a permission entirely.
A relevant threat scenario is when you're using your phone in a public place. Modern cameras are good enough to read your phone screen from a distance, and it seems totally realistic that a hacked airport camera could email/password/2FA combinations when people log into sites from the airport.
Ideally, you want the workflow to be that you can copy the secret code and paste it, without the code as a whole ever appearing on your screen.
This attack wouldn't work if every app, even an "offline game", has those implicit permissions by default. Many apps should at most have "Only while using the app" permission to access the Internet. Which would not be complete protection -- there's always the risk you misclick on a now-malicious app that you never use -- but it would make the attack far less effective.
Peter Pomerantsev's books are eye-opening on the previous generation of this class of tactics, and it's easy to see how LLM technology + $$$ might be all you need to run a high-scale influence operation.
And I think that is actually the most natural meaning for "vibe engineering", actually: Parallel to "vibe coding" where you serially prompt the AI to write the code for you, "vibe engineering" should be serially prompting the AI to do the entire engineering process for you.
I also predict that a precisely defined term for what Simon is describing will inevitably end up being primarily used by people who are actually doing "vibe engineering". Being disciplined and careful is hard.
People and organizations love to claim/pretend they're doing expensive, mostly invisible work like "building secure software". Given nearly every organization claims they use security best practices no matter what their actual practices, I imagine it will be that way with "actually reading and verifying what the LLM generated for me".
Certainly I've been disappointed how often someone presents something they've "written" in recent months that turns out, on inspection, to be AI slop that the "author" hasn't even read every sentence of carefully.
While I can't promise we won't ever change our exact monetization strategy, we're not venture-funded, and thus are immune to the usual enshittification pressures. https://zulip.com/values/ has some context.
(I lead the Zulip project)
And yet the "red flag" being discussed is Zulip having monetization for self-hosted business use? (Mobile notifications have always been free for most communities, and we have discount programs for various use cases detailed on our pricing page).
Look, in 2025, and one should be very wary of rugpulls. But Zulip has no venture investors. I've personally funded the project for almost a decade now, so that it can operate in line with our values (https://zulip.com/values/).
I want to use applications that are ethically managed, self-hostable, privacy-supporting, open-source, and excellent. Zulip aims to be that kind of project, and even with all the community contributions that we've fostered, I don't see how we could maintain Zulip responsibly without our professional team.
Should it be a red flag for an open-source application to have monetization that charges businesses for using services operated by its professional team? Or would the red flag be a project that lacks a professional team who one can count on to maintain it responsibly?
Our Flutter experience over the last few months since launch has been very positive. Most importantly, development velocity is much faster than it was on React Native.
What's dangerous is the framing; many E2EE messengers give the server a LOT more power than "just stores the ciphertext". https://news.ycombinator.com/item?id=33259937 is discussion of a relevant example that's gotten a lot of attention, with Matrix giving the server control over "who is in a group", which can be the whole ball game for end-to-end security.
And that's not even getting into the power of side channel information available to the server. Timing and other side channel attacks can be powerful.
Standard security practice is defense in depth, because real-world systems always have bugs and flaws, and cryptographic algorithms have a history of being eventually broken. Control over the server and access to ciphertext are definitely capabilities that, in practice, can often be combined with vulnerabilities to break secure systems.
If the people who develop the software are different from those who host the server, that's almost certainly software you can self-host. Why not mention self-hosting in the article?
If you're shopping for a third party to host a self-hostable E2EE messenger for you. The framing of the server as just "storing ciphertext" would suggest trustyworthyness of that hosting provider isn't relevant. I can't agree with that claim.
Authoritarian jurisdictions with a modus operandi of compelling their businesses and citizens by force are thus much riskier than Western democracies, even flawed ones. I at least expect it's a lot harder to say no to demands to break your promises that come with credible threats of torturing your family.
I'll also say that it's quite hard to make a messaging app without the servers that run the service having a great deal of power in the protocol. Many types of flaws or bugs in a client or protocol go from "theoretical problem" to "major issue" in the presence of a malicious server.
So if end-to-end security is a goal, you must pay attention to not only the protocol/algorithms and client codebase. The software publisher's risks are important (E.g., Zoom has a lot of risk from a China-centric development team). As are those of the hosting provider (if different from the publisher).
And also less obvious risks, like the mobile OS, mobile keyboard app, or AI assistant that are processing your communications even though they're sent between clients with E2EE.
Reflections on Trusting Trust is still a great read for folks curious about these issues.
We once had a Zulip update rejected by Apple because we had a link to our GitHub project with the source code for the app in the app itself. And it turns out, if you then click around GitHub, you can find a "Pricing" page that doesn't pay Apple's tax.
Details are here for anyone curious: https://news.ycombinator.com/item?id=28175759
For a community: A lot of communities want anyone in the public to be able to join their spaces and read their channels. For that use case, E2EE makes the chat system slower and less usable, with limited security benefits over using web standard encryption.
What E2EE may protect you from is a malicious server operator reading the organization's messages. If the server operator is a leader in the organization already, that person may already be directly a recipient of all the interesting messages anyway. The practical benefit I see for E2EE in Zulip is mainly for Zulip Cloud or other settings where a third party is hosting the Zulip server for you.
As for decentralization, a self-hosted Zulip server only talks to external infrastructure for sending notifications (Emails, mobile push) and to implement user-controlled features (E.g., outgoing webhooks). The Zulip API is fully documented, easy to understand and implement, and has community-developed clients.
Maybe you're thinking of federation? Matrix has fancier mirroring functionality, which can be important if your use case requires sharing dozens of channels with dozens of other servers. But Zulip is supported by Matterbridge and has various nicer mirroring integrations for sharing a channel with another server on various protocols.
But I don't know of many use cases where the usability of the core chat system isn't far more important than the usability of federation features.
Society is within its rights to demand that financial institutions both (1) protect their customers' sensitive personal information and (2) fight money laundering, which AFAIK is impractical without KYC rules at institutions like Coinbase that connect crypto to the traditional monetary system.
I can't say that the mobile notifications model is perfect. I have my frustrations with it. One of them that might not be obvious is that various military/government installations on airgapped networks are all freeriding.
Alternative options are (1) having no meaningful monetization of self-hosting or (2) moving away from being fully FOSS.
What would you like to see us do?
I think they've had a pretty frustrating experience with corporations freeriding on their work. See, for example: https://news.ycombinator.com/item?id=34129623.
I wish them the best of luck, and hope they're seeing success. Certainly in 2025 Europe should be thinking hard about the risks of a huge portion of workplace communication happening in Cloud-based chat and email applications owned by a few US companies (Microsoft/Google/SalesForce).
Honestly, we've had the same problem that Arathorn highlights in that thread. When I first started, I imagine businesses self-hosting Zulip would pay for support and fund the project that way. In practice, Zulip is too easy to operate for self-hosters to do that, and the better we make the product, the less support people actually need. The support-only business model can work, but I don't think it's workable for chat.
For example, Mattermost's $10/user/month plan is proprietary software with roughly the features that Zulip provides as entirely FOSS (with a $3.50/user/month push notifications service).
By the way, since you mentioned Signal: Signal is great, but it's really just not comparable.
Signal is a SMS replacement/messenger app with minimal features that requires very low COGS per user, and launched with a $50M grant from a billionaire. Zulip is a team chat app designed to replace much more complex and capable products (Discord, Slack, Microsoft Teams).
While I'm here, the Flutter rewrite of the mobile app is launching next month, and while the initial launch won't add much functionality over the previous React Native apps, the rewrite is way faster, less buggy, and a lot more pleasant to add new features to.
For what it's worth, Zulip is entirely FOSS. In contrast with most commercial open source software on the market today, Zulip charges only for Cloud hosting, push infrastructure, and support -- not the software itself. We offer free or highly discounted plans for non-business use.
And yet, even we get attacked because we stopped letting businesses use our push infrastructure for free too. How are we supposed to publish a professional quality self-hostable product without any monetization?
If you oppose all forms of FOSS monetization, no matter how reasonable, you're advocating for a world where FOSS cannot compete in many product categories.
And if you want FOSS to succeed in team chat specifically, the real issue is that Microsoft Teams and Slack have entrenched their duopoly with some pretty effective anti-competitive tactics (Microsoft Bundling and Slack Connect, most importantly), and that fact isn't on many people's radar as an issue at all.
Even when no errors are introduced in the process, the outcome is always bad: 3 full paragraphs of text with bullets and everything where the actual information is just the original 1-2 sentences that the model was prompted with.
I never am happy reading one of those; it's just a waste of time. A lot of the folks doing it are not native English speakers. But for their use case, older tools like Grammarly that help improve the English writing are effective without the problematic decompression downsides of this class of LLM use.
Regardless of how much LLMs can be an impactful tool for someone who knows how to use one well, definitely one of the impacts of LLMs on society today is that a lot of people think that they can improve their work by having an LLM edit it, and are very wrong.
(Sometimes, just telling the LLM to be concise can improve the output considerably. But clearly many people using LLMs think the overly verbose style it produces is good.)