Comment by bartread | Hacker News Reader

Comment by bartread | Hacker News Reader

Parent Full thread

I had a complete shocker with all of Claude, GitHub Copilot, and ChatGPT when trying to prototype an iOS app in Swift around 12 months ago. They would all really struggle to generate anything usable, and making any progress was incredibly slow due to all the problems I was running into.

This was in stark contrast to my experience with TypeScript/NextJS, Python, and C#. Most of the time output quality for these was at least usefully good. Occasionally you’d get stuck in a tarpit of bullshit/hallucination around anything very new that hadn’t been in the training dataset for the model release you were using.

My take: there simply isn’t the community, thought leadership, and sheer volume of content around Swift that there is around these other languages. This means both lower quantity and lower quality of training data for Swift as compared to these other languages.

And that, unfortunately, plays negatively into the quality of LLM output for app development in Swift.

(Anyone who knows better, feel free to shoot me down.)

Going from past discussions, there seem to be two issues there. One is that Swift has changed massively since it came out and huge swathes of examples and articles and such online, that LLMs are trained on, are out of date and thus pollute the training set.

Another issue is that Apple developer docs are largely sequestered behind JavaScript that makes them hard for scrapers to parse.

At least, those are the two explanations I’ve seen that seem plausible.

Yeah, I'm not a Swift expert by any means - this is literally something I spent a few days on - but this in particular:

> One is that Swift has changed massively since it came out and huge swathes of examples and articles and such online, that LLMs are trained on, are out of date and thus pollute the training set.

100% jibes with my experience. The amount of times it would generate code using a deprecated API, or some older mechanism, or mix an older idiom with a newer one... well, it was constant really.

And a lot of Googling when I was fixing everything up manually drew me toward this same conclusion: that high quality, up to date information on Swift was in relatively short supply compared to other languages. Couple that with a lower volume of content across all Swift versions and you end up with far from great training data leading to far from great outputs.

> Apple developer docs are largely sequestered behind JavaScript that makes them hard for scrapers to parse.

Yeah, and honestly - even if there's a solution here - the documentation isn't that great either. Certainly not compared with .NET, Ruby, Python, TypeScript, etc.

If I were a vibe coder I'd certainly avoid Swift like the plague.

(Btw, this isn't a knock on Swift itself: as a language I didn't mind it, although I did notice when debugging that the Objective C underpinnings of many APIs are often on display.)

As someone who gets useful Clojure out of Claude quite consistently, I’m not sure that volume is the only reason for output quality.

I think what you are saying is true for CLI-only development using Swift. It is possible, but LLMs often get the commands wrong or don't realize how to accomplish something. There have been a number of times when claude/codex has told me I have to edit a plist manually in XCode before progress can continue.

This is more or less my experience with Go right now.

For a bunch of reasons I want to avoid the standard React, Typescript, and Node stack but the sheer velocity that might enable from the LLM side might make it worth it.

Wait...

Are you saying that your experience with Go has been bad? I would think Go would be as good as any other language (if not better). The language itself is simple, the Go team is very methodical about adding new features so it changes fairly slowly, it has excellent built in CLI based tooling that doesn't require third party packages or applications, and there are plenty of large open source Go codebases to train on. Seems like the perfect language for agentic tools.

Yep my experience has been pretty bad. As in Claude with Opus can rarely produce even compiling code in my particular project (a year old, mid-complexity one). This is with adhering to best practices including a robust Claude.md and detailed PRD's.

Reply on news.ycombinator.com