An aborted experiment with server Swift
flak.tedunangst.com
flak.tedunangst.com
I have recently been developing a server-side Swift application, a few months of sporadic evenings and weekends. My starting point was having a year or so of professional iOS experience and a ton of web/server/infra experience.
My experience with it has certainly been mixed, but in many ways a breath of fresh air compared to some web development ecosystems. At work I'm mostly using Java on the backend and I think Swift could be a really nice replacement for that for larger systems – the type system is better, the ergonomics are better, the developer tooling is much better at the lower end (although there's plenty of specialist JVM tooling that is better).
Ecosystem-wise, Swift has few but high quality libraries available in general. Apple's first party libraries are exceptional. Vapor is in a weird place though. Some parts are really mature and well considered for their relatively young age (e.g. parts of the ORM, async stack), others feel very early days for things that have been around for a number of years now (e.g. queues). Others feel like an impedance mismatch in the framework design – I find the Content system both too limiting and not providing enough.
Overall, I'm loving it for a personal project. Would I use it as the foundation of a new startup? Probably not. Would I use it in a iOS heavy company? Yeah probably. I think you can do productive, high-quality backend engineering in Swift, today.
I think a lot of the author’s problems started when he bailed on Vapor and went down into the depths of SwiftNIO. I’m not really sure this was necessary. I could be wrong, but it feels to me like Vapor should let you do pretty much everything that you would need for an ActivityPub server.
Writing request handlers in Vapor with async/await is delightful - it’s really not any harder than writing Flask code. But going down below that level is pain. Lots and lots of pain.
You're right about the NIO part. I've had to dig into it a few times, but rarely found it productive in solving problems. Vapor is pretty flexible if you drop past it's first layer of abstractions, and as long as you pick one of futures/async and stick to it, it's fairly straightforward.
Most Java tooling works with Scala out of the box or at most needs a plugin added, and interop when calling Java from Scala is very smooth. Even if it wouldn't be acceptable for production code you might consider trying it for migration scripts, admin tools, testing etc. (though personally I'd stay away from ScalaTest et al) - that's how I got my start with Scala many years ago.
FWIW Scala can actually be pretty useful in that space with Zeppelin - it's not quite as smooth as some of the Python-only notebooks out there, but sometimes being able to use your Java libraries in your notebooks makes up for a bit of clunkiness.
The community is a bit worried over these developments, and there has been some sniping over the disruption caused by Scala 3 (mainly to projects that relied on macros, even though macros were marked as an experimental feature).
IMO Scala is still the best language on the JVM, and last I checked there were plenty of jobs for experienced Scala devs. The foot guns weren't that severe, or at least I am used to them (in my opinion, Java coding has more foot guns due to nulls and mutable state everywhere, YMMV).
Scala 3 was more incremental, than radical, and was massively less disruptive than you see with other attempts to clean up a language (Python 3, Perl 6, anything involving PHP) due to good design, careful planning and backwards compatibility options.
The best side benefit for our Java heavy org was that the migration risk is very low due to excellent JVM/Java integration.
As the project progressed we started adding more in Kotlin and it massively improved developer experience.
Haha, what?
Linux/Windows tooling is lacking, I'll give you that, but given where Swift started from I feel like they're already outside the norm for Swift development and debatably "low end" on complexity (although not deployment, I haven't had a problem with Linux deployment).
By "high end" I mean anything that is trying to do complex runtime analysis, and the sort of tuning that you can do on the JVM that is way outside my comfort zone.
"Install Xcode, click New Project" is a much lower barrier to entry than setting up the toolchain in other ways.
They have a lot of shared ideas and even shared developers.
Swift is effectively ref counted rust imho
I know of a medium-size startup that is (attempting) building an industrial/IoT software stack on just Swift. Last I heard things weren't going great...
Erm, including the apparent signal race and the illegal instruction?
Yes. They're probably both caused by this person setting their CPU microarchitecture wrong.
I have been toying with the idea of experimenting with Swift under Linux myself for a while now. I guess this puts that idea to rest.
I'm not even talking about the illegal instruction thing or the capricious looking syntax changes. Just the fact that his hello world app caused 350MB of build artifacts in the filesystem, and the five minute build time for the project, are enough to turn me off of Swift.
I come from the good old Turbo Pascal under MS-DOS days. Building even complex applications took mere seconds on an 16 Mhz 80286 with 1 MB (yes, one megabyte) of RAM. I see no excuse for binaries to be big. This is turning me off of Go and Rust, too.
Gotta admire the cojones on those people to call their product "Swift".
I come from a time where the whole application fit into 512 KB RAM. It had to. And it was good that way, because the human brain can't grog that much context at the same time. Nowadays people are pulling in javascript frameworks larger than the whole payroll system of those old days without even a second thought.
I find the build times on Linux quite reasonable.
Users shouldn't be met with compile times that long on a "Hello, world" program.
For comparison, a "Hello, world" server in Rust with Axum and Rustls takes 25s and 16s in release and debug mode respectively.
https://darkcoding.net/software/a-very-small-rust-binary-ind...
Computers do much more than they could in the 80s.
I still don't understand why this argument is used as much as it is. Every language ends up having to deal with either hardware or system libraries, which means that any language that provides further assurances beyond what the platform does has to have glue code to maintain its invariants. Whenever these languages have to dereference pointers they use runtime checks which would otherwise not hold its invariants. Interfacing with existing libraries is the same. Using a C library from Swift doesn't make it an "unsafe language", it makes it a flexible one that can leverage the existing ecosystem without reinventing every wheel.
If Swift falls into the first category (maybe it doesn’t!), then this seems like a fair criticism to me.
swift, a general-purpose programming language built using a modern approach to safety, performance, and software design patterns, to create the best available language for uses ranging from systems programming, to mobile and desktop apps, scaling up to cloud services.
With these rather expansive pronouncements, one can reasonably assume Swift won't drop to C++ in the very next program after "Hello World".
If it was using the system library and a vulnerability was found in it and the distro shipped a fixed package, then the swift coding using it would benefit from that fix without any recompilation needed (for back-patching distros at least, which is from my experience what people use in production systems). Ideally, even if you manually compiled BoringSSL for your system and the distro wasn't providing it, you could build again from a newer version or at worst patch the source of the library you originally built and rebuild it. Automatically building a specific version of a library, which may not be (and likely won't be) the latest version of that library with the latest security fixes, is a real problem.
Now, I don't know if BoringSSL was pulled in from vapor, or a dependency vapor relies on, but it is problematic to have the language libraries do this, at least if there isn't an option to do otherwise (and maybe there was, and it just wasn't surfaced through the layers as vapor built its dependencies). This might very well also be a problem with vapor and not swift, as it could be other swift modules do the right thing.
From the article: "but it raises the bigger question of why are we compiling BoringSSL in the first place?" (emphasis mine). I think the "compiled" part of that is doing the heavy lifting.
> On Rust-land, I've noticed that some libraries like to have a fallback or compile time config flag to use the vendored version, but still provide a way of using the system's version.
This is probably a good middle ground, but it might be more harmful than helpful than if it instead just linked to direction for how to install it yourself (at the simplest level it's almost always ./configure; make; make install), because then at least it's obvious what happened and what libs are being used and you can find them (in /usr/local or wherever).
> I appreciate when both options are available because I've encountered some C libs that were very difficult to set-up on first run and the vendored version usually works out of the box.
Anything that the install process can automate should be easy enough to provide directions and maybe some simple helper scripts for. If it's too complicated for someone to do, how is it able to be done in an automated fashion by a dependent in a way that seems to work for most people? (and if it doesn't work for most people, why even offer it)
And yeah, safety is clearly a continuum: hardware is not perfect, operating systems are not perfect, etc. However, being defeatist about this point is not really the way to go. We should aim to replace less safe things, especially in safety-critical parts of applications like the cryptographic operations and TLS implementations, with safer things. That's definitely why in the Rust ecosystem there's been a huge emphasis on pushing rustls forward; it's a shame to put all of the work into making safe and robust software, only to have a giant OpenSSL or BoringSSL-shaped liability sitting in the same address space as your program. Battle testing alone has proven insufficient over and over and over and over, whereas statically verifying the absence of (at least some kinds of) flaws remains very promising still.
In practice I'm sure it's obvious that to make any application secure and/or robust it's going to take multiple layers of defense where ideally there is no single point of failure. But to make progress, you have to... you know, make progress. Programming languages, toolchains, and standard libraries can't stop your operating system from being insecure, but they do have the ability to not put less safe/verified things into your address space.
Maybe some day, all of these safe Go/Rust/Swift programs will have much more robust hardware and operating systems to run in that are, themselves, "verified" to have certain security assurances. For that to really be all that meaningful, though, the programs themselves have to actually be "safe".
And hopefully, even when standard library code can't be proven to be "safe" using the language's normal mechanisms, we can still prove it using heavier duty tools. But, it certainly isn't happening over night...
On some older operating systems, the system was safer in some ways.
On the Genera operating system for Lisp machines, almost all code ran in a single address space, which was possible because Lisp is garbage collected and doesn't allow arbitrary reads and writes through arbitrary pointers. You could still write code in unsafe languages like C, but those programs did need their own address space, like in traditional OSes.
On the Burroughs MCP operating system from 1961, it had a similar mechanism. It was written in the language NEWP, which had certain unsafe constructs marked, and if code used unsafe constructs, it had to be approved by the system administrator to be executed.
https://en.wikipedia.org/wiki/Single_address_space_operating...
If you use Java, your xml parser, json parser, character encoder, date/time formatter, jdbc library, etc. likely are mostly written in a JVM language.
If you use Swift, they were (at least in the first versions) almost entirely written in (Objective-)C, so the amount of unsafe code is larger. Wrapping that in a safe language doesn’t remove all security concerns of that code.
There's a big difference between using a C library which does extra processing + maybe uses some syscalls and doing the syscalls directly. The interface is much stricter for the system interface and the memory ownership is better defined. You don't have to think about the library using a different allocator than the app does.
So no, I wouldn't say every language needs to do that. Golang definitely didn't https://github.com/golang/go/issues/36435
The Linux kernel is a separate project from the several different libc implementations and the Linux kernel project treats the syscall API as a first class contract. If a language wants to use it directly for some reason that is an option on the table.
In Swift the best advice I found was https://forums.swift.org/t/read-text-file-line-by-line/28852...: use the C functions. I experimented with using FileHandle in Swift and manually scanning the return Data value for newlines, and found it added a 6x slowdown. The loop for scanning the newline does not optimize even the array access, it was calling a lookup function for every byte and there was of course no SIMD acceleration. Converting the Data value to a native Swift array did at least eliminate most of the lookup overhead, but from the profile it seems like it still does bounds checking on every access, so it doesn't optimize to SIMD or anything like that.
So that's the issue - the Foundation library isn't a portable wrapper over the native system calls, it is a thick wad of gunk that adds significant overhead to even simple programs. There was an effort to make a better API, https://github.com/apple/swift-system/, but it seems to have stalled (or perhaps an improved cross-platform API was never its goal).
What Swift currently isn't is thread safe. They're working on it for Swift 6, but it's still probably years away, and will be a massive breaking change. Vapor has done a great job putting a thread-safe layer on top of this mess, but once you go down to say that mentioned SwiftNio level, you really need careful with concurrent memory access.
Note that this is changing, on Darwin and Linux alike. Foundation.framework is transitioning from being a Swift interface over a C / Objective-C implementation to an Objective-C interface over a Swift implementation (https://github.com/apple/swift-foundation).
The language still has native C interop, and is gaining the same for C++, so it can delegate stuff like crypto to existing solutions like BoringSSL, but Swift-native solutions are becoming more common.
I've seen this one, "Move away from BoringSSL", actively in development (PR opened two weeks ago):
That's why I try to push back when people say Apple "doesn't care" about server-side Swift – they're regularly releasing these packages that they wouldn't bother open-sourcing and maintaining if they "didn't care." Things like NIO-based implementations of IMAP and SSH, an OpenAPI generator & runtime, HTTP type abstractions, a Cassandra client, etc. Most of those are from the last year, in addition to those certificate packages. They're paying teams of full-time employee salaries to work on this stuff, including open-sourcing it.
(Or to summarise the situation in another way, it got mercilessly shunned in terms of wider community enthusiasm versus Rust)
I am reminded of the Black Mirror episode "Arkangel". Children are their own people, they're meant to go into the world and be themselves. A programming language isn't a person but its destiny is distinct from yours, and it deserves the opportunity to be its own thing and develop without you hovering over it constantly.
That's some A-tier manipulation tactics.
(Casca in Julius Caesar, however the play is in this case describing an event actually mentioned in Plutarch)
The real story of server side Swift is a small but dedicated working group [0] continuing to make real measurable progress. Most of the frameworks have fallen out of favor or have been abandoned, except for Vapor, but the workgroup is active with representatives from several major tech companies. Obviously the workgroup's scope is much broader than just a single framework, and their current initiatives identify and address most of the shortcomings noted in the article. [1]
A supposedly wise man once said "...you can not fool all of the people all of the time." [0] Plenty of things that are usable are also failures (e.g. Microsoft Zune was, by all accounts, a competent media device). Any technology that is not winning [sufficient] mindshare over time will be considered a failure soon enough.
[0] What, you don't know who said this? Copy pasta plus a three-second search on any search site or AI chat will set you straight.
Defining "sufficient" here is important, and I think that server side Swift's progress meets a bar here that prevents it from being a failure. That might not last though.
The Swift FOSS folks are the ones that have tried to look into it, as there is hardly any other use case outside Apple's ecosystem.
I believe most end up learning this either the hard way or by simple observation ; and one cannot come to any other conclusion than avoid Apple stuff outside of Apple hardware...
The pain came later. They eventually deprecated Vapor 2, and I found that the new version of Vapor required essentially a complete rewrite, using some kind of async paradigm I couldn't quite grok. I kept using the program I wrote for longer than I anticipated, and more pain came when I could no longer figure out how to set up the toolchain needed to even build the thing. Eventually the only way to build and run it was to use an old Ubuntu 16.04 installation.
This experience left me with a distaste for modern tooling with lots of churn, so last year I decided to rewrite the thing in C with minimal dependencies, so it's guaranteed to compile with minimal modifications even in 30 years time. Really happy with the result[3]. I added new features, optimized the websocket messaging with binary messages and zlib compression, and generally made it way faster than needed.
It compiles in less than a second vs. the minutes it took to build the Vapor version, and the resulting binary is 324K with debug symbols, 144K when stripped + -Os, and 68K when UPX-compressed. It also uses a local SQLite db instead of a separate MySQL server, which makes deployment quite trivial (git clone && make run). The web client is still built with React, I've been thinking of rebuilding it with a single .html file and minimal JavaScript.
Googled around to get some more context on Vapor's history and sure enough, found this thread from today.
Sounds like I should point him away from Vapor in your opinion?
Vapor 5 is definitely going to be a breaking change, but that's not a bad thing. The current major version predates async/await, actors, Sendable, macros, move-only types, variadic generics, and lots of other language features. It's been able to adopt some in incremental releases, but it wasn't designed with those things in mind.
The direction of the language is of breaking changes that really should be breaking, secret development of features that then get thrown onto the community during a keynote and just an unfathomable amount of complexity and cruft building up in every corner. Seriously, I would hate to learn Swift now, I barely tolerate it having grown up with it.
Swift 3 was almost perfect. I loved it. If they had of added async/await to Swift 3 I would use it for everything.
As for C remark, our computers have long stopped being a PDP-11 clone.
Yeah, I'm sure he said that at the outset in 2014 when Swift launched, but in 2023, that's not where Swift ended up.
> As for C remark, our computers have long stopped being a PDP-11 clone.
There is absolutely no need for a language like Swift to be as complex and bloated as it is; browse the language spec, and about 50% could be discarded and you'd still have a very complex langauge.
The Swift team would do well to take inspiration from C and keep it simple; just stop adding new langauge features and let the existing language breathe. Full stop. Each new release keeps adding new features without any regard to how new ones negatively interact with previous ones. It's a language being designed by compiler engineers and not app developers.
And don't get me started on the bloated language parser - editing Swift projects beyond a certain size in Xcode still gets abysmally slow. All they need to do is lock down a core subset of the language and just fix bugs; make the compiler engineers speed up the parser. That's it.
Trap signal -> stop handling new requests -> wait for the requests to finish -> (force children to quit if needed at timeout -> exit
Outside of it, is as good as trying to ship a GNUStep application in Objective-C 1.0, while catching up NeXTSTEP and OS X evolution.
I give swift a try every once in a while.
Problems of my last attempt:
- The Swift REPL (lldb to be precise) depends on libpython-dev (v3.8). WSL (ubuntu 22.04) comes with 2.7 and 3.10, so the lldb dependency depends on an old python version. The fix was to copy the 3.10 binary to the directory where lldb expects it, and rename it to libpython3.8.so. It seems that it's backwards compatible so it worked on my machine.
- I tried to compile and statically link a hummingbird (web framework, more light weight than vapot, https://github.com/hummingbird-project/hummingbird) hello-world example. I had a liking error. Luckilly someone in the swift forums helped me. The problem was that my package.swift file had the "-static-stdlib" flag only as a swiftSettings. It needed to be also a linkerSetting.
In the end I was able to build my example. But at this point I already spent multiple days debugging problems that are unrelated to my applicationn or the Swift language.
It's a shame because I see its potential. Fixing the (linux/windows) tooling problems would probably only take "some" developers. Apple has near infinite money and could sponsor them if they wanted to, but they gain little by it. Besides them, not many are invested in it.
It is a OK language if you don't want to worry every memory allocations and wants a simpler / effective language to work with. From the other path (scripting language and expect to see performance improvements by moving to compiled language), you will be disappointed. Honestly, bun.js / deno with TypeScript is much more pleasant.
Do you mean Swift or Golang? If golang, why is bun.js more pleasant? I'm asking as someone who doesn't know either stack.
The alternative is Kotlyn but it comes with its own set of issues. Gradle, for one, although I see that you don't need to use Groovy any more (Good!). However, it remains a rather messy language with a lot of necessary Java interop. And by confusing I mean that everyone is trying to use the language to build their own DSL.
You can also use TypeScript, if Web clients are your thing. It's pretty performant and the build systems are somewhat easier to understand. Just stay away from ReactNative.
The rest, Go, C#, etc, are great server-side languages but you have to use a second or third language if you want a native app and web interface.
This is one area where the JVM ecosystem definitely takes a huge lead. There just isn't much use of native code in that space. And often a library that does use native code will only use it optionally, for extra performance if you want, but there's a pure Java implementation available as well that runs well enough most of the time.
That may start to change now binding to native code is getting easier, but it also may not. It's a culture thing as much as a tech thing.
This sort of exercise needs to get done to bring Swift up to date with other native stacks, be it runtime or tooling.
But also, I remember being highly motivated to learn Go and debugged my way all the way through the runtime to memory allocation stuff in order to understand why I accidentally made an endless loop of type conversions that filled RAM.
Learning is hard. Some learning contexts (Swift today on Linux apparently) are harder than others.
But ultimately this does seem to be standard issues you might see working with swift and lacking inference but the dev didn't know much about Swift to work his way out of it.
I do agree that the way Package.swift works and how it relates to your imports is really confusing. Not to mention how the name in the Package.swift relates to the package names your importing (they don't, they relate to the repo or directory instead of the Package names in the repo's Package.swift).
It seems like the dev just kind of randomly plugged away and they weren't really willing to learn anything and is holding everything at arm's length.
I also would not have known what to do about pretty much any of the problems he encountered.
It can be, it’s designed for that (as opposed to something like an Xcode project file), but it doesn’t come up for most of us.
Obviously it’s different if you’re not developing on a Mac. But given the usual audience of Swift users, I bet it’s hard to find good tutorials for this kind of stuff.
I really don't get Mac/iOS developer's weird stockholm syndrome with XCode -- when I worked doing iOS for a period of time, I used JetBrain's AppCode and it helped take the edge off things.
Granted, I haven't ventured into XCode & iOS dev since Swift came on the scene. That does look to have improved things.
I used to love Visual Studio. Not MFC and its crap-heap of hokey macros, but the IDE. Project Builder was primitive as hell by comparison.
Fast-forward 20 years, and Visual Studio is still plagued by the same ridiculous UI defects it suffered from in the '90s, and Xcode is vastly improved from Project Builder.
I've heard (and evidence bears it out) that there's no one left at Apple who understands how Xcode works at this point. The giant piles of settings, many of which are specified in multiple places in multiple ways, and the corruptibility of the project files... once your project goes wrong there's often no coming back. Granted, this is far less frequent than it used to be, but still... this thing needs a ground-up rewrite behind the scenes.
I work in Xcode daily, and I would be open to trying something else if I thought it would fully integrate all of the non-coding BS that encrusts development. Specifically things like code-signing, certificates, SDK locations, simulator integration... Do third-party IDEs offer seamless support for all that stuff?
when I used it, AppCode did a lot of song-and-dance to delegate out to XCode to do a bunch of the stuff like that. and it was by no means seamless
On the other hand, I did a desktop project in Qt and was shocked at how decently it generated and built an Xcode project. I did have to write a script to sign and package the app for distribution, but it wasn't as painful as plain-old iPhone development used to be simply because of the wildly unreliable certificate BS in Xcode.
That is how anti-IDE rants sound to me.
The problem with XCode isn't that it's an IDE. It's that it's a shitty one, and Apple keeps trying to shove it on people by coupling the dev experience with their own IDE.
The right thing for platform makers to do is provide tools and infrastructure which can be used in a modular fashion by an ecosystem of third party tools. Offering their own IDE is fine, too, but it should not be required. This is best for the long term health of the platform, and best for developers, too.
Being used alone, the usual talk of UNIX way, when in reality most CLI stuff has a bazillion of flags.
JetBrains gave up on selling AppCode, as the large majority of developers on the Apple ecosystem couldn't care less.
And he is correct. The problem is that Apple's development environment is such a pile of poorly-understood spaghetti and band-aided shit that I don't think anyone at Apple has the chops (or certainly the time) to clean it up.
"The right thing for platform makers to do is provide tools and infrastructure which can be used in a modular fashion by an ecosystem of third party tools."
That Package.swift file is the build configuration file. No Xcode required. It describes how to build your program, like a Makefile. Plus, the file is itself a Swift program, where the PackageDescription API serves as a kind of domain-specific language for declaratively describing your dependencies, etc: https://developer.apple.com/documentation/packagedescription
It's been fine, but not an order of magnitude better than using Xcode. Having copilot is nice, but the auto completion is somehow worse than Xcode from my experience so far. And the list goes on, some things are better in vscode, some in Xcode. I might eventually switch back to just using Xcode.
I do it all the time. It is useful test to see practical use of a language on actual project at hand. If a language is not productive quickly and need lot of background there maybe nothing wrong with language. But at same time nothing wrong with devs trying to see where it takes.