A New AWS SDK for Rust | Hacker News Reader

A New AWS SDK for Rust

aws.amazon.com

219 pointscarllerche61 comments

As good as rusoto is, an official SDK is a huge step forward. Many were wondering how AWS could be "commited to Rust" but not provide an official SDK, this looks great!

> We are exploring ways to support multiple Rust async runtimes.

Looking to be runtime agnostic from the start is a good sign, although it can be challenging in async rust's current state. It looks like they aren't tied into much async IO, so it should be relatively easy for this library.

> As good as rusoto is, an official SDK is a huge step forward.

Yes. It might not be "a huge step forward" in tech terms, but it would be definitely very positive for wider adoption, especially from "enterprises" where all is based on "standards".

TBH it will be a huge step forward in tech terms - Rusoto is maintenance mode, and it's missing functionality. I had to jump into it just the other day to add S3 Select support.

It will be (hopefully), but it isn't yet. This crate doesn't even suppory S3 at all so far.

For sure :) I just have expectations.

Shit man, as long as they do a better job than they did with Amplify.

Not holding my breath there...

Ugh. To anyone contemplating using Amplify I would say:

Rolling your own UI components and straight up using the amazon-cognito-identity-js library and wrapping the bits you need in promises, is far far far less of a headache than using Amplify, unless being tied to Webpack and dealing with disgracefully lazy types is your idea of fun.

That's what we're doing actually. I get the impression that Amplify was designed more for vendor-lock-in than actually solving the problem in the best way

> Just like our newer SDKs, AWS used the Smithy toolchain and service models to build AWS SDK for Rust

This is pretty impressive. I haven't looked at the SDK in detail, but managing to produce ergonomic APIs (semi-)automatically based on some service models for language as complex as Rust seems like a major accomplishment. I have most experience with the Python library (boto3) which is bit weird at places, but the Pythons dynamic nature is pretty good at hiding lot of that weirdness. Interesting to see how that works out for Rust where so much is static and explicit instead.

This also means that the possibility for multiple runtimes might not be that far fetched idea, if they can just make a code generator that spits out different flavors of the SDK.

The Java SDK is not especially ergonomic and the docs are exactly what you’d expect from auto generation.

This universal for all of their SDKs. They seem to get community examples, too, which often contain some of the worst code you'll find.

But the consistency is worth it. If you know how to work with boto you can use the java, JS, go, etc sdks easily.

> managing to produce ergonomic APIs (semi-)automatically [...] I have most experience with the Python library (boto3)

This almost sounds tongue-in-cheek: the slightest use of boto3 makes it seem obviously auto-generated.

It's fine, it even has advantages (whole classes of bugs removed, or at least fixed once fixed everywhere), I just wouldn't call it ergonomic or idiomatic.

's3 = boto3.client("s3")'.

I think there is a trend for a set of tools which auto-generate stuff. It started back with web tools generating websites from WYSIWYG, now it is getting everywhere.

We build services and models using https://www.protoforce.io, which also auto-generates client and server side. Generally, it is impossible to generate good code using templates only, so it has to be transpiled.

I think from amazon's side it was coming sooner or later anyway, with the amount of APIs they have - it is inevitable. I just can't see how many teams you need to have to manage all of the client side code for many languages.

How often do the domains of systems programming and cloud administration intersect?

I'm assuming that rust programs are primarily used in systems programming and that this SDK would be used for cloud administration but I could be wrong.

I see an obvious match with python and boto3 but not this.

There's a lot or growing number of people who don't use Rust for "systems programming" per se.

People are writing web services or data products that would need to interact with cloud services. Object stores, SQL interaction, lambda/serverless, etc.

I'd think that people using this SDK for cloud administration would be in the minority, but it's just a hunch.

I'll give it a go, as long as the documentation is good.

I worry about that.

> How often do the domains of systems programming and cloud administration intersect?

I can see Rust being attractive for high-traffic network apps, based on the idea of requests-per-second not being amortized over GC pauses.

Anyone in this space would already have been using apache or nginx, which are in C or C++. How mature are rust's HTTP implementations?

https://linkerd.io/

Seems relevant.

Used by real companies at scale. The primary maintainer of the largest one even works at Amazon, though I don’t know if they use it in a network-facing way there. You can also do what folks do in other languages too, and put nginx out in front, if you prefer.

> I'm assuming that rust programs are primarily used in systems programming and that this SDK would be used for cloud administration but I could be wrong.

I often see Rust used for microservices (where Go is often used), for which access to AWS is very nice to have.

Having a mostly statically linked executable handle this rather than some bastard child of virtual environments and docker images is a dream come true.

Shoutout to the Tools teams which develops cross, it's really amazing, did a project ago and had some problems to compiling static with rustls (OpenSSL really bugs with me, who knows why) do I tried Cross and felt in love, just works

If you have problems using musl with some libs, give it a go, you gonna love it ;P

https://github.com/rust-embedded/cross

Edit: my dumbass english failed me, sorry.

In a world where you rent computers by CPU time and RAM usage, enterprises that grow large enough find significant cost savings by implementing high-volume services in more resource-efficient ways.

Users in this space also appreciate Rust's reliability properties. They say things like "It's never my Rust services that set off my pager, and that makes me happy."

Systems programming is just the recommended domain.

Personally I think having a strong type system with an IDE that instantly tells you if the data is the right shape is very valuable when interacting with network interfaces.

In Python you either send the request at runtime and see what the server responds with, or you read the docs, both of which are at least 10x slower than an IDE telling you nearly instantly.

Rust's type system can represent null and polymorphic types quite well, both of which are quite common in JSON-like network interfaces.

Macros work well for code generation that is often used to convert a network api spec into code.

Of course you can do types and macros in Python, but neither are primary features of the language so are not as well supported.

Boto typing stubs are changing this. Prior to that I'd of agreed 100%.

I know quite a few cloud administrators that are systems programmers. They love to work in languages that they can lean on their compiler and not runtime to find a lot of mistakes.

Our product is built on AWS and is primarily written in Rust. If you're renting compute you'd better make the most of it, cause that's just money down the drain.

This is huge for Rust. With Microsoft Windows SDK and now with this, Rust is starting to get first party support on platforms.

More so than language syntax, the driver of adoption of a language is ecosystem. Rust is rapidly building up a very impressive ecosystem around it.

Now they need an official Jetbrains IDE and we'll know Rust has made it.

It's basically already here (in the form of a plugin): https://www.jetbrains.com/rust/ I've been enjoying it quite a bit.

I believe Jetbrains have stated that there would need to be more value to create a whole new product.

Using the plugin with CLion feels very well integrated, and I’m not sure what else we’d get out of a seperate IDE other than more distinct branding.

Honestly though, VSCode has been pretty good in this respect. Although I wouldn't use it to write Java or Kotlin, but for C/C++/Zig/D/Rust it's been good.

<raises hand> writing Scala here in VScode.

Some things are better, some worse. Highlights for unused code or imports is imho worse in vscode/metals and refactoring isn't as easy. But running tests for me has always been much faster. Not sure if you can use bloop with intelij but incremental compilation is much better in vscode.

I've had mixed success with VSCode and JVM related stuff. Although the projects I was working on were using some really obscure Maven stuff and god forbid gradle.

CLion works very well for Rust. It's much better than Android Studio for Flutter/Dart. I use both every day.

I also use CLion and it's very nice. I feel like IDE support is about 80% of what I'd like to see. Some rough edges, especially with the debugger.

But competitive with other languages like Go or Java.

Rusoto was an amazing effort. I was pretty sad when it was semi-sunset, but then I realized how all of this was entirely voluntary work.

There's been a massive amount of new AWS features recently and this made the inofficial SDKs lag behind. Looking forward to having an officially maintained SDK. Hope the documentation will be better than the generated ones in Rusoto.

Rusoto is awesome, but looked to have an uncertain future. I hope the new SDK can reach the same level of quality and breadth fairly quickly.

Also, rusoto is a great name and I hope we get to keep it.

Off hand question for Rust: if memory is borrowed, I gather Rust does not need --- perhaps resists --- C++ style allocators? Here I give emphasis to owner of the (heap) data rather than efficiency. Both languages might use an allocator for fast re-use, less fragmentation, MT support etc.. On the the hand STL strings and containers are replete with allocators in their constructors leading to such questions like if two non-empty std::vector<T> A,B vectors are equal (same size, specialized both on T, resp. elements equal) is A==B true if A has a different allocator from B?

Now granted allocator aware containers is a STL thing not per se a C++ thing. But the STL should leverage C++ design to the good ... so it's not too wrong or off base to think about the language in terms of a primary exemplar library.

This is wonderful to see and a great outcome from the effort put into Rust and Rusoto.

When is it actually desirable to imperatively program your cloud provider rather than using something like Terraform?

Obviously there are services meant to be consumed from application code, but the infrastructure provisioning stuff?

Maybe this makes it easier to develop things like Terraform using Rust?

> When is it actually desirable to imperatively program your cloud provider rather than using something like Terraform?

One place I've done is when your infra needs to dynamically scale up/down based on load.

TF supports this just fine

Even if I wanted to use real programming language for that, Pulumi seems to be better choice.

It's not, trust me. Use the terraform CDK instead.

I have built several services that do dynamic infrastructure provisioning in response to API calls as part of the core functionality. This isn’t unusual.

When implementing a Terraform provider, for one. At one point I had the scaffolding up to build providers in Rust (contributing to Tonic and Rusoto along the way) - many of the bugs in TF providers would be resolved by generics and type safety.

Still waiting on GCP to announce theirs

The Rust community opened an issue on their tracker [0] - it seems that they are not working on one right now:

> Greetings folks! I work on the team that builds the Cloud Client Libraries. Per the last comment - I can absolutely share that we're not actively working on a Rust SDK right now, and we have no immediate plans to start. We completely understand this would be useful, and we'd love to go build it. However, we're focusing all of our energies on Java, Go, Node.js, Python, C#, Ruby, PHP, and C++ at the moment. If you're looking to build something, I'd be happy to bounce ideas with you!

For now, we have https://github.com/Byron/google-apis-rs.

[0]: https://issuetracker.google.com/issues/124538614

In turn with this release I learned that S3 isn't supported from the beginning because it's the only AWS service using an XML protocol.

Made me chuckle a bit.

There's a handful of services that use XML protocols, but certainly not a lot.

That's an odd set of initial services.

I was thinking that it might be just some newer services, but then I saw DynamoDB which has been around for quite a while.

I think, they started with the services that share a protocol.

Does SDK for rust provide any advantages or this is more for people who develop in RUST and can use single language across their environment ?

Does this still run against the ts/js runtime like the other sdks?

I believe you are thinking of the CDKs. The SDKs are all fully language native.

you are right, my brain must have developed fixation on cdk due to work

Reply on news.ycombinator.com