Hacker News Reader

Comment by qppo | Hacker News Reader

If you're writing async code you don't need to worry about pinning unless you're manually writing futures or designing an executor. But if you are curious, this chapter explains it pretty good:

https://rust-lang.github.io/async-book/04_pinning/01_chapter...

> Apparently, recursion adds a new level of complexity to async.

It really doesn't - it's just some users may be surprised by the lack of magic when it comes to Rust. Async is implemented with futures, futures are types, and you can't recursively define a type without boxing.

> Apparently, this boxing and pinning is what you get when you don't have a GC, and that when you do have a GC, you simply don't need to deal with it. So that was the final straw for me.

It's kind of weird to me that language developers would pass on Rust because they don't understand memory management. How is memory going to be managed in Dark? If it's GC'd, how is that implemented?

View on HN

>It's kind of weird to me that language developers would pass on Rust because they don't understand memory management. How is memory going to be managed in Dark? If it's GC'd, how is that implemented?

This was weird to me as well, but it looks like Dark is more like a RAD tool with a custom integrated scripting language. In that case I don't think you need a sophisticated GC.

I can understand the point that X Lang is easier to write a DSL parser/interpreter in than Y Lang for a bunch of reasons, including memory management (I've hit the recursion snags a few times writing parsers in Rust myself - I get it). So I don't want to denigrate their design decision.

Just the logic seemed out of nowhere to me, like this shouldn't be terribly surprising to a language developer, and it's framed like they're discovering things for the first time.

Its just delegated to the host platform I'm assuming. Also, I don't know anything about Dark, but it's a DSL, and not a general purpose language, and could very well be running interpreted right now, just mapping things back/forth to the host.

This is all hypothetical, I actually don't know, just my best guess.

> If it's GC'd, how is that implemented?

The language in which the Dark interpreter is built has a GC, and we just use that.

A GC is a huge undertaking, and we already have to build an editor, a language, a stdlib, and infrastructure.

Why do you need to build an editor too?

Side note, I've written a GC in rust (stop the world, compacting). It's not a cutting edge, concurrent generational GC but it's not that big of an undertaking. And it performs very well for small problem sets since memory is all slab allocated. Only a few hundred lines in Rust.

I do definitely see how writing a tree walking interpreter is non ideal in Rust if you don't have a ton of experience writing Rust. I have a tiny lisp in Rust compared to doing it in dynamic languages it is quite verbose and hard to grok.

The premise of Dark is that we can remove a huge amount of accidental complexity by having an integrated toolkit of language, editor, and infrastructure. This allows for three major steps forward:

- we're able to deploy (safely!) in 50ms rather than minutes because of it. [1]

- we completely host the infra and you don't need to think about it (including DBs and queues, which are nicely integrated into the language) [2]

- trace driven development, where we use real values from production and show you them in your editor [2]

    [1] https://blog.darklang.com/how-dark-deploys-code-in-50ms/
    [2] https://darklang.com/launch/demo-video

I get the value prop and totally believe in your approach, I'm just confused why you would opt to write an entire editor when the LSP can do whatever you want and integrates into numerous editors.

They can significantly reduce tooling complexity with their own web-based editor. It's also a much more graphical application than a traditional text editor. Take a look at some screenshots from their demo video: https://www.youtube.com/watch?v=orRn2kTtRXQ

Of course, the hazard is that you then have to deal with offering a value proposition that's still appealing after adjusting for your target userbase having to give up all the plugins and tweaks they configured into their editors.

https://blog.harterrt.com/coding_in_textboxes.html

I’m guessing Dark is interpreted and that its objects are managed by the host language’s GC?

Yes, that's right.

It's pretty easy to understand non-moving garbage collection for a language where everything is boxed and heap-allocated without having to think about the concerns that lead to pinning. Pretty sure that's the order in which I encountered the concepts.

> because they don't understand memory management. How is memory going to be managed in Dark? If it's GC'd, how is that implemented?

That's one more thing you can afford to not understand and still be productive.

Sometimes, until you can't, then you have a really bad time.

For instance:

- You shouldn't rely on finalizers to manage resources due to the lack of determinism. No compiler support to help you through that.

- Similarly, when the magic blob GC starts acting up, how do you solve that?

- What about memory usage being orders of magnitude higher in a GC'd system?

It's just different problems. To an extent the line where you have to care is much closer in Rust than Java, but in Java, when you cross the line, good luck. Rust forcing you to be expressive from the get-go pays dividends down the line, allowing you to be relatively more productive, later.

I guess it's like skiing vs. snowboarding. Skiing is easy to get going, but really difficult to get good at. Snowboarding is really hard to get started, but once you do, it's relatively much easier to be great at it.

If you don't do embedded programming (where you shouldn't do dynamic memory allocation anyway, being that with or without a GC) memory is often not a constraint at all. And in a lot of situation it cost you less to increase the RAM (if you can) than optimizing the code.

> when you cross the line, good luck

And if you never cross the line? You wasted thousands of dollars for developing something in Rust that you could have written in Python in 1/4 of the time and be fine.

To me the job of the programmer is not to reason about memory. The job of the programmer is to reason about algorithms, data structures, and thus a language that abstracts the memory management is better if you can afford it. It has a cost, sure like it has a cost using C instead of assembly.

One thing that gets lost in these discussions is, a GC only helps with memory. Rust's ownership system (and to some degree, RAII & friends in C++) let you manage arbitrary resources here. There's a lot more than just memory going on.

> memory is often not a constraint at all

This type of thought process is how we get Electron applications and when several of them are running at the same time they can bring your OS to a crawl by having consumed every bit of memory available for some simple GUIs. Memory absolutely always matters.

Also the people who I see saying memory doesn't matter are usually developers who are running on 32GB+ work stations. They forget about the user still running with 4GB or 8GB of memory.

As much as I hate electron I'm still waiting for an equivalent, low memory native GUI toolkit that has an equally low barrier to entry.

I think you misunderstood, if you run into memory pressure issues in Java you need to spend tons of time tweaking and optimizing your specific garbage collector by poking the black box and hoping.

Of course in certain circumstances where it really doesn't matter you get your choice of languages.

> To me the job of the programmer is not to reason about memory.

The job of a programmer is to describe what needs to happen to a set of inputs to yield a set of outputs, and there are many different ways to do so. Sometimes you do need to describe the relationships between data. Rust attempts to leverage your description of the transformation and relationships to make memory management something you don't have to worry about either.

There are limits, of course, and sometimes you need to help it out, however this amounts to improving or clarifying the description of your system. On the other hand resolving garbage collection issues often involves making surprising and seemingly irrelevant changes to your system that happen to influence via spooky action at a distance the behavior of an unrelated component of the system. And if you remove them it all falls apart again.

To borrow a turn of phrase, it has a cost, sure like it has a cost using C instead of assembly.

I've seen memory leaks because of optimization and other stupid excuses enough times to know this is not a valid statement. it doesn't matter GC or know. It's minimum to know how to manage your memory.

Reply on news.ycombinator.com