The upcoming iterator design for Go 1.23

I didnt had a single line of code broken by golang version up updates in 7+ years, scanning through golang source code is always a pleasure. No hidden magic. I dont read documention either - just surfing inside of source code by clicking with CMD key.

My main KPI for Golang is the ammount of clicks required to understand internals ... very few compare to other languages.

If its stamped by guys like Russ Cox (rsc), I can sleep well too.

But the author of the article does point at an important reason to do it like this:

> Allow for clean-up with defer

When the range syntax would be syntactic sugar for an iterator created as a local variable which handles the state somewhat like this:

    iter := Iterator{
        Next: func() bool {...},
        Current: func() T {...},
    }
    for iter.Next() {
        loopvar := iter.Current()
        ...
    }

The only gripe people may have is when it turns out to be much less efficient than a regular range loop.

tldr: The gold standard is if your iterator syntax / syntaxes can support these two types of iteration:

1. Interleaving two sequences

2. In-order tree traversal

I'd argue the primary magic in large-scale languages should mainly be in those two places: inversion of control and concurrency.

I don't mind the pace that Go has been moving at when it comes to new features. It still feels very conservative while still evolving.

I still grumble every time I have to implement one. Something about the interface just bothers me. Feels sloppy.

Look at Rust iterators which are just simple and readable ‘next() -> Option<Item>’ that can be targeted with ‘for item in iter’ and chained iterator expressions or C#’s yield return or IEnumerator<T> (which support cleanup on panic too) which works with foreach and the same kind of chaining. Both are faster, easier to read and write too.

Instead of pulling the same quote from Rob Pike over and over and over again, people should watch/read is recent talk about Go titled "What We Got Right, What We Got Wrong": https://commandcenter.blogspot.com/2024/01/what-we-got-right.... A few excerpts:

> Given the title of this talk, many people might expect I'm going to be analyzing good and bad things in the language. Of course I'll do some of that, but much more besides, for several reasons.

> But the real reason I'm going to talk about more than the language is that that's not what the whole project was about. Our original goal was not to create a new programming language, it was to create a better way to write software. We had issues with the languages we were using—everyone does, whatever the language—but the fundamental problems we had were not central to the features of those languages, but rather to the process that had been created for using them to build software at Google.

> The creation of a new language provided a new path to explore other ideas, but it was only an enabler, not the real point. If it didn't take 45 minutes to build the binary I was working on at the time, Go would not have happened, but those 45 minutes were not because the compiler was slow, because it wasn't, or because the language it was written in was bad, because it wasn't. The slowness arose from other factors.

> And those factors were what we wanted to address: The complexities of building modern server software: controlling dependencies, programming with large teams with changing personnel, ease of maintainability, efficient testing, effective use of multicore CPUs and networking, and so on.

> In short, Go is not just a programming language. Of course it is a programming language, that's its definition, but its purpose was to help provide a better way to develop high-quality software, at least compared to our environment 14 plus years ago.

> And that's still what it's about today. Go is a project to make building production software easier and more productive.

I will repeat it because I think it is very important:

> And that's still what it's about today. Go is a project to make building production software easier and more productive.

Another quote about Go being more than the language:

> A few weeks back, when starting to prepare this talk, I had a title but little else. To get me going, I asked people on Mastodon for input. A fair few responded, and I noticed a trend in the replies: people thought the things we got wrong were all in the language, but those we got right were in the larger story, the stuff around the language like gofmt and deployment and testing. I find that encouraging, actually. What we were trying to do seems to have had an effect.

And finally:

> Perhaps the most interesting consequence of these matters is that Go code looks and works the same regardless of who's writing it, is largely free of factions using different subsets of the language, and is guaranteed to continue to compile and run as time goes on. That may be a first for a major programming language.

> We definitely got that right.

So now that we know better what Go is about, we can try to see why the iterators are added. From https://github.com/golang/go/issues/61405#issuecomment-16388.... I'll quote some parts:

> Can you provide more motivation for range over functions?

> The most recent motivation is the addition of generics, which we expect will lead to custom containers such as ordered maps, and it would be good for those custom containers to work well with range loops.

> Another equally good motivation is to provide a better answer for the many functions in the standard library that collect a sequence of results and return the whole thing as a slice. If the results can be generated one at a time, then a representation that allows iterating over them scales better than returning an entire slice. We do not have a standard signature for functions that represent this iteration. Adding support for functions in range would both define a standard signature and provide a real benefit that would encourage its use.

> There are also functions we were reluctant to provide in slices form that probably deserve to be added in iterator form. For example, there should be a strings.Lines(text) that iterates over the lines in a text.

To me this provides enough justification for adding the iterators. But that's not the important part. The important part is listening to what people say about Go, not just "Go the language", but Go as a whole. And listening to what people said recently, not what they said a decade ago. This is how you genuinely engage with other people, I think.

Either give us magic or make it supremely readable. This accomplishes neither - it's a barely legible mess of questionable value.

To be clear, I'm not against features. But I am against anything that makes code this ugly by default.

But hey, maybe I’m just being a curmudgeon and it’ll be great.

As the article mentions, there was nothing stopping you from doing this sort of thing (and worse; channels as “iterators”, anyone?) in older versions of Go. But it would be silly to do, and you’d get (rightly) scolded for doing it.

Now I guess that kind of thing is encouraged. :)

I dont mean that as a statement against the author, I mean it as a statement to the language designers. people are gonna start writing code like this, or worse, and Go is gonna end up like many other unreadable languages. shame.

As someone who has seen the cost of C++ templates metaprogramming wizardry gone wild I understand part of this reaction. I reckon we would have a real talent shortage bigger by orders of magnitude if we required familiarity with Alexandrescus' Modern C++ Programming and Herb Sutter's Exceptional C++ to onboard new programmers in a project.

Yeah, definitelly there's such a thing in programming as "too clever".

But should we so radically into the opposite direction? Should we sacrifice any attempt at expressiveness into the altar of simplicity? Aren't we going too far into the opposite direction?