Hacker News Reader

Comment by watt | Hacker News Reader

I don't get why address of slice returned from "append" does not change. Maybe in a trivial program like this the backing array can always be extended in-place, because there in memory fragmentation.

Is that still true in an app that has considerable memory pressure and has GC running now and then?

View on HN

Even if slice is reallocated, the information about new reallocated slice is still stored in variable s. ps is merely pointing to that variable. The fact that the contents of the variable changed does not mean that its location has to change.

In other words, ps is a pointer to a pointer to array data. Append may change the inner pointer's value but that's about it.

Append returns a value, the new slice struct generated by append. Append always generates a new struct and returns it by value, because even if the array pointed to doesn't change, the length property of the slice changes. This value is then assigned to the local variable s, which didn't change its memory location.

>Maybe in a trivial program like this the backing array can always be extended in-place,

In this example the backing array didn't get extended in-place. The first backing array starts at 0xc0000be000. After the append() call, there's a new backing array, which starts at 0xc0000b8030.

Despite some syntax glosses in Go making it superficially sometimes look like a dynamic language, it is actually in the C heritage of what a variable is. A variable is not just a label pointing at an arbitrary value that can be moved to other arbitrary values like it is in most dynamic languages (like Python), it is a specific chunk of memory. (Or, more precisely, it is something that will be a specific chunk of memory if anything ever needs it to be, or the compiler decides it needs to be; recently things can in theory be register-only, but this is a detail the Go programmer operating at the Go level need not worry about.)

Specifically, what happens in

     s = append(s, "something")

is that append generates a new slice value (the tuple of backing array pointer, length, and capacity), which may or may not be pointing at a new backing array. These three values are then copied into the memory "s" was allocated with. So there is a new slice value allocated, but it is copied back to the original storage, and since there's no way to "get in between" those two things, the effect for a programmer at the Go level is that s is modified in-place.

In Go, allocation is very important and it is always explicit, except this explicitness is sometimes masked by the fact that the := operator is not a simple "allocate everything on the left using the types of the thing on the right" operator, but "allocate at least one thing on the left using the types on the right (it is a compile-time error if there isn't at least one new value) but use normal equality for everything already allocated", which is very convenient to use, but can make developing the proper mental model for how Go works trickier. Arguably, there's some sort of design mistake in Go here, though the correct solution doesn't immediately leap to mind. (It's easy to come up with more abstractly correct options but they have poor usability compared to the current operator.)

Similarly, it can be easy to miss that Go, like C, cares deeply about the size of structures, and every = statement has, at compile time, full awareness by the compiler of exactly how much memory is going to be involved in that equality statement. Interfaces may make it seem like maybe I can have an "io.Reader" value, and first I set it to some struct that implements it with a small amount of RAM, then maybe later I can set it to a struct that uses a large amount of RAM, but the interface value itself is actually a two-word structure with two pointers in it that is all you are ever changing, and, again, those two words are given a specific location in RAM (possibly virtually, if you never use it they could conceivably never been out of a register, but the Go compiler and runtime will transparently make it live in RAM if you ever need the address for any reason) and any setting of the value of the variable that has an interface value will set only those two values, with no other RAM changing as a result. You can use the same io.Reader variable through its interface implementation as a "handle" on a wide variety of differently-sized values under the hood, even in the same function (I do this all the time when progressively "decorating" an interface value within a function), but the in-memory size of the handle itself never changes no matter what value you ask it to handle.

This is not intended as criticism, praise, defense, attack, or anything else on Go itself; it is descriptive of what it is.

s changes but &s doesn't: https://play.golang.org/p/Xs1SYXqEl9i

Because there’s still space left in that slice (capacity > len), and the strategy of pre allocate capacity is to double the current (1,2,4,8,…), in this case 3 elements added => that slice was having a capacity of 4.

No, you can see that it was reallocated. At first the backing array started at 0xc0000be000. The append needed to do a reallocation and created a new backing array that starts at 0xc0000b8030.

Reply on news.ycombinator.com