Go Memory Management
povilasv.me
povilasv.me
You can always configure the CGO_ENABLED or -tags netgo flag in order to cause networking to not use libc either.
Not necessarily true, unless you use strategies like the one suggested in the top level post above this. https://www.osso.nl/blog/golang-statically-linked/
In practice, those dependencies are ubiquitous enough that the "DLL hell" half of the dynamic-linking problem is avoided. The other issues (system-dependent behavior quirks, estimation from static libraries, linker caching, etc. etc.) remain in many cases.
So, to answer the GP: yes, resident memory from the libc ends up being amortized between all the processes that use it. But it doesn't even matter much as there's a mere 5 MiB of resident memory involved here. The “hundreds of MiB” are only of virtual memory, and that shouldn't be of any concern (see my other comment in this thread).
On modern 64 bits architectures, you have plenty of it to spend. To give an idea of how much of it you have: since Linux 2.6.11 on x86_64, one can address up to 128 TiB (that's 128 thousands of GiB) of virtual memory per process. Starting with Linux 4.12, one can address up to 128 PiB (that's 128 millions of GiB) of virtual memory per process. That's completely unrelated with how much physical memory you have.
Much as housing developers are incentivized to get permits for a large block of land at a time, general purpose memory allocators are incentivized to ask the OS for permission to use large blocks of virtual memory at a time. Asking the OS for virtual memory is very slow, so to avoid doing it a lot, general purpose memory allocators ask for big chunks and manage those chunks themselves.
To quote from the official docs: https://golang.org/pkg/net/#hdr-Name_Resolution
> When cgo is available, the cgo-based resolver is used instead under a variety of conditions: .... when /etc/resolv.conf or /etc/nsswitch.conf specify the use of features that the Go resolver does not implement, and when the name being looked up ends in .local or is an mDNS name.
Almost any linux machine's `/etc/nsswitch.conf` will contain features that the go resolver doesn't implement. Because of that, almost any dns resolution will default to cgo.
In addition, selecting between netgo and cgo is a runtime feature by default (see GODEBUG=netdns=go|c), so it still has to build it in regardless unless it's explicitly disabled.
You can also determine this by using `ldd` on a go binary.
> Only programs that expliticly call C functions (i.e. those that use ‘import “C”’) do.
That is exactly what go uses in programs that use networking. Here's the 'import "C"' in question: https://github.com/golang/go/blob/fe8a0d12b14108cbe2408b417a...
Do you know of a short list of top features missing? I would kinda hope that modern and generally useful features would be in netgo. And old rotted code, security risks and anachronisms would be abandoned. But from your tone, I gather that this is more than a "people only use 10%, but never the same 10%" type of thing?
Notably, it can only handle the following sources: files, dns, myhostname, mdns* (but only a subset of those if you read the gory details).
It doesn't handle the fairly uncommon "mymachines" or "resolve" (sometimes used in the systemd world nowadays)
I probably spoke too pessimistically there. Many machines won't have an nsswitch.conf that requires using cgo for common domains. After re-reading the code, it does handle more than I thought (mainly mdns).
> I would kinda hope that modern and generally useful features would be in netgo. And old rotted code, security risks and anachronisms would be abandoned
The fundamental problem here is not that. The nss system libc uses for network resolution is a pluggable system, and it chose dynamically linked c modules as its plugins.
It's not that they're old rotted insecure bits of code, it's that the interface is inherently a c-centric one, and go early on decided it wished to do syscalls directly instead of call them through libc (as literally every other programming language does).
Finally, that means go can't ever reach full parity as long as it wishes to avoid c, since anyone can simply write a new nss plugin in c and list it in resolv.conf, and go has no way of having that same arbitrary behaviour in go.
> that means go can't ever reach full parity as long as it wishes to avoid c
Well, go has for better or worse decided on a "we'll bring our own batteries" approach - and that does imply a burden of re-inventing a lot of wheels. On the other hand, those wheels are all likely to be painted in just the right shade of gopher brown.
At a glance, nis/yp appear to be missing. Not sure if nis doesn't count as outdated though.