Hacker News Reader

Comment by TonyTrapp | Hacker News Reader

Not that it helps here, but Microsoft never considered the MSVCRT that ships with Windows to be public API. This is not the "Windows allocator", this is the (very) old MSVC runtime library's allocator. Of course that doesn't keep anyone from using this library because it's present on any Windows system, unlike the newer MSVC versions' runtime library. Using the allocator from a later MSVC's runtime library would provide much better results, as would writing a custom allocator on top of Windows' heap implementation.

MSVCRT basically just exists for backwards compatibility. It's impossible to improve this library at this point.

View on HN

There's also UCRT, which ships with the OS since Windows 10. The logic of this rant was a real head-scratcher. If you must blame one side, it's LLVM. Fragmentation of C runtimes is annoying but inescapable. Glibc for example isn't any better.

The UCRT has even been present since Windows 7, if users keep up with updates. Or if applications bundle the UCRT installer with their own.

> Or if applications bundle the UCRT installer with their own.

But the UCRT is proprietary, so there are often legal issues with doing so.

There is an official Universal CRT Redistributable.

And java ships it, and installs it into its own java bindir, not the global system dir.

So you have two competing copies, which leads to race conditions (different "global" locks!) and nice cashes, until you can analyze it with DrMemory or the kernel debugger. Nobody does that.

Could you elaborate why Glibc isn't any better?

I remember some funny problems with Glibc, like, 20 years ago, but it's been invisible to me (as a user) since then. You get a new Glibc, old binaries still work, it's fine.

I'm pretty sure I've run into binaries breaking on new versions of Glibc but maybe it's because the architecture or calling convention changed. I've never really gotten the sense that GNU cares much about binary compatibility (which makes sense, they argue that sharing binaries is mostly counter productive.)

Just like with Windows the challenges affect developers rather than users.

> You get a new Glibc, old binaries still work, it's fine.

Indeed, but when you need to build for an older glibc it's not so simple. This is a common use case, since e.g. AWS's environments are on glibc 2.26.

Ideally you'd like to build for all targets, including older systems, from a single, modern environment (this is trivial in Windows) -- and you can do some gymnastics to make that happen[1] -- but in practice it's easier to just manage different build environments for different targets. This is partly why building Linux wheels is so convoluted for Python[2].

Hardly a world-ending problem, but my point is simply that C runtimes are a pain everywhere.

1. https://stackoverflow.com/questions/2856438/how-can-i-link-t...

2. https://github.com/pypa/manylinux

> Ideally you'd like to build for all targets, including older systems, from a single, modern environment (this is trivial in Windows)

https://github.com/sjmulder/netwake does what you're talking about, but it does a lot of gymnastics to make it work, and it also needs to use MinGW rather than MSVC for that to be the case.

I was thinking specifically of how easy it is to build for Windows 2000 even today. Going back to 3.11 is pretty cool!

> Fragmentation of C runtimes is annoying but inescapable. Glibc for example isn't any better.

Glibc very much is better.

There cannot be more than a single version of a (tightly coupled) ld.so+libc.so pair in a given address space any more than there can be more than a single KERNEL32 version in a given address space, and given that some system services are exclusively accessible via dynamic linking (accelerated graphics, for one), this forces essentially everything on a conventional Linux desktop to use a single version of a single C runtime (whether Glibc or Musl). No need to guess whether you need CRTDLL or MSVCRT or MSVCR70 or MSVCR71 or ... or UCRT (plus which compiler-version-specific overlay?): you need a sufficiently recent libc.so.6.

I cannot say I like this design (in some respects I actually like the Windows one more), but it does force Glibc to have very strong back-compat mechanisms and guarantees; e.g. it includes a bug-compatible version of memcpy() for old binaries that depended on it working byte by byte. As far as I’m aware, this applies from the point GNU stopped pretending Linux did not exist and the “Linux libc” fork died, that is 1998 or thereabouts.

(There are a myriad reasons why old Linux binaries won’t run on a new machine, but Glibc isn’t one of them.)

This is not to say that Glibc is perfect. Actually building a backwards-compatible binary that references old symbol versions is a gigantic pain: you either have to use a whole old environment, apply symbol versioning hacks on top of a new one and pray the resulting chimera works, or patch antique Glibc sources for new compilers and build a cross toolchain. But if you already have an old binary, it is better.

Glibc is major source of "binary from X years ago doesn't work" in my experience, to the point that if I want to run stuff older than let's say 5 years I start by getting a docker container - or a specially taylored set of libs starting with dynamic linker itself. Somehow proprietary OpenGL drivers still worked, it was glibc related files that caused crashes.

Thanks to symbol versioning I never know if a binary will work unless I can dismiss it early by versioning causing it to not link at all.

And given that glibc backward compatibility for all practical purposes is shorter than 10 years, having to support centos can get you a lot of those issues

> Given that glibc backward compatibility for all practical purposes is shorter than 10 years

As far as I’m able to tell[1], Glibc has never removed a versioned symbol as long as symbol versioning has existed, so the intended backwards compatibility is “forever”. No binary built against a(n unpatched) Glibc should fail to load on a later Glibc due to an absent Glibc symbol version. (Of course, if you play non-version-aware dynamic linker and dlsym() not dlvsym() things in libc, you lose; if you bundle an “updated” part of libc with the application[2], you lose; etc., so this may be true but still not entirely foolproof. And given how many things there are in libc, if you’re going to lose it’s probably going to be there.)

[1] E.g. https://sourceware.org/git/?p=glibc.git;a=history;f=stdlib/V...

[2] https://news.ycombinator.com/item?id=29479903

For example: https://github.com/microsoft/vscode-remote-release/issues/10...

So a vscode thingie uploads a newer binary to an older host, tries to run it there, and fails? Because the people who built said binary did not care to make it backwards compatible (or better yet, statically linked)?

... Duh?

(I’m not an expert in VSCode DRM, to put it mildly, so I might be misinterpreting this discussion, but that’s what it looks like to me. Also, isn’t it referencing GLIBCXX i.e. libstdc++, which not even a part of Glibc?)

> referencing GLIBCXX i.e. libstdc++, which not even a part of Glibc

You're right, so not an example of a glibc failure, but rather another standard library. Thanks!

This is the opposite problem of what's being talked about. Binaries built against old versions of glibc should run just fine against newer versions. This is about a binary built against a newer version of glibc that doesn't run on an older one. This is common, and super annoying. There are ways to build to eliminate this problem, but they all feel like hacks, or involve a lot of extra work.

(Hacks such as https://github.com/wheybags/glibc_version_header -- which apparently does work very well, but still feels like an annoying hoop that should be unnecessary to jump through. I wish glibc's shipped headers actually could support this out of the box so you could set a preprocessor define like `-DGLIBC_TARGET_ABI=2.12` and it would just work.)

Glibc 2.34 broke FreePascal. It calls some unversioned symbol for initialization that was removed

> Actually building a backwards-compatible binary that references old symbol versions is a gigantic pain: you either have to use a whole old environment, apply symbol versioning hacks on top of a new one and pray the resulting chimera works, or patch antique Glibc sources for new compilers and build a cross toolchain.

This is the example I gave downthread. It's an everyday problem for a lot of developers targeting Linux, whereas targeting decades-old Windows versions is a breeze.

Glibc isn't better, it just has different problems.

Targeting old GNU/Linux is a breeze too. Containers make compiling stuff for old distros super easy. I even use them to crosscompile stuff for Windows, macOS, Android etc. too, since I got tired of having to set up development environments on new machines.

> There cannot be more than a single version of a (tightly coupled) ld.so+libc.so pair in a given address space

That's an odd restriction, and it is related to the fact that all symbols live in one global namespace in a process. It's annoying if you are trying to build something like a plugin system, or if you are using a dynamic language which by definition loads all libraries dynamically. This is also the reason that you cannot mix Glib (GTK+) versions in a process.

I think you should be able to dlopen some library, or heck just load some machine code, and be able to run it, just take care to only pass POD over the boundary and never `free` stuff you didn't `malloc`.

what has LLVM to do with software development on windows?

because developing for windows is cancer that people fall back to LLVM and others

it's all on microsoft for not cleaning their mess

even apple offers a better story than the platform for "developers, developers, developers, developers"

a shame, a well deserved shame, a hall of shame

Eh, no, this is strictly a Windows problem. On Windows every DLL can be statically linked each with its very private copy of the MSVCRT, which means -for example- that you'd better never ever pass one DLL's malloc()'ed memory pointers to another DLL's free().

On Unix systems (and Linux) this sort of thing can only ever happen if you have a statically-linked application that is linked with libdl and then it dlopen()s some ELF -- then you need the application's C library to be truly heroic. (Solaris stopped supporting that insanity in Solaris 10, though glibc apparently still supports it, I think?)

AIX, which happens to be a UNIX, has similar dynamic linking model.

Then we have the surviving mainframes and embedded RTOS, which aren't UNIXes either.

It's effectively mandatory. Microsoft provides about twelve different C Runtimes. But if you're building something like an open source library, you can't link two different C runtimes where you might accidentally malloc() memory with one and then free() with the other. If you want to be able to pass pointers around your dynamic link libraries, you have to link the one C runtime everyone else uses, which is MSVCRT. Also worth mentioning that on Windows 10 last time I checked ADVAPI32 links MSVCRT. So it's pretty much impossible to not link.

It isn't mandatory. I have never actively linked against MSVCRT on Windows. From my experience it's mostly software that isn't built with Visual Studio that uses MSVCRT, or software that that takes extreme care of its binary size (e.g. 64k intros). MSVCRT is not even an up-to-date C runtime library. You wouldn't be able to use it for writing software requiring C11 library features without implementing them somewhere on top of it.

It's true that you cannot just happily pass pointers around and expect someone else to be able to safely delete your pointer - but that is why any serious library with a C interface provides its own function to free objects you obtained from the library. Saying that this is impossible without MSVCRT implies that every software needs to be built with it, which is not even remotely the case. If I wanted, I could build all the C libraries I use with LLVM and still link against them in my application compiled with the latest MSVC runtime or UCRT.

The much bigger problem is mixing C++ runtimes in the same piece of software, there you effectively must guarantee that each library uses the same runtime, or chaos ensues.

If you're writing in C++ on Windows, expose only COM (or at least COM-style) interface called through virtual functions on an object pointer. Then you can use whatever C++ run-time you want, internally. What you don't want is the other library calling C++ functions by name. Like you pass it some ostream object and it calls ostream::put or whatever, where that symbolically resolves to the wrong one.

It's also why all, or nearly all, new API on windows are done through COM.

It also means that C++ or other runtimes don't pollute your ABI and make it annoyingly hard to access features from any random code.

Another reasonable choice for some components would be to have a purely C api; everything extern "C", and only PODs (plain old datastructures) in the arguments.

Exactly. Using a pure C interface escapes COM's requirement to register every component (regsvr32) and its overblown "GUID for everything" model, and its baroque way of creating components (somewhat alleviated by various macros and templates, but still). Making an object oriented interface is slightly cumbersome, but you can do it by sending back and forth a cookie (void* or int) of your object as the first parameter.

This will also make your interface accessible to other languages, if the need arises, since the C ABI is a de-facto stable ABI on all platforms (disregarding library objects and issues).

Another alternative if you want to stick with C++: make up your own lightweight COM! I've done this successfully in my own code at work, it works great, and has been working for 8 years now.

This method allows people to write C++ components against my app without using COM and without having the exact same compiler versions and having symbol problems. It may seem like a lot of work but it really isn't.

1) Expose a pure virtual interface IFoo, that doesn't include any implementation. Once it is released, never touch it, only create new versions which inherit from it, IFoo2, IFoo3 etc.

2) Expose an extern "C" CreateFoo() that will return FooImpl : IFoo. Also a matching DeleteFoo(IFoo).

3) For all structs / classes you need to receive/send from IFoo methods, create another interface. This includes all C++ classes, including std::string, hide it behind IMyString or something. This is a minor inconvenience but it sidesteps all ABI incompatibilities.

4) Have all interfaces inherit from some IBaseXX which has a GetVersion() method A component which uses this component could call this, and if it returns e.g. 2, then it can safely cast IFoo to IFoo2* and use IFoo2 methods. Else it can return an error message or use something from IFoo*

This relies on the fact that C++ vtable layout is essentially an ABI that will never change, at least under Windows, since the whole of COM relies on this and MS will never change it. Anything other than vtable layout in an object is subject to change, so the trick is to only have pure virtual interfaces.

I have no idea if this trick will also work on Linux, I don't know how stable GCC / Clang's vtable layout is from version to version, but I suspect it will.

This was taken from a CodeProject article I read a few years back, but I can't find it anymore... the closest I can find is [0] (DynObj - C++ Cross Platform Plugin Objects), but it is more complicated than what I read back then. I didn't use it, I wrote my own as I outlined above. Like I said, it isn't really that complicated.

[0] https://www.codeproject.com/Articles/20648/DynObj-C-Cross-Pl...

Not kept up with the times?

Registration free COM exists for several years now.

I was aware of registration free COM, and I did consider it, in fact I tried it out several times during the last decade.

However in my experience it was always hard to get working properly, and it was always poorly documented. I'm talking primarily about the XML assembly manifest, which is very easy to get wrong.

In fact I remember vaguely the only complete documentation & example I could find at the time was an article in MSDN magazine, but now I can't find it, only bits and pieces of information scattered around, nothing in MSDN except a vague article. Most references I can find are for consuming dotnet COM objects and I also need to consume components written in C++. So the situation has gotten worse, documentation-wise.

Another couple of points:

1) Personally I don't want to use COM at all, I think it's too complex. I think it's really a brilliant idea but wrapped with over-engineered, idiosyncratic idioms and API. I tried to distill it to the minimum required by me for my "COM lite" above.

2) I'm not the one creating those components, I'm in charge of the consuming application. The people creating those components (in house) are essentially my support people, they're not really professional developers, and get confused with anything slightly more complex than plain C++ code. None of them has ever learned anything about COM, I'm the only one who knows all about it. Meaning I have to support them when things go wrong with COM registration or when they compile their code and it doesn't work. So I'm on a mission to get rid of all COM dependencies in my application, and replace them either with plain DLLs with C API, or (in one specific case) with my "COM-Lite" I outlined above.

Wasn't there also a hacky way to achieve the same goal as registration-free COM even before it existed?

Probably, although I tend to stay away of such solutions, as they eventually turn into headaches to sort out in some machine, where the hack doesn't work as expected.

For in-process DLL-based servers, you could try to implement CoCreateClass (IIRC) manually and bypass GUID-to-server lookup and go directly for loading a DLL and calling the method for getting the object from it.

> Using a pure C interface escapes COM's requirement to register every component (regsvr32) [...] Another alternative if you want to stick with C++: make up your own lightweight COM

Or you can combine both: use a pure C interface which returns COM objects. That way, you can keep using COM without having to register anything with regsvr32 or similar.

> I have no idea if this trick will also work on Linux, I don't know how stable GCC / Clang's vtable layout is from version to version

Recent (as in, since around the turn of the millennium) GCC and clang on Linux use a standard vtable layout, defined by the "Itanium ABI" (originally created for Intel's IA-64, but like UEFI and GPT, became a standard across all architectures), so it's also very stable.

Ah! Yeah, I've completely forgotten that the Itanium ABI also guarantees a standard vtable layout. Thanks!

Were starting to convert my app from a Windows-only app to a cross platform one, so this makes me happy that my "COM lite" scheme would also be stable on linux.

As for returning pure COM objects through a C interface: Yeah, I've considered that, but like I wrote in my comment below to pjmlp, I don't like COM personally, and I want to simplify writing the components' code as much as possible, since the team which writes them isn't a team of software developers but rather in a supporting role. They do know basic C++/C# but don't know COM.

Also getting rid of COM would be the first thing done for cross platform support, anyway.

Due to experience trying to link some code, especially C++, I'd probably rather start by implementing a simplified COM system first rather than dealing with extern C

Itanium ABI is somewhat stable. As in there exists a chance that two binary files compiled with exact same set of libraries and headers will be compatible, which is something that will get broken the moment two files were compiled with different switches in GCC that do not technically change things at Itanium ABI levels, but cause incompatibilities in various standard types.

Our programs ship their DLL dependencies in their own installer anyway, like most others on Windows. Just ship your FOSS library with a CMake configuration and let the users build it with whatever runtime they want.

C applications targeting Windows must provide their own C library with malloc and free (if they are using the "hosted implementation" features of C).

MSVCRT.DLL isn't the library "everyone" uses; just Microsoft programs, and some misguided freeware built with MinGW.

Even if ADVAPI32.DLL uses MSVCRT.DLL, it's not going to mistakenly call the malloc that you provide in your application; Windows DLL's don't even have that sort of global symbol resolution power.

I would be very surprised if any public API in ADVAPI32 returns a pointer that the application is required to directly free, or accept a pointer that the application must malloc. If that were the case, you'd have to attach to MSVCRT.DLL with LoadLibrary, look up those functions with GetProcAddress and call them that way.

Windows has non-malloc allocators for sharing memory that way among DLL's: the "Heap API" in KERNEL32. One component can HeapAlloc something which another can HeapFree: they have to agree on the same heap handle, though. You can use GetProcessHeap to get the default heap for the process.

It may be that the MSVCRT.DLL malloc uses this; or else it's based on VirtualAlloc directly.

> MSVCRT.DLL isn't the library "everyone" uses; just Microsoft programs, and some misguided freeware built with MinGW.

There's a third set of users: programs built with old enough versions of the Microsoft compiler. Before Microsoft decided that every single version of its C compiler should use a different C runtime (and much later they changed their mind again), all Microsoft C and C++ compilers linked their output with MSVCRT.DLL. In fact, that's probably the reason MinGW chose to use MSVCRT.DLL: to increase compatibility with the Microsoft compilers, by using the same C runtime.

> Before Microsoft decided that every single version of its C compiler should use a different C runtime (and much later they changed their mind again).

It is... not as simple as that[1]. MSVCRT itself is the fifth version of a Win32 C runtime, after CRTDLL (used to ship with the OS but no longer), MSVCRT10, MSVCRT20, and MSVCRT40. It’s just that both the toolchains linking with MSVCRT and the OSes shipping it (and nothing newer) endured for a very long time while Microsoft went on a collective .NET piligrimage.

Of course, “NT OS/2” started the same year ANSI C was ratified (and Win32 only a couple of years later), so some degree of flailing around was inevitable. But the MSVCRT Garden of Eden story is not true.

[1] http://www.malsmith.net/blog/visual-c-visual-history/

MinGW chose MSVCRT.DLL because it meets the definition of "System Library" referred to in the GNU Public License. There is a special exception that GPLed programs can be linked to a proprietary, closed-source component if it is a system library; i.e. something always installed and always present on a certain type of system to which that program is ported. Without that, you couldn't have GNU utilities as replacements for Unix utilities on a proprietary Unix, linking with its libc.

>but Microsoft never considered the MSVCRT that ships with Windows to be public API

It was in the past. At first msvcrt.dll was the runtime library used up to Visual C++ 6. Later, VC++ moved to their own separate dlls, but you could still link with system msvcrt.dll using corresponding DDK/WDK up to Windows 7.

I'm also not sure that this is just ancient library left for compatibility, some system components still link to it, and msvcrt.dll itself seems to link with UCRT libraries.

> It was in the past. At first msvcrt.dll was the runtime library used up to Visual C++ 6.

At that time it was already a big mess, because at first it was the runtime library of Visual C++ 4 in fact! The gory details are here: https://devblogs.microsoft.com/oldnewthing/20140411-00/?p=12...

> some system components still link to it

Some system components themselves are very much ancient and unmaintained and only exist for backwards compatibility as well.

Ancient or not, I don't think it really matters for allocation performance: malloc in both msvcrt.dll and ucrtbase.dll after some indirection ends up calling RtlAllocateHeap in ntdll.dll

It does, because modern allocators do a lot of house keeping on top of the system heaps, e.g. to quickly reuse identically-sized memory chunks that were just freed.

For posterity, here's a link to a version of that page with the original comments section intact:

https://web.archive.org/web/20160410051513/https://blogs.msd...

I don’t think msvcrt is exposed to link against in the DDK anymore. I maintain this, with the caveat that you really need to know what you’re doing: https://github.com/neosmart/msvcrt.lib

Win32 has an allocator (HeapAlloc), and it is similarly slow and low-concurrent. Even if you enable the newer stuff like LFH.

Reply on news.ycombinator.com