Hacker News Reader

Comment by JonathonW | Hacker News Reader

MSYS and MSYS2 actually are Cygwin-- the original MSYS being a (horribly out of date) fork of Cygwin that never really pulled much from upstream, and MSYS2 attempting to track upstream Cygwin more closely.

You're getting confused with MinGW, which uses MSYS to build native Windows executables. They need MSYS (as a Cygwin-derived emulation layer) because tools like GCC or Bash expect the system to support POSIX APIs and have POSIX semantics-- for example, Windows has no equivalent to a POSIX fork() call. The code you're compiling under MinGW has no MSYS or Cygwin dependencies, but the compiler and tools themselves (gcc, bash, the linker, etc.) do.

View on HN

>MSYS and MSYS2 actually are Cygwin

Not the person you're replying to, but interesting ...

>tools like GCC or Bash expect the system to support POSIX APIs and have POSIX semantics-- for example, Windows has no equivalent to a POSIX fork() call.

So do Cygwin and/or MSYS emulate the fork() call on Windows? and if so, do you have any idea how that is done? Just interested, since I have a Unix background - not at deep OS level, but at app level and also at the level of the interface between apps and the OS (using system calls, etc.).

> So do Cygwin and/or MSYS emulate the fork() call on Windows? and if so, do you have any idea how that is done?

Cygwin does some pretty horrific hacks to emulate it. It basically creates a paused child running the same binary, fills in its memory, stores the register context of where it came from in a shared memory, and then resumes the child. The child on startup detects that it was forked, and then looks into shared memory to resume running at the place of the fork.

edit: It's even worse than I remembered: https://www.cygwin.com/faq.html#faq.api.fork

Wow. That sure is complex.

> So do Cygwin and/or MSYS emulate the fork()

Yes. That's one thing we spent considerable engineering effort on in this first version of the Windows Subsystem for Linux: We implement fork in the Windows kernel, along with the other POSIX and Linux syscalls.

This allows us to build a very efficient fork() and expose it to the GNU/Ubuntu user-mode apps via the fork(syscall).

We'll be publishing more details on this very soon.

Interesting! thanks. The original Unix fork() was found to be somewhat expensive in resources (a little surprising since it was the only way to create a child process), later there was vfork() (copy-on-write) (maybe innovated by BSD), and I read Linux's clone() does even better, though not looked into it in detail.

fork would copy the entire address space of the process, which was wasteful when you're usually going to just throw all of that memory away by calling exec. BSD added vfork to optimize the `if (fork() == 0) exec(...)` scenario, by not copying the memory, and just pausing and then borrowing memory from the parent, until exec is called. Modern operating systems use copy on write pages for fork, so instead of copying all of memory, you just need to copy the page tables.

> then borrowing memory from the parent, until exec is called

What does that mean?

It means that if the child process actually modifies the memory then those modifications will be visible in the parent process, because they're both using the same address space. It's essentially an awful hack that was added to BSD at a time when it didn't yet use copy-on-write for fork() to achieve the same performance with vfork()+exec() that you would get from a CreateProcess()-like API.

That's why the child is allowed to do almost nothing:

       the behavior is undefined if the process created by vfork()
       either modifies any data other than a variable of type  pid_t  used  to
       store  the  return  value from vfork(), or returns from the function in
       which vfork() was called, or calls any other function  before  success‐
       fully calling _exit(2) or one of the exec(3) family of functions.

Thanks.

In Linux, fork(), vfork(), and clone() all use the same underlying machinery, with just a few different flags. clone() is the most general, with flags for what to share; those flags in particular include all the Linux namespaces that serve as the basis for containers. fork() just uses clone() with a hardcoded set of flags, and vfork() does the same as fork() except that it doesn't schedule the parent process until the child calls exec.

Interesting, will have to check that out.

>We implement fork in the Windows kernel

So will this be only on Windows 10?

Yes-- the Linux stuff is all coming in the Windows 10 Anniversary update this summer.

How does this new fork differ from the already existing NtCreateProcess with a NULL section handle that was used to implement fork in the old SUA/POSIX subsystem?

MSYS (as part of MinGW project) doesn't use Cygwin AFAIK; with the licensing implications that it has, because Windows programs written with Cygwin run on top of a copylefted compatibility DLL that must be distributed with the program, along with the program's source code (quoting Wikipedia).

In fact the binaries, that are compiled with MinGW, link with MSVCRT by default (Microsoft Visual C Run-Time DLL). So no compatibility layer, and they don't rely on Cygwin.

Please distinguish carefully between MSYS and MinGW. MSYS (or MSYS2) programs runs on top of a copylefted compatibility DLL that is suspiciously similar to Cygwin. They must be GPL-compatible. MinGW programs link with MSVCRT. They are compiled by GCC toolchain programs, most of which are MSYS programs.

Reply on news.ycombinator.com