I first learned about this while trying to understand processes being killed by OOM in production. We had python 2.x batch jobs being executed by long-running python worker processes -- some of the arbitrary application code in some of the arbitrary batch jobs would occasionally want to execute some command line tool in a new process, and to create the new process under the hood python's subprocess library would fork/exec, and if the parent worker process had already accumulated a large virtual memory footprint, linux's approximate memory accounting heuristics would kick in during the attempted "fork", decide that we were obviously going to run out of physical memory, and kill the process.

We didn't actually want to fork anything and share gigabytes of virtual memory with the child process, we wanted to spawn an almost entirely independent process to do something and report results, but that got implemented under the hood by fork.

Spawning processes is one area where Windows is more elegant than linux: windows offers spawn. Apparently macos and solaris implement a posix_spawn that avoid the complications of fork/exec.

linux offers posix_spawn, apparently which may may or may not call fork under the hood depending on which libc you're using. If libc implements posix_spawn by calling fork then you're back in the same mess with linux heuristic memory accounting and overcommit. E.g. old versions of glibc will fork when you posix_spawn, newer versions of glibc may vfork . musl apparently will always vfork.

It looks like cpython's subprocess.Popen was patched in python 3.8 to detect some cases where posix_spawn can be used -- it reads as if it will only kick in on linux if it detects a sufficiently new version of glibc: https://github.com/python/cpython/blob/main/Lib/subprocess.p...

edit: Python 3.10 now supports using vfork for linux inside subprocess: https://bugs.python.org/issue35823

  docker run --rm -it --entrypoint=/bin/sh python:3.9-alpine
  # apk add strace
  # strace python -c "import subprocess; subprocess.run(['ls', '-l'])" 2>&1 >/dev/null | grep fork
  fork()                                  = 88

  docker run --rm -it --entrypoint=/bin/sh python:3.10-alpine
  # apk add strace
  # strace python -c "import subprocess; subprocess.run(['ls', '-l'])" 2>&1 >/dev/null | grep fork
  vfork()                                 = 15

edit 2: here's a similar tale from go, replacing use of fork in fork/exec:

https://github.com/golang/go/issues/5838

https://go-review.googlesource.com/c/go/+/37439/

https://about.gitlab.com/blog/2018/01/23/how-a-fix-in-go-19-...