Hacker News Reader

Comment by me551ah | Hacker News Reader

I wish Apple would give Docker some love like Microsoft does. Using Docker with WSL is a breeze, and it runs so much better than MacOS. And as an advantage you get access to Linux package managers which are so much better than brew. Brew is good, but Linux package managers like pacman and apt are so much better. Having a proper Linux distribution open in one window while I play 'Call of Duty' in another, is one reason why I've moved to Windows again.

View on HN

The only problems I’ve had with docker slowness (though it’s been a very significant problem) is shared file system performance. It’s made many use cases and tools that are slow to begin with (like webpack) basically unusable for a development workflow.

So the main thing Apple could do to show some love to docker is build out full apfs support in the Linux kernel. I have no clue how much work that entails but presumably it’s pretty massive, and it seems totally unlike them. Maybe one day they’ll have a come to Jesus moment like Microsoft and start caring about developers (non-iOS developers) but I don’t really see it happening.

If you found this unusable, you probably haven't had to work with the Virtualbox-backed version of Docker for Mac :-)

Have you considered storing the npm_modules and running webpack on a tmpfs/volume (so that it doesn't have to go through the shared FS layers) and only copying the end product to a shared volume?

Or maybe we can realize that npm and webpack are both trash. And we should stop embracing them and move web dev away from this mess.

Don't get me wrong. They were good workaround years ago. But I hope we can do better.

Creating and mounting a loop-volume within the HGFS and then running NPM in there, would probably also be helpful. It certainly is when wrangling a directory with lots of tiny inodes over SMB. (It’s also what Apple themselves do for running Time Machine over SMB: they create a sparsebundle image on the remote and mount it locally, and write to that.)

(The efficient thing about this class of solutions, if you’re wondering, is that the client mounting the loop-image ends up owning + managing the loop-image’s internal filesystem’s metadata within its own local disk cache, such that it can coalesce filesystem metadata writes and only push a new copy of the disk-image blocks backing the filesystem metadata after a potentially-huge number of changes. With a network/host-guest filesystem, meanwhile, every filesystem metadata change must become its own synchronous message to the host, to be pushed to the host’s filesystem driver for linearization, so it can succeed or fail relative to other things going on within the host.)

I used to run my own Xhyve based VM just to avoid the brutal VirtualBox Docker system on macOS. Man that’s some bad memories.

The newer Docker for Mac is better, but file system perf still could be massively improved. Big node_module directories can still cause pain even today

> Big node_module directories can still cause pain

is there any other kind?

Bigger node_modules directories

Yep, using Webpack in Docker on a Mac is essentially unusable. I see around a 5-10x slowdown in Webpack builds. Using docker-sync seems to help the CPU out a bit, so my MacBook Pro doesn't do the jet engine space heater routine quite as much, but the performance is still nearly unusable.

https://docker-sync.readthedocs.io/en/latest/

webpack can be made use-able relatively easily if you do not mount node modules over the shared file system. I've been doing this for quite awhile with a volumes declaration in docker-compose that looks like this (running nextjs, assuming /usr/src/app is where your dockerfile has your node stuff):

(on service definition): volumes: - .:/usr/src/app:cached - node_modules:/usr/src/app/node_modules/ - next_artifacts:/usr/src/app/.next/

and then in the top level volumes key defining node_modules and next_artifacts as blank/default.

That means I mount everything except node modules and the build artifacts so the shared filesystem does a LOT less work trying to sync stuff. The downside, of course, is that I need to run npm commands both inside the container and outside if i want them in my IDE. A fair trade for decent performance. That setup is still not as fast as native but definitely usable and does not send my machine into space header mode much more than normal usage.

Interesting. I have always had the same docker-compose setup with a separate node_modules volume inside Docker, but I've always still had the jet engine space heater issue.

I've been able to improve performance in docker-sync by adding high-churn folders that I don't care about to sync_excludes

Virtualbox-backed Docker on Windows was my first introduction to Docker - it literally put me off touching Docker for 1-2 years!

After eventually falling in love with it on Windows and Linux, I later tried it on MacOS... oh my. Not fun!

The fact you need to partition off RAM and CPUs for Docker on the Mac is the killer for me.

Don’t you have to do that under windows too given WSL2 also runs as a VM?

no not in wsl2. (you can, but there is no need for that) (btw. disk storage can be too small, but that is the only problem that can occur https://docs.microsoft.com/de-de/windows/wsl/compare-version...)

Actually that's done for you, whether you control it or not. WSL2 is essentially a VM with better host integration.

actually under hyper-v windows is a vm aswell, it's just not visible. it's the reason why the hyper-v hypervisor platform was needed.

The shared filesystems are also quite buggy.

I was developing a database as a personal project that involves mmapping a file and the Docker shared filesystem had some peculiar behavior they was different from how normal filesystems work.

That _used_ to be a problem for me, until I found Mutagen. It is _surprisingly_ easy to set up, and my webpack builds in Docker are only about 5-10% slower than running natively.

https://nodewood.com/blog/how-to-speed-up-docker-on-macos/

Sounds like you like Linux but run windows ;) yeah outlook word

Apple (as you may know) did in fact have a spell as open-systemsy Unix-loving guys, starting from the NeXT acquisition but ending no later than the release of the iPhone. The usual pattern, basically: https://news.ycombinator.com/item?id=7525256 .

Ah, yes, the Unixness of the Lisa, the LCII, the Quadra, the System 7, Mac OS 8.1, Mac OS 9.2...

So much Unixness on the Mac OS boot ROM, the Pascal API, the lack of memory protection, and so on...

Please reread my comment.

> Brew is good, but Linux package managers like pacman and apt are so much better

I used starting fresh on the M1 as an excuse to give MacPorts a go, and I like it much better than Homebrew. There's some smaller packages that aren't on MP, but all the big stuff is there and to me it feels much more like a Linux package manager.

I’ve always preferred MacPorts to brew. I think you’ve nailed it, it just feels much more like the Linux package managers I’m used to. Homebrew has definitely won the popularity contest though.

IIRC my issue with MacPorts was that it insisted on recompiling the world from its ecosystem as dependencies because it kept everything super-stable, whereas brew would attempt to use system ones if they were viable. It was more or less FreeBSD Ports vs. Debian apt-get as the model at that point, and MacPorts was (predictably) FreeBSD-like.

Over time I'm not sure brew is any better now, though. I have unlinked versions of gcc, python, etc., etc. under /usr/local that are there just to handle brew packages that listed them as pinned dependencies. It's nice that brew doesn't expose them to CLI unless I want it to do so, but it's not less complex.

Assuming MacPorts has a good "bottle" type concept of precompiled packages now too (haven't looked for years) it's probably about the same as brew now, just more stable. If they still compile from source every time a la BSD, that would be my main sticking point.

MacPorts has binary packages for most things these days.

I used MacPorts ages ago, and honestly thought it went away, which is why I switched to Brew, which I've never gotten comfortable with.

Could you expand on what is better about macoorts over brew? I quickly got onto the Macports page and saw first off that sudo is needed for updating itself and second that I have to select the right version for my OS: Brew is a single install that does not require sudo to update so there’s a few things that already make Brew more appealing for me. What does MacPorts do better?

This is the article that prompted me to give it a try, and summarises my feelings better than I could:

https://saagarjha.com/blog/2019/04/26/thoughts-on-macos-pack...

Also, the manual of macport is beautiful, as well as the port file.

I'd like to be done with Homebrew. (I asked you to install pv, not thrash the world.)

Anyone tried Nix? I'm trying it soon. If it doesn't stick, yep, back to MacPorts like it's 2007.

Yes, Nix is totally viable. Steep learning curve, but many benefits.

I switched to Nix a few years ago and have had no regrets.

> Using Docker with WSL is a breeze

Have there been some updates recently? About a year ago we were trying to use Docker on a windows host at work, and dealing with things like file system paths was a nightmare

Several months ago. A year is a long time for the microsoft bleeding edge these days. You can come back to something a year later and it is completely revised in many respects. Bugs gone, new ones created and discovered, whole subsystems rewritten etc...

enable WSL2, install Docker Desktop in windows and it just works. you dont even need to install docker in WSL, its done automatically and kept updated by docker-desktop.

You might also need to update some settings in your BIOS to enable the hypervisor, but once you get it working it's a breeze to use.

WSL is really remarkable.

WSL is Linux. They run the unmodified Linux kernel in a pico-process an docker runs on top of that.

Technically there shouldn’t be file system issues right?

It's even mode magical than that : the docker containers run inside a Linux kernel that is not the one running your WSL2 environment, but the two are in cahoots such that the filesystem is shared and hence container mounts are as fast as they would be on a bare metal Linux box (e.g. -v $PWD:/workdir).

WSL1 used a syscall translation layer in a "pico process", WSL2 simply uses Hyper-V

WSL2 is now just a regular VM

There are some differences in the software thought.

True, but it uses a Linux kernel over hyper-v rather than a wrapper over the win32 api like WSL1 (from what I understand!)

Yes, but it is a slightly modified distro which causes some issues

And not all are minor. Not being be able to run LXC is a deal breaker for me.

You could with a little work in June 2019, not sure what the picture is like now but it might be possible. I've used nix quite a bit on WSL2 which was surprisingly good.

https://blog.simos.info/how-to-run-lxd-containers-in-wsl2/

What makes you think WSL2 isn’t able to run LXC?

WSL2 is a custom Microsoft kernel

If you can't do what you want in it, please open a ticket on https://github.com/Microsoft/WSL - there is also a repo for the kernel, but AFAIK technical feedback is best sent through this one.

(disclaimer: not directly involved with WSL, just trying to help)

How does it work with local directories? Is there a separate file system for WSL, or does the linux kernel translate windows fs paths or something?

It's a separate filesystem, but from WSL you can access Windows files in mounted drives via the `/mnt/<drive letter>` directory.

And vice versa: linux home dir is at something like \\wsl$\ubuntu\home\username

Yes, there is a separate filesystem for WSL, but you can still traverse the Windows filesystem as well, including mounting external drives. You can also browse the linux filesystem from windows explorer as expectded.

https://docs.microsoft.com/en-us/windows/wsl/compare-version...

https://devblogs.microsoft.com/commandline/access-linux-file...

https://community.openbiox.org/d/72-windows-subsystem-for-li...

It's the same filesystem with some kind of translation/adapter. I love it because it makes supporting our multi-platform OSS projects easy.

You do have the occasional issues regards line endings; that's the only negative I've had so far.

EDIT: H12 is more accurate. The Linux OS is separate, but the Windows FS is mounted to a logical place and "just works".

Yes line endings for configuration files was one of the issues we ran into

I do have locking issues with some sqlite files. I guess it has to do with the fact that the filesystems are shared. I wish I had time to investigate more.

It’s not an unmodified kernel.

It is basically an unmodified kernel. Like in any purpose-built usage of Linux, it is compiled with specific options and patches for the environment it is made to run in, but you can replace it with your own kernel image if you want.

It's not an unmodified kernel.

Can you explain what modifications you believe it has that typical installations of Linux don't?

Don’t try to rephrase what I said to put words in my mouth. I stated a fact, which you can verify here if you please. https://github.com/microsoft/WSL2-Linux-Kernel Please let us know what you find.

I am not putting words in your mouth. I am trying to understand what your concern is. Every major vendor of Linux distributions maintains their own source tree with patches specific to their usage. That is the way Linux is normally used. So what?

I don't see how it's useful to say that it's a "modified kernel" due to it being packaged for a specific application in a way that is necessary to use it.

Huh. I've seriously used... I dunno, maybe five Linux package managers, on workstations and servers alike, plus poked at a couple others, and the only one I'd almost rather use day-to-day than Brew is Portage, but even that, probably not.

I like having the system strictly separate from my crap, and I think the UI is fairly good. The variety of packages available out-of-the-box is outstanding. I miss it when I'm on Linux, now, in a workstation-not-server context (yes, I know, there's LinuxBrew, but the package set is much smaller and less well-maintained). I started on MacPorts but got sick of it borking itself every few months such that it was faster to nuke the directory and reinstall everything than to figure out what it'd screwed up this time (granted, that was about a decade ago, maybe it's great now).

Brew gives off all kinds of signals of being something I'd hate (cutesy; a system tool written in Ruby; breaks with norms) but I like it a ton.

Brew consistently upgrades things that shouldn't need to be upgraded. I've spent more hours than I care to count fixing broken Postgres installations which regularly get major version bumps as a result of a completely unrelated install or upgrade.

I've never used linux as a dev machine, but in general apt seems much more reasonable in this regard. My frustration with Brew is pushing me to seriously consider a linux machine, so if anybody has counterpoints here I'm definitely interested in hearing about your experiences.

I don't care that Brew "breaks with norms" I just care that it breaks my shit.

I think I must use it differently from how other people do. I use it to install tools I'll use directly, so I practically always want those to be at latest (or otherwise don't really care what version they are). If I need something at a particular version I'd use a container, or install it manually in some isolated folder, or something like that, since odds are if I need something at a particular version I'm going to need it at multiple particular versions and to be able to re-create the installation on other environments.

So I end up with:

System -> Apple-managed

My tools, as in programs I personally use -> pretty much entirely Brew, in fact I think on my current workstation this category is 100% brew-installed

Dependencies of anything I'm working on -> some language-specific version manager (which itself is may be brew-managed, actually) plus containers or VMs with scripted installs, probably.

On linux my experience is typically more like:

System -> package manager

My tools -> package manager, plus some sketchy extra repos that I hate to add but do anyway because I don't want to screw with manually updating things, plus several things installed manually, plus a bunch of things on older versions than I'd like but not worth the trouble/risk of finding some way to upgrade without it being a PITA.

Dependencies of anything I'm working on -> some language-specific version manager (almost certainly not available in the distro's official repos) plus containers or VMs with scripted installs, probably.

So for my use, Brew cleans up the "My Tools" workflow very nicely compared with Linux, excepting, kind of, my days back on Portage/Gentoo, which of course has its own problems.

I don't quite understand your argument. I use my distros package manager in a very similar way to how you describe brew for my tools, and also use containers in the contexts you describe.

Only difference I see is that my containers are running natively (eg no VM in the background) and that I've not had any random errors from my package manager in years. Not sure what brew is like these days but last I used it the experience felt like a half baked apt/yum to me (3+ years ago though)

GP and a sibling comment (quote: "Something as simple as "brew upgrade youtube-dl" could end upgrading dozens of _unrelated_ packages, such as postgres--which ends up breaking my local development environments.") seem to describe using Brew to manage dependencies of applications they are developing, which I do not do, and wouldn't do with a Linux workstation's package manager either, so I never have those problems. That's what I was pointing out.

For me, Brew is for managing my personal software I use that doesn't come from Apple. Project dependencies, including the version of the compiler or interpreter for the language you're writing, don't belong brew-managed in most cases, which seems to be what's tripping people up when they try to use it for that.

Yes, containers run better on Linux because they're native. No quibble there. I just find I'm much, much better able to cleanly manage my personal software (not project dependencies, which, again, I wouldn't try to manage with my workstation's Linux package manager, either) with Brew on macOS than in any Linux distro I've used. 99% of what I ever want to run (outside the base OS, and project dependencies) is on there, available at a single "brew install", after I do nothing more than install Brew itself, versus 50-95% on Linux (depending on the distro), where I find myself adding all kinds of extra repos and installing one-offs a variety of ways just to get to a baseline level of having all the stuff I need at new-enough versions. And the interface is above-average, in my opinion (but again, Portage/Emerge is my favorite package manager on Linux and maybe the only one aside from Void's that I've found pleasant to use, so I may just be weird)

I've had this upgrade probablem recently as well, with specifically your example. Even though I like brew much more than linus package managers, this seems like a massive issue. Why the hell is this a thing? Has nobody fixed it?

My example of what? I’m not sure what you mean. As far as I can tell my example is that I only use Brew to install software for which the question “which version do I want?” may always be answered with “the latest” or “I don’t care”, so I don’t have an upgrade problem.

Anything I need at a particular version gets installed some other way, unlike how I operate on Linux, where anything I need at a particular version plus a bunch of other stuff all gets installed outside the package manager (talking workstation usage specifically)

Indeed. I've run into this multiple times. Something as simple as "brew upgrade youtube-dl" could end upgrading dozens of _unrelated_ packages, such as postgres--which ends up breaking my local development environments. Perhaps that's the wrong command to use, but either way, it's still frustrating when it happens.

Why the hell is this happening? Happens for me too.

Why don't the brew developers change this/add an option/make it opt in?

If having a specific version of a tool is important for your work, it’s probably best to install that tool separate from homebrew and manage that install separately. The Postgres docs seem to recommend the EDB installer.

https://www.postgresql.org/download/macosx/

Likewise Python and other programming tools where running a specific version is important are best managed outside homebrew.

Yes, Ubuntu/Debian are very conservative with their updates. Especially if you're on an LTS release, so you're unlikely to have something break from an upgrade like that. Brew tends to be very up-to-date.

this right here. When I instruct brew to install something, I didn't instruct it to upgrade everything else. And I didn't instruct it to do a 30 day clean up.

apt and yum don't go doing things you didn't tell them to do (usually)

Yeah, let's talk about how brew still can't just remove the leftover dependencies or uninstall a package with its otherwise unused dependencies without some extra command or hacks.

‘brew autoremove’ does this.

Is this new? I swear I searched as recently as last week and the only thing that came up was "rmtree" and the "brew bundle dump && brew bundle --force cleanup" workarounds.

It’s not “last month new” for sure. It was in the list of commands in the manpage last week.

brew might have a discoverability challenge. I've been installing an rmtree helper to do this for years and didn't even know it was an option to autoremove them.

As a casual user I like Brew. As a long time mac developer and private cask maintainer(no longer thankfully), I wish there was something close to what the Linux ecosystem has.

As a casual user (install something once every 6 months), I can't even keep brew's insane terminology straight. If I was a REGULAR casual user, I'm sure it would be fine.

Have you tried MacPorts?

I tried it, but everything required sudo and that wasn’t an option at the time. I primarily develop in Windows(on a Mac) now because Unity runs much better on windows than Mac OS.

You can install MacPorts into a local directory without admin privileges.

I'm 100% with you. Moved from MacOS to Windows because of WSL1.

Tangent warning:

When WSL2 came out I ended up moving away from WSL and moved to running my development environment inside a VMWare VM and connecting to it via VSCode remote ssh development feature (the same one used for WSL). This is essentially a manual version of WSL2, something people have been doing for ages.

Now, whether I develop from Windows or MacOS, I am always connecting to my Linux VM. If I need to run docker, it's running in that VM.

I got to the point now where I forward my SSH port from my public IP to my desktop PC at home. When I am away from home (e.g. working in the office with my laptop), I connect to my desktop (via ssh on my public IP) and due to the seamless integration VSCode has for remote development, It feels like everything is running on my local device.

The PC is of course handling compilation, intellisense and all that good stuff while my laptop is essentially a thin client. The laptop now runs faster, has a longer battery life, doesn't spin up the fans and also doesn't need to be upgraded.

My home PC, electricity and internet bills are also tax deductible because I use them for work purposes.

I'm definitely interested in WSL for developing, but last time I tried it a while back (WSL2 just came out I think), there was basic functionality missing e.g. I couldn't write and run my own SystemD services. Has the situation improved since then?

When you say "proper Linux distribution", I assume that's still CLI only? Do you develop with e.g. emacs, vim on WSL, or do you have some IDE with remote running and debugging into WSL? Or have a missed a trick and in fact X/Wayland applications can be run on WSL?

> still CLI only?

With openssh in windows, it's been quite easy to run an x server in windows, and use ssh with x forward for gui access.

But rdp feels more native to windows, and both x and Wayland has rdp server backends - so you can do things like: https://www.nextofwindows.com/how-to-enable-wsl2-ubuntu-gui-...

But apparently ms is working on a Wayland compositor allowing directly running gui apps - I don't think it's quite there yet:

https://www.phoronix.com/scan.php?page=news_item&px=Microsof...

https://github.com/Microsoft/WSL/issues/938#issuecomment-763...

Somewhat related: https://ltsp.org/ I'm not sure about the state of a non-x rdp server for thin clients though.

Thanks for the links, some interesting stuff there! I think my end goal is to have a contained development environment whilst not being forced to use specific tools (i.e. VSCode remote debugging, eugh) and not sacrificing compilation times by running in a VM that's too slow.

From the look of it, if I were to set up the WSL2 Ubuntu GUI with RDP, that'd tick all the boxes, right? And as a bonus I'd be able to access files in WSL2 from Windows.

Any idea how WSL2 interacts with VPNs? I've seen some mixed reports around the web but this is also a must-have for me, if I can't use a VPN in my dev environment it's game over.

I have it working with vpn. We have vpn at work and it is forced tunneling mode. It modifies routes on my laptop. Since wsl2 is a vm on a different network the vpn client does not know about this network and doesn’t know how to route traffic. If you follows these instructions it solves the problem https://github.com/sakai135/wsl-vpnkit

YMMV but for me personally with Dell Sonicwall it basically does not work (even workarounds that mess with MTU sizes are unstable). In my case the behavior is that general networking speed drops massively as soon as both VPN and WSL2 are active. I had to revert back to virtualbox, which does not have such problems and as a bonus allows me to run proper systemd (another thing missing in wsl)

Interesting, that might be a dealbreaker then. Have you found significant slowdowns in working in a VM vs native, both in terms of compilation and day-to-day work, or has it been negligible/unnoticeable?

P.S. thanks for making an account just for this reply :)

Needed to create one for a long time ;) General cpu-bound tasks seem fine in a vm; i/o is definitely slower but much to my surprise linux’s ext4 filesystem is so much more efficient (?) than ntfs for small files that even inside a vm (vbox or wsl) git actions are noticably quicker than on native Windows! I’d prefer is wsl2 for convenience over vbox if I could but my VPN is indeed the dealbreaker and corp. won’t support any other VPN software

> sacrificing compilation times by running in a VM that's too slow

Fwiw wsl2 runs in a vm now (hyper-v). AFAIK it's in order to increase fs performance "inside" Linux (so faster compiles if compiling in wsl).

It's also possible to use cifs/samba shares - but I don't know what you're compiling? Chrome/Firefox sized c++ projects?

Might be worth it to try with a tmpfs/ramdisk either way?

> there was basic functionality missing e.g. I couldn't write and run my own SystemD services. Has the situation improved since then?

No. WSL2 uses it's own, minimal (proprietary) init, which basically launches default shell for the configured user and that's it. No service management in sight, and no equivalent of systemd's user scope or user session either.

You can boot the WSL vm with systemd but it requires some fiddling.

That said, I've been playing around more and more with WSL2 on a secondary machine as a current primary Fedora user (for about 10 years now) and I really haven't found a good reason why I would need systemd in the WSL2 VM vs the custom init.

A major use case I have for WSL2 requires systemd, full stop (it's critical for the package manager I use.) It's pretty painful not having it available, and this is really something that needs improvement IMO. To be fair, this could be fixed in the software too (to some extent, not fully but working) which would also be a solution. But I suspect this isn't the only problem people encounter. There's a lot people use modern Linux for.

In practice that hasn't been a roadblock for me, because VMWare Workstation 15.5 finally supports running on top of Hyper-V, so I can have both working at once. Moving everything to a single hypervisor API has some nice benefits...

One possible solution to this is removing systemd from all distros. That way you won't need it for your package manager anymore.

Yeah, and let's continue with removing Win32 from Windows. That way other folks do not need to bother with Wine anymore.

/s, obviously.

A more realistic option is to abandon WSL as yet more crap from Microsoft.

Please take the crybaby shit to Slashdot or something, man. It's not a good look.

It is like Excel - most people use at most 20% of the functionality, but any competitor that implements only 20% is not usable.

Most WSL2 users use it as shell to run occasional ELF/x64 binary that runs in console; for services, they would use Docker Desktop for Windows. Anything beyond that and you will quickly find out, that it is not really a standard linux distribution.

It's not full-on systemd, but you can run init scripts on WSL startup as of Insider Build 21286.

https://blogs.windows.com/windows-insider/2021/01/06/announc...

There's definitely a few quirks. My biggest annoyance right now is https://github.com/microsoft/WSL/issues/5762 .

Note this only impacts calling the Linux go toolchain from the Windows side (via the /$wsl/ path). Its not an issue inside WSL... but it comes up when you want to say configure Windows IntelliJ/GoLand to use the Go compiler hosted inside of the WSL VM.

> When you say "proper Linux distribution", I assume that's still CLI only? Do you develop with e.g. emacs, vim on WSL, or do you have some IDE with remote running and debugging into WSL? Or have a missed a trick and in fact X/Wayland applications can be run on WSL?

You can run X apps just fine in WSL2 as long as you have a display server running on the Windows side. I use X410. Important to note, Microsoft is working on native Wayland support long term. For what it is worth, I run IntelliJ/GoLand from inside of WSL and use X410 to render the GUI. This works great.

Your set-up sounds exactly what I'm after - from my comment to a different reply:

> I think my end goal is to have a contained development environment whilst not being forced to use specific tools (i.e. VSCode remote debugging, eugh) and not sacrificing compilation times by running in a VM that's too slow.

Are there any other gotchas to working like this? As it happens I'm a go dev using GoLand so your go-specific issues are of interest to me. I'm not too bothered about not being able to use Windows GoLand as I'd be just using it from WSL2 anyway, but I'd be interested to know if there are any other pain points. Any issues with VPNs, if you're using them?

I haven’t tried a VPN yet, but I have heard of some possible quirks due to how the networking is handled between Windows and Linux in HyperV

Edit: Im switching from Fedora to a WSL setup for work in the next week. Ill report on issues I encounter in that migration. I suspect VPN will come up because one of the key reasons im switching off Fedora to Windows+WSL is due to corporate VPN requirements.

This is the fix for vpn and wsl2 been running this for 6 months no issues. https://github.com/sakai135/wsl-vpnkit

That'd be super great if you could let me know how it goes, I'm tempted to make the same jump. Thanks!

I'll reply to this comment when I do know

I wish it was possible to use USB devices directly inside WSL e.g. for embedded development. I believe there are "tricks" like using USB over LAN, but I have not tried it yet. Once they make possible to use USB devices natively I'll be over the moon.

> I assume that's still CLI only?

If you install an X server on Windows you can run graphical WSL apps. It runs really well too. Years ago I used to run Sublime Text straight from within WSL.

I've been using WSL / WSL 2 for a few years now for full time web development. A while back I made a video going over all the tools I use and how I have Docker, WSL 2 and a bunch of other things configured at: https://nickjanetakis.com/blog/a-linux-dev-environment-on-wi...

I run VcXsrv X-Window on Windows (Multiple windows mode) and then from WSL2 Ubuntu I've run e.g. "terminator" and Jetbrains CLion C++/Rust GUI, Linux Version "~/bin/clion.sh &". It works very well.

Yep, that's what I cover using in the video linked above too. VcXsrv is so good. Nowadays I mainly use it to share my clipboard between WSL and Windows so I can use native Linux tools that copy to the clipboard without resorting to any hacks.

>When you say "proper Linux distribution", I assume that's still CLI only? Do you develop with e.g. emacs, vim on WSL, or do you have some IDE with remote running and debugging into WSL?

VScode directly connects to WSL. You can open folders/files with 'code <folder>' from the WSL terminal, save, run whatever from VScode etc.

As for Docker, you can also use the GUI from within Windows that automatically connects to WSL but admittedly I almost never use the GUI.

I did try VSCode remote running and debugging, but found it fiddly to work with and half-baked. Plus, I was sorely missing IntelliJ, so I quickly stopped trying it out - maybe I'd get used to it eventually, but it didn't seem worth the onboarding cost. The RDP and X410 solutions suggested in the replies to GP seem solid though.

Re: CLI, there is ongoing work on getting graphics support for over a year now: https://lkml.org/lkml/2020/5/19/742

There's this for running systemd inside WSL2: https://github.com/arkane-systems/genie

Where do you see the limitations of Brew? Despite it being a little wonky on beta releases of a new macOS iteration it works fine for me.

As others mentioned (it is slow, has problem with multiple users, cannot pin versions) it is also missing features the linux distribution have. For example, you cannot have Provides: alternatives as rpm/dpkg do, you must use packages for resolving dependencies as they are provided by upstream.

For example, when postgresql 12 was released, it took some months to appear in brew. Meanwhile you could not use alternate taps to resolve dependencies, if some package required postgres, it had to be the original one.

Why would you install postgres through Brew though?

Those times are way gone, that's the purpose of containers.

I've been using Brew for a while to just install "core" packages like python, curl, wget and such, and everything else like a postgres, nginx, whatever..a go to a container.

Because if you need to ingest some data, it is much slower with Docker Desktop for Mac :/

Also, I have some tools installed with brew, that have postgres as dependency (e.g. pgloader or mapnik).

> Those times are way gone, that's the purpose of containers.

Please elaborate on the claim that "running a SQL database" is the purpose of containers.

To be fair to GP, running sql database inside a container does have benefits for development, and in many situations, for deployment too.

However, if I do some exploratory experiments, where I don't care about repeability and where I use other local tools (like the mentioned mapnik, or jupyter), having it in container is needless complication.

Why wouldn’t you want to run a database under namespaces and cgroups from a dependency-bundled live archive file tree?

By and large, there’s no such thing as a container, there’s just sprinkles of housekeeping magic. To wit, Docker implemented in around 100 lines of bash:

https://github.com/p8952/bocker

Problems come when we think that today’s containers manage to actually contain anything, bring any security guarantees, or do much else than just slightly-more-successfully jump start a configurable bundle of dependencies.

I think you're being unkind to containers. Yes it's easy to say that "containers aren't a thing" and then list all the little tools that are used to implement them. That doesn't make them not real any more than any other abstraction.

Why wouldn't you want to run a database under VT-x, with random emulated hardware and a dependency-bundled disk image? By and large there's no such thing as a VM, there's just sprinkles of housekeeping magic?

Containers as specced and implemented do come with security guarantees. And if they fail to meet them it's a bug.

Not going to wade into the "should" or "shouldn't" of this, but I have used postgres-via-docker for ... few years now, and it is a DREAM. And I never have to worry about versions or dependencies (at least I haven't yet).

I've installed Postgres on my Mac with Homebrew, Docker, and https://postgresapp.com. There are arguments for each of them. On the pro side:

- Homebrew is a general purpose package manager, and Postgres is a package you might want managed.

- If you're using Docker/Docker Compose for a project anyway, that's the obvious way to do it.

- Postgres.app is a specialized tool just for managing Postgres installs, so it's hard to beat if that's what you need.

Some thoughts on the tradeoffs though:

- Homebrew really doesn't like the idea of "versions". It wants everything to be on the latest. That can be fine if you just need a tool locally, but if you want dev and prod to match, it is a pain in the ass.

- Docker isn't really very good at persistence. That's probably not a problem for local development, but you should be aware of it. Running it on a Mac introduces speed and memory issues you wouldn't otherwise have. And now obviously there's the M1 problem.

- Postgres.app is another thing to install. If you just need Postgres for one particular project you might not know about it or want to deal with installing something new.

It's been argued that the recommended way it works is a security issue:

https://applehelpwriter.com/2018/03/21/how-homebrew-invites-...

https://askubuntu.com/questions/261326/is-it-safe-to-chown-u...

On Apple Silicon it now installs to /opt/homebrew, a change they’ve been wanting to do for a while.

Does /opt/homebrew still end up in root’s PATH, I wonder? That has the same issue that /usr/local has I think. Letting users mess with root’s environment basically means there is no real distinction between root and non-root.

No, it doesn’t.

It’s not really insecure. See: https://security.stackexchange.com/q/187502

Last time I used a Mac was around, maybe 3 years ago, so this certainly may have improved.

I had a lot of issues with Brew, but the biggest one was how slow it was. Upgrading all packages on my Mac used to take hours.

Still quite slow. As far as I know, it's just a big heap of small ruby scripts that invoke each other, and ruby isn't known for blazing-fast performance. It also uses git internally, and quite heavily, so that probably adds some overhead as well.

It’s been a little while since I relied on it heavily, but you still can’t install/pin specific versions, right? That’s a huge limitation if you want to do any reliable development on macOS directly without using a vm or docker.

It’s also just so slow to update if it’s been more than like an hour since you last updated, the way it uses one big git repo under the hood is just chaos.

You can pin versions in brew (brew pin <package>) to prevent upgrades.

You can install specific versions, but it requires some gitfu - you need to uninstall, find the brew commit where the package is at the version you want, then install from that specific git blob.

Can you pin versions in apt? Last time I checked it involved a lot of work and swearing, but that might have improved lately.

Not really, it was pretty verbose, but wasn't hard. Pinning is for setting package priority between multiple repos.

What you're looking apt is apt-holding:

apt-mark hold libxfont1

That been around since 2013 IIRC. And there was dpkg way of doing it before.

I second that. On my M1 MacBook Pro, I have both the M1 version and Intel versions installed. I have found that if I use the Intel installation to install something like MIT-Scheme in a terminal running Rosetta, the it is available everywhere. It took me a little while to get that sorted out.

I think the main criticism of homebrew is that it's really slow.

Slow where? Searching could be faster, but it’s a few seconds. Installing is fast except where compilation happens. I have to admit I don’t know why it compiles when it does, but it isn’t all packages that get compiled.

It wants to update a few times a day by default when you call it, so running a simple brew install anything normally takes about 30 seconds (I'm in a MB Pro), even if it's just to say the package you want wasn't found.

If you don't run it daily, it takes about a minute or two to update.

But even when it doesn't update, it is extremely slow compared to any other package managers. It is disruptively slow and it takes a lot of resources, even in a powerful machine (and I'm not talking about compilation here).

I think I’m just not used to faster package managers and I don’t install packages often enough for it to feel disruptive to wait a few seconds.

I do like tools to be as fast as possible, though, and I’d forgotten about that update it does sometimes when I run it - that does seem to take a long time.

I’ll have a look at how it works and see what the things are that take time. I would expect network traffic for updates, perhaps whatever it’s getting updates from does some processing (I’m sure it said something about GitHub), perhaps there is some dependency resolution that needs CPU...

It would be interesting to compare its architecture to other package managers if they’re significantly faster.

The slowness is mostly due to Ruby and the git pull. I contributed in the past and reimplemented it in bash, and there isn't much going on, honestly.

99% of the time, installing a package consists of downloading a few zips from their CDN, decompressing and linking. For those cases Brew could just be checking an API instead of constantly cloning the git repository.

I'm quite surprised nobody has reimplemented it in Rust or Go. The architecture is quite simple compared to a normal package manager. Maybe it's just superstition: people see "package manager" and assume it's complicated instead of digging into the code and finding out how it works.

Well, a few seconds for package search in a local index is slow. It's instantaneous with apt on my 3 year old Linux desktop, where here is the famous M1 advantage? :)

Installing anything with brew involving multiple dependencies is also taking forever-ish, compared to mere seconds with apt.

It also has a fair share of problems. Like multiple users on the same machine, I had issues with permissions, etc

Am I the only one who, after installing some brew tool, every bin in /opt/homebrew is immediately kill -9’d at launch, and then I have to brew list | xargs brew reinstall?

Seems to be some signing issue but it’s only happened to me twice and I didn’t have time to properly investigate at the time...

As a daily brew user, I've never heard of that problem.

Do you use any other virtualization software along side HyperV? We have a lot of legacy stuff that integrates with VirtualBox, but wanted to start edging towards Docker.

However, when I last spent a couple of sprints attempting to get them working side-by-side, it was a pretty big failure. I couldn't get VirtualBox 6.0 running without falling back to soft virtualization which was painfully slow (booting a Ubuntu box took the better part of an hour).

Why not using HyperV itself instead of Vbox?

Because of a lot of existing internal tools that directly call and manage VBox to set up local testing environments and stacks.

It's a planned initiative for the future, but it means a fundamental change in a number of tools and would require a rather large set of rewrites.

On top of that, out of the large pool of users, only a small handful would actually need to run Docker side-by-side with these tools, meaning that it's a huge rewrite to allow a small number of people to use Docker on their desktops. That ultimately means it's getting very little traction and keeps getting pushed down the backlog.

> I wish Apple would give Docker some love like Microsoft does

It is my hope that Docker continues to accelerate down its current path toward replacement by other systems such as podman.

I suspect that Apple has little motivation to add a Linux kernel alongside the existing BSD-compatible kernel, even though macOS does have a built-in hypervisor framework/API.

BSD containers (e.g. jails) seem like a better fit than Linux containers (e.g. docker, lxc) for a BSD-based system.

It’s still a nightmare to run IntelliJ on the WSL mounts. Vscode support is much better but it’s support for JVM languages is definitely a downgrade

Have you tried JetBrains Projector? They have a special installer for WSL that sets everything up, and then you access the IDE from your local browser. It works very, very well.

https://github.com/JetBrains/projector-installer

Apple is not giving proper love to their breakthrough laptop chipset. They amazed the world with "neural net on chip" for ML, even mentioning Tensorflow explicitly in its launch. Here's the real deal: getting anything Tensorflow related to even compile on M1 is nothing short of a miracle. Yes, Apple has a binary version (yuck) of TF you can download, but that's not good enough as many projects need specific versions and not everyone (ie OSS devs) can spend that much effort on a fringe tech.

If I were Apple (ehem) I'd spend a small but meaningful budget supplying devs to projects like Docker or TF to help speedup M1 adoption. Given that the chip market is open for grabs I'd say it could give Apple a much stronger headstart with their in-house silicon strategy, even if that means helping improve products or the bottom-line for well-established corps, some of them competidors.

Give me a break.

The M1 migration is less than a year old and it’s already the single best/ most successful CPU migration in my memory. Far better than Apple’s PPC -> Intel Migration and massively better than whatever half-steps Microsoft has done to port over to ARM.

Apple absolutely should invest in much of what you suggest. In 6-12 months, after the platform is complete and mature enough to identify where the big problems are.

Most of us understand that you exercise caution when migrating mission critical work onto a 6 month old platform.

In the mean time, Apple has been hiring top Docker talent like Michael Crosby.

https://www.protocol.com/apple-hires-cloud-open-source-engin...

It’s possible they are hiring top container developers just to improve their internal cloud infrastructure. But what kind of hardware do you expect these guys will be running?

EDIT: Trimemd some repetitive stuff.

Unfortunately, I think apple only invests in things apple. So the "target market" is not developers for other platforms.

Too bad because a native apple docker would be really really useful. imagine:

  FROM macos:10.13.3
  RUN xcode-build

(I'm not talking about the current docker on mac which runs linux in a vm)

But where are you going to run the image? On a Mac Mini in your DC?

BTW macOS being a BSD derivative of sorts might benefit from Docker development on FreeBSD, which uses jails instead of cgroups, etc. At least, that could be easier to port, assuming that macOS kernel has the needed facilities.

I was thinking more of docker used for mac development, not docker used to deploy web apps.

It’s not too difficult to get a Linux VM running on top of MacOS on the M1. Then you can do whatever you want on your M1 Mac and have a full Linux distribution in another window.

Call of Duty is a whole other problem.

> Using Docker with WSL is a breeze

Not if your Windows users have to use Direct Access to connect to your container registry...

Are there any good iTerm like terminals for Windows available?

Last time I checked, the terminal was horrible to work with.

Not sure if it's what you're looking for, but Microsoft did recently release a new terminal: https://www.microsoft.com/en-us/p/windows-terminal/9n0dx20hk...

They’re investing in it but it’s no iTerm

https://github.com/microsoft/terminal

I don't think there is anything as good as iTerm. ConEmu is not bad.

Alacritty is cross platform and the terminal I use over iTerm on MacOS as well (it’s so freaking fast). It doesn’t have tab/windows management stuff but works great with Tmux.

Windows Terminal and Fluent Terminal are both good

WSL is probably easier to write for XNU than it was for Windows. Yeah, silly that Apple hasn't yet.

https://www.opengroup.org/openbrand/register/

No need for WSL.

The problem is that Apple does not want to spend money where they don't absolutely have to. In their mind Docker is someone else's problem and if that 3rd party will also pay salaries and taxes, it's perfect for Apple. I am sure once Docker commit resources and solve it, the Apple will be boasting how great Docker works on their platform.

There's a great deal of cases where Apple went out of their way to help out 3rd parties with compatibility, including quite a few cases with the M1 Macs themselves. Is your comment based on anything or just typical HN "Apple = evil"?

The is because those 3rd parties matter to Apple's ecosystem. And hence it is of their interest, Adobe, AutoDesk, Panic, or any creative tools.

Docker and Developer Tools does not happen to be one of them.

I don’t know, supposedly [0] Apple provided patches to help get Blender running on M1.

[0] https://www.reddit.com/r/blender/comments/jsc03l/blender_on_...

And many others, transcribed list here https://news.ycombinator.com/item?id=23643425

And many of those were ported or tested on ARM before M1.

Should I use WSL 1 or 2? Coming from Linux trying to make something workable.

What kind of machine are you running windows on?

It's silver... :D

https://www.protondb.com/app/2620

I heard WSL2 was no good.

Without more detail it's hard to say much that's useful but in my experience WSL2 has been nothing short of excellent. Also switched from Mac to Windows with WSL and the development experience is much better.

People were frustrated mostly with 2 issues...

a) It doesn't play well with deep sleep mode, and crashes. Perhaps it's been fixed, I don't know, I just disabled deep sleep.

b) The networking is different than WSL1 and requires odd workarounds for things like X11 to work normally. I had to use a 192.168.1.x address for DISPLAY instead of localhost. Which required some VB scripting to reliably work for me.

My main issue is the lack of support for serial (so, no USB support): https://github.com/microsoft/WSL/issues/2195.

Other that this I love Windows 10 + WSL2 as a dev environment.

It's a little clunky, but you could use a Windows RFC2217 remote serial server with socat on the Linux side to make a virtual serial port tied to a real one. I've done this (albeit not with WSL), and it did work.

https://gist.github.com/DraTeots/e0c669608466470baa6c

Too late to edit, but there's also an unapplied pull request that seems to make usb passthrough work:

https://github.com/microsoft/WSL2-Linux-Kernel/pull/45

I have WSL1 alongside WSL2 for this. WSL1 works fine with my FTDI adapters to flash ESP8266 and all.

VS Code with remote container dev on top of WSL2 works really well for me:

https://code.visualstudio.com/blogs/2020/07/01/containers-ws...

It's been flawless and I've been using it pretty extensively as a replacement for vagrant development VMs.

I believe one of the issues laptop-related, and I work exclusively on a desktop these days, so that was a non-issue for me.

Reply on news.ycombinator.com