djs55

(Joining the reunion thread) I was just explaining to a new starter yesterday why Docker Desktop used the codename "pinata". Also I reckon some of the more advanced ideas from back then are still relevant today, probably even more now that everything is changing (or can be changed) with agents. Hope all is well!

(co-author of the article and Docker engineer here) I think WireGuard is a good foundation to build this kind of feature. Perhaps try the Tailscale extension for Docker Desktop which should take care of all the setup for you, see https://hub.docker.com/extensions/tailscale/docker-extension

BTW are you trying to avoid port mapping because ports are dynamic and not known in advance? If so you could try running the container with --net=host and in Docker Desktop Settings navigate to Resources / Network and Enable Host Networking. This will automatically set up tunnels when applications listen on a port in the container.

Thanks for the links, I'll dig into those!

I believe even with the visa it’s still up to the immigration agent. I came close to trouble once when asked for my H1B visa petition document (not the visa in the passport). I had a photocopy and was told that wasn’t enough and although they’d let me in this time they expected to see the original in future. I also travelled with a letter from my employer explaining where I worked, job title etc as extra documentation just in case to derisk further.

I feel exactly the same way. College had just the right amount of private space, lots of shared spaces for social occasions / group working and a maintenance department to look after all the tedious domestic repairs. Bliss

Do you mean TSG? From https://wiki.qemu.org/ChangeLog/6.0#TCG

> TCG > Added support for Apple Silicon hosts (macOS)

(I work for Docker on the M1 support) I'm glad it's working for you! There's a bug in the recent Docker Desktop on Apple Silicon RC build which affects some users of vagrant at the provisioning stage when the new ssh key is copied into the machine. It turned out that the permissions of `/dev/null` inside `--privileged` containers were `0660` (`rw-rw----`) instead of `0666` (`rw-rw-rw-`) In case you (or someone else) runs across this there's an open issue with a link to a build with the fix: https://github.com/docker/for-mac/issues/5527#issuecomment-8...

Thank you for this -- I've been bothered by my Windows PC not sleeping properly for the best part of a year. `powercfg lastwake` indicated the Ethernet adapter and then disabling the option "Wake on Pattern Match" has allowed the computer to sleep soundly.

Don't worry, we (at Docker) have been working on Apple Silicon support for a while. The command-line tools work under Rosetta 2 but the local VM inside Desktop will take a little bit longer to port. Just in case you haven't seen it there's some further info on Docker+M1 in the blog post: https://www.docker.com/blog/apple-silicon-m1-chips-and-docke...

Thousands (30) of people are in the trial, half were assigned randomly to the control group. So far 95 people in the trial have caught COVID and, when they unblinded the data, they discovered that 90 of those infections where in the control group. Since participants were randomly assigned into the test group vs the control group and so both groups should have the same amount of exposure, this is a strong signal that the vaccine was effective. Here's an article about Moderna's trial with a link to their 135 page (!) design doc https://www.livescience.com/moderna-vaccine-trial-protocol.h...

I do OCaml programming on Windows and I found that it was a bit confusing at first with too many different ports and install options. However once I settled on https://github.com/fdopen/opam-repository-mingw I was fine. To my surprise I was able to extend existing C bindings to use Win32 APIs fairly painlessly (for example https://github.com/mirage/mirage-block-unix/commit/7cf658f8a... ) . I did have problems with I/O scalability at first but I fixed these by using libuv via https://github.com/fdopen/uwt . The core compiler and runtime are rock solid on Windows. Docker (where I work) ships OCaml/Windows binaries to lots and lots of desktops with no problem.

Apart from the too-many-ports problem, I think the main remaining problem is that too many 3rd party libraries require Unix-isms to build, like shell scripts. This necessitates the presence of cygwin for build (but not at runtime). However the ongoing "dune-ification" of the OCaml universe should help fix this since dune can do everything directly from OCaml code. I'm really looking forward to being able to open a powershell window and type "git clone"; "dune build" and have everything just work.

Yes -- opam and dune are complimentary. I normally use dune (formerly known as "jbuilder") as the build system within my packages, which I then publish and install via opam. Dune does the fast incremental builds, while opam deals with version constraint solving, downloading and general package metadata.

There are some interesting experiments combining the two more closely -- take a look at "duniverse" https://github.com/avsm/duniverse which is a prototype system which would use opam to solve package version constraints and download the sources, allowing dune to build everything at once. The nice thing about this is that you could patch one of your dependencies and then dune will be able to perform an incremental build, as if all the code was in one big project. I'm personally hoping this will help speed up Mirage development as it can be time-consuming to propose a change to an interface and then find all the places that need changing (a cost of having lots of small repos versus a big monorepo)

Are you trying to avoid linking against Cygwin at runtime or trying to avoid Cygwin in the build environment too? In my case I use Cygwin in the build environment (see https://github.com/moby/vpnkit/blob/master/appveyor.yml) using Andreas Hauptmann's https://github.com/fdopen/opam-repository-mingw but I end up with an executable which doesn't require the Cygwin dll at runtime.

I'm looking forward to the day when I won't need Cygwin even in the build environment. Since the OCaml compiler itself works fine on Windows and modern build systems like "dune" are also Windows-friendly I'm fairly optimistic this can happen soon. I think it'll mostly be a matter of removing accidental Unix-isms (like unnecessary use of symlinks) in the build scripts.

I think `opam` failed to find `jbuilder` on the `$PATH`, even though (presumably) `opam` has built it and installed it in it's own `bin/` directory. Try running `eval $(opam config env)` to update the `$PATH` and trying the commands again.

There's a very good OCaml binding: https://github.com/fdopen/uwt . It works really well in the networking component of the Docker for Mac and Windows desktop apps (see https://github.com/moby/vpnkit)

The 1.5% that's in the hyperkit repo is the shim to the Mirage block layer. The qcow2 implementation that's normally linked in is from https://github.com/mirage/ocaml-qcow .

I'm also an OCaml old timer and I think I can relate too. I believe the recent tooling changes are going in the right direction and will eventually fix several of these problems, for example:

There's a push to remove "optional dependencies" which are the reason why opam dependencies rebuild again and again: http://rgrinberg.com/posts/optional-dependencies-considered-... For example in the Mirage project we've been working on this https://discuss.ocaml.org/t/ann-major-releases-of-cohttp-con... but it has caused some breakage here and there.

jbuilder (from Jane Street) is excellent: expressive, easy to understand, builds packages extremely quickly, is actively developed, has minimal dependencies and a lovely manual http://jbuilder.readthedocs.io/en/latest/ It takes care of generating boilerplate for other tools like merlin (which to be honest I never got around to manually configuring). There's also work to integrate it with utop https://github.com/janestreet/jbuilder/issues/114

jbuilder also supports building multiple libraries in one big source tree, so we could switch to a package lockfile model: the author uses opam to create a solution to the package constraints and checks in the specific versions known to work, the build clones the dependency sources and jbuilder builds it all simultaneously. I'm keen to try this on one of my larger projects so that "git clone; make" just works, irrespective of where the host OCaml comes from.

PPX syntax extensions depend on specific compiler versions, so when (for example) homebrew updates to OCaml 4.05 you might find that extensions you need have not been ported yet. ocaml-migrate-parsetree aims to fix this problem http://ocamllabs.io/projects/2017/02/15/ocaml-migrate-parset...

There's obviously still plenty of work to do, but I think things are improving!

I don't know the answer to the mix old/new compilers question but it looks like it's only the directory structure of the compiled artefacts that's important rather than the directory structure of the source code. It looks like you could keep all your source in one directory (unlike Java IIRC) and then compile it with `-I` and `-P` options.

(I work on Docker for Mac.)

Apologies for the inconvenience this has caused.

There was a race condition in a previous release which could allow multiple hypervisor instances to open the Docker.qcow2 simultaneously. Unfortunately this can corrupt the file by allocating the same physical block (cluster) twice, resulting in bad things happening. When this file-locking bug was fixed we also added an integrity check which checks the structure of the Docker.qcow2 on every application launch. For safety the app refuses to start if corruption is detected.

I believe that in these cases, the corruption happened in the past and is now being detected since the upgrade. Unfortunately if the app refuses to start it makes it difficult to reach the "Reset to Factory defaults" menu option. The workaround described here https://github.com/docker/for-mac/issues/1159#issuecomment-2... is to remove the qcow2 and restart the app. Unfortunately containers and images will need to be rebuilt.

For what it's worth after the integrity check and the locking fix went in, I've not seen any recurrence of this error. Please open an issue if you see any other problems!

In the past I worked for a UK company from the UK which was owned by a US parent, where employees in the UK were granted options in the US company under the HMRC Enterprise Management Incentives (EMI) scheme: https://www.gov.uk/tax-employee-share-schemes/enterprise-man... . I think without the scheme if I had exercised an option (whether to hold onto the share or to sell immediately) then I would have owed income tax on the difference between the strike price and the current value i.e. the discount my employer effectively gave me. I think I would have owed the tax even if I was forbidden to sell the stock (e.g. if the company had stayed private). With EMI there was no income tax on exercise and I only owed capital gains tax when I eventually sold the stock.

I'm looking forward to trying APFS, particularly support for sparse files. But you're right, it'll take while before we can rely on it.

We'll probably use AHCI since hyperkit https://github.com/docker/hyperkit (the Mac hypervisor based on xhyve based on bhyve) lacks a virtio-scsi implementation at the moment. When the VM sends a TRIM ("these blocks aren't used any more, you can free them") we need to make use of this on the host. On Linux/BSD you could tell the host OS that the blocks aren't needed in the file using something like FALLOC_FL_PUNCH_HOLE and the kernel would take care of it e.g. by send TRIMs to the real storage device or shuffling filesystem metadata around. Unfortunately the Mac filesystems in common use don't support this so we'll need to manage this ourselves, e.g. by moving blocks from the end of the file into the middle and shrinking the file.

(I'm a Docker employee working on this very issue. I guess my plan to take a break by reading HN failed!)

You're completely right -- it's a problem caused by lack of TRIM in the storage path. In the next beta of Docker for Windows (beta 31 due today hopefully) TRIM should be enabled. The Mac will take a little longer as we need to switch protocols and do more work on the host side -- unfortunately the default Apple filesystem doesn't support sparse files so we can't "cheat" by simply passing the TRIM down to the filesystem layer. We'll probably need some kind of explicit block-level compaction to shuffle blocks from the end of the file into holes that have been created by TRIM.

(I work at Docker)

If you haven't already and can spare the time, could you file a pair of issues: perhaps one for the shared volume problem and another for the networking on https://github.com/docker/for-win ?

We've been fixing bugs in both areas and the fixes should arrive in the beta channel over the next few updates. Thanks for your patience!

The DNS resolving code in Docker for Mac is in the VPNkit project which is open-source: https://github.com/docker/vpnkit. A DNS timeout is a fairly general symptom and it's hard to diagnose fully without a packet capture, but there's one issue that I'm aware of: if the primary server is down then the UDP DNS queries and responses will use the host's second server. However if a response is large and requires TCP then unfortunately we will still forward it to the primary server, which obviously won't work :( I've filed this issue about it: https://github.com/docker/vpnkit/issues/96. We hope to improve DNS, VPN and general proxy support for 1.13 -- please do file issues for any other bugs you find!

If you have the time, could you make a report on the issue tracker https://github.com/docker/for-mac/issues and include the contents of /etc/resolv.conf and "scutil --dns" when you connect and disconnect to your VPN? Ideally also include an example resolution of a name by the host with something like "dig @server internalname". I suspect the problem is caused by a DNS server in the "scutil" list being missing from /etc/resolv.conf. We're planning on watching the "scutil --dns" list for changes, but it's not implemented completely yet.

A minor addition: to use 'localhost' in beta 8 you may need to also run the internal debug command:

  pinata set native/port-forwarding true

In previous betas this setting was implied by "VPN compatibility" mode, but in beta 8 it was made an independent setting.

In beta 9 using localhost will be the default -- as avsm says we've been iterating on the network configuration trying to find the most compatible / least surprising defaults. Hopefully after beta 9 it will be stable on "localhost".

(I work for Docker on Docker for Mac)

Our approach is to focus on functionality and correctness first, and then improve performance over time.

We're building up a suite of performance benchmarks to help us track progress-- are there particular benchmarks that you would recommend we add? I'll certainly add "CPU load while idling" to the list.

Our focus so far has been more on reliability, but we intend to increase performance steadily over time. There are lots of interesting optimisations we can make across the whole stack.

Please do check it out and suggest some particular benchmarks that are important to you -- we're busy building up a performance benchmark suite atm.

That's right -- it'll negotiate 9P2000.u if it can and otherwise drop back to 9P2000. I don't think it'll ever support 9P2000.l the very Linux-specific variant.

We had a quick look -- we think it was taken down as part of an automatic github+travis redeploy which took longer than expected. The deploy scripts could use some more work.