rckrd

Software engineering is one of the highest leverage things you can do. Keep it up!

I've actually found that knowledge diffusion between different languages and different types of developers can be significant - e.g., game developers work much differently than distributed systems engineers.

Not always, but sometimes, an interesting knowledge arbitrage opportunity when fields start to collide.

Yep, modularity is immensely valuable. Requirements change. Technology changes.

Author here, ironically, guilty of this https://matt-rickard.com/virgo-lang/

Great idea. Some in-depth follow ups are on my radar (I've been trying to publish 1 post a day, yet, never run out of ideas.)

Agreed - tough to nail down a definition of deliberate practice for software engineering. I credit three sources

- Working on a large open-source gave me access to high-quality reviewers from around the world.

- Likewise, at Google, I was surrounded by some exceptionally smart people.

- Finally, like you said, working on projects outside the scope of my work and expertise.

Agreed!

Thanks, fixed!

I agree - I think I needed to add a bit more nuance, which I did in https://news.ycombinator.com/item?id=28087165

Lucky for you #6 is "Have a wide variety of tools and know which ones to use for the job." :)

Fair enough, there are clearly lots of useful syntactic shortcuts. I'm referring to things like prefix/postfix increment/decrement, object spreading vs. object rest, and ternary expressions.

I'd avoid cases where extending the language is done in a non-obvious and ambiguous way.

Thanks! Likewise. Truly a great sandbox for budding programmers.

Yes, maybe usually was too strong of a word - I think this varies with the language. In Go and JavaScript (where I mostly work), it's usually not that useful.

In languages like C# and Java, I've found syntactic sugar to be very useful (e.g. lambda notion)

You're implying the reverse causation. The reflection is "can't find answer" -> "might be hard", not "might be hard" -> "can't find answer".

Surely there is copius documentation on a significant number of hard problems.

One interesting fact is that minikube actually used to be very similar to k3s. (I used to be a maintainer of minikube)

We called it localkube, and it was an all-in-one binary with features stripped out that ran the kubelet, apiserver, kube-proxy, etcd, and other components as Go routines. Eventually it had to be removed - upstream k8s was moving too fast, and there were fundamental issues that couldn't be fixed without large patches (TPRs, the predecessor to CRDs were one).

The problem with this approach is that it requires a ton of maintenance and there can be subtle bugs that come up.

The other distinction not found on this list is that only Docker for Desktop and minikube are cross-platform solutions. Running Kubernetes and docker on Linux is somewhat trivial, but managing a lightweight virtual machine and adding that extra layer of abstraction is what makes it really difficult.

Author here, thanks for posting.

Here's some more color on the reflection on comments, straight from the Linux Kernel documentation that capture my intention much better:

"Comments are good, but there is also a danger of over-commenting. NEVER try to explain HOW your code works in a comment: it’s much better to write the code so that the working is obvious, and it’s a waste of time to explain badly written code.

Generally, you want your comments to tell WHAT your code does, not HOW. Also, try to avoid putting comments inside a function body: if the function is so complex that you need to separately comment parts of it, you should probably go back to chapter 6 for a while. You can make small comments to note or warn about something particularly clever (or ugly), but try to avoid excess. Instead, put the comments at the head of the function, telling people what it does, and possibly WHY it does it."[0]

[0] https://www.kernel.org/doc/html/v4.10/process/coding-style.h...

My favorite tool for visualizing a codebase is Gource. Here's a 1 minute visualization of the Linux Kernel repository from 1991-2015 https://twitter.com/mattrickard/status/1423366779590430721

It parses a Dockerfile (or queries dependencies in a tool like bazel) to get a list of file dependencies and then kicks off an optimized rebuild and redeploy. If you're using buildkit and caches this can be done really quickly (close to native compile without a Docker image)

You can tell it to just sync certain changes (e.g. syncing a javascript file to a container running webpack).

No but I heavily drew from three tools:

(1) minikube, which I also maintained at Google. I saw how difficult getting started with Kubernetes could be even when you were able to create a local cluster. I also knew all the shortcuts that you could take to streamline local Kubernetes workflows (e.g., loading images directly into the minikube docker daemon).

(2) docker compose, which remains a great tool but had no analogue for Kubernetes. It also didn't have a concept of syncing for hot-reloadable projects or built-in file watcher.

(1), Draft by the Deis folks at Microsoft, which was a great tool but suffered from having a component that you needed to install on the cluster to work, where the skaffold architecture was purely client side (making it a lot easier to get started, and faster). I also wanted to make skaffold pluggable, so it supported helm, kubectl, docker, bazel, and other tools right from the start. It also had clear boundaries between build and deploy and ideas for CI and gitops workflows built in.

I built something like this in Go a few years ago but it reloads your infrastructure when it changes, recompiles your code when it needs to, and syncs static files or interpreted files to wherever your app is running (desktop or cloud). Think of it like docker-compose but with file watching + sync + smart rebuilds. https://skaffold.dev/

I just wanted to say that today I hit the 2 month mark for consecutive days of writing. I published 60 posts, one every day. I wrote about startups, engineering, and other thoughts.

I've found that doing something daily is the best way for me to build habits. It's been a fantastic experience. My writing became more concise. It's given me clarity.

And great idea for the site- I submitted a pull request for my blog [0].

[0] https://matt-rickard.com/

On more state externalized to the cloud - Kubernetes is the logical choice. To do that, you need to match dev/prod parity, and you need to have service abstractions so you can easily swap out components.

For more declarative systems - Kubernetes gets us close. We've seen that systems that are "too declarative" like Bazel are too cumbersome for developers to do. What developers need in an environment is just declarative enough to diff - which Kubernetes provides for infra, and tools like skaffold provide for applications.

There's a prevailing narrative that Kubernetes is insanely complex. Probably driven by vendor FUD - if it isn't so complex, why should you pay someone else to do it for you? I think eventually customers will realize that there's not as much magic under the hood.

Author here. I would say that even though you're grabbing the markup from an API, trying to get structured data from presentation logic still seems closer to RPA than not.

And Wikipedia might be an unique example - the markup structure is fairly stable vs. commercial SaaS.

If a competitor offered API access with the data you wanted, you would always choose the API over screen scraping. If a screen scraping business becomes big enough, they usually either partner with the existing sites (i.e. Plaid) or a competitor does.

I love when underlying data structures boil down to directed graphs. Turns out that indentation is a bad way to represent graphs (as someone who has tried many times to encode large Kubernetes service topologies in YAML). A language like dot or virgo (https://github.com/r2d4/virgo) seem more appropriate for this task

Minikube uses a 9p FS for sharing files between host and VM. Fairly easy to implement, and runs completely in user space

Author here - thanks for submitting.

Happy to answer any questions. I have some more ideas on more useful frontends, but I thought I'd publish a simple implementation that others could follow along with and use it to build their own.

I work on skaffold and with the jib folks at Google. Why not have the best of both worlds? :) The code that lives here will use jib for the build and skaffold for the deploy, so you get those really fast (and native) workflows.

https://github.com/GoogleContainerTools/skaffold-tools-for-j...

- We've published libraries to interact with the registry without docker or the docker CLI, which we use in these projects

https://github.com/google/go-containerregistry

https://github.com/google/containerregistry

- Our team has built something exactly like you're describing https://github.com/GoogleCloudPlatform/distroless

Dockerfiles without RUN commands are technically more correct: reproducible, much easier to inspect. However, its quite limiting for the existing corpus of Dockerfiles.

I like to think of kaniko as the (pull) + build + push decoupling of the docker monolith. Other tools, like cri-o, have implemented the complement (pull + run).

Disclaimer: I work on kaniko and some of these other tools at Google

The idea is to not have to maintain separate CI infrastructure in addition to your Kubernetes cluster.

Disclaimer: I worked on this kaniko at Google