javierhonduco

Would love to know from a Googler if this is the #1 model that Google engineers use for coding or if they rely on 3rd parties.

I agree it sounds inclusive but I find it to be in tension with some of his previous writings:

- https://world.hey.com/dhh/as-i-remember-london-e7d38e64

- https://world.hey.com/dhh/wolves-sheep-and-gypsies-ba44af6a

I am all in seeing more people run Linux and I think this might help the whole ecosystem and other distros long term.

That being said I’m a bit concerned about videos like this https://xcancel.com/thestanduppod/status/2091511392717213707 (really hope this is AI generated) and that so many people are okay to give money to David’s project despite his commentary on various societal topics.

I like many of David’s technical ideas, especially back when I did Ruby, but I’m sad he’s decided to spend time and energy on certain topics. Tech needs more kindness, not less.

I really love Zed and I want them to really succeed. Unfortunately it seems that they are swamped with various things and the issue tracker is not a priority for the team, which fair enough. OSS is a gift and I take it as such. Wild proposal but if someone is willing to start a "user advocate" contributor, I'll be happy to donate monthly so user requests can be taken a look at :)

This is very sad. I bought a PS5 with disc precisely to “own” the games. End of an era, I guess.

Great write-up. As a side note (not a Googler myself and this is 100% my opinion) Lalit’s team was hiring in London, UK. If you are interested in working in low level performance tools, this might be a very cool opportunity!

Super cool project! Curious on how this compares to Meticulous in terms of functionality and approach. Thanks!

Wondering to what degree this was done to support Anthropic’s web crawler. Would assume that having a whole JS runtime rather than just a HTTP client could be rather useful. Just hypothesising here, no clue what they use for their crawler.

I am incredibly happy that Apple has added MTE support to the latest iPhones and perhaps the M5 chips as well (?). If that’s the case I don’t think any other personal computers have anything close to Apple machines in terms of memory safety and related topics (Secure Enclave etc).

Hope other vendors will ship MTE in their laptop and desktop chips soon enough. While I’m very positive about x86_64 adding support for this (ChkTag), it’ll definitely take a while…

In my opinion a worthwhile enough reason to upgrade but feels like a waste given my current devices work great.

Really enjoyed reading this. Thanks for writing it!

These metrics can be used in performance reviews at Facebook.

Zig 0.13 is required according to https://ghostty.org/docs/install/build

This is pretty cool work.

Something that’s been on my mind recently is that there’s a need of a high-performance flame graph library for the web. Unfortunately the most popular flame graph as a library / component, basically the react and d3 ones, work fine but the authors don’t actively maintain them anymore and their performance with large profiles is quite poor.

Most people that care about performance either hard-fork the Firefox profiler / speedscope flame graph component or create their own.

Would be nice to have a reusable, high performance flame graph for web platforms.

Axum + minijinja is quite close to this I would say. Been using it for a little while and I am very happy so far.

Haven’t had the chance to play with WezTerm just yet but wanted to share that the author is an incredibly smart, friendly, and humble.

Had the opportunity to work on a project together at work some years back and I can only aspire to be 1/10th as good of an engineer as him. A true hacker.

Or in other parts of the kernel. It's been the case in multiple occasions that buggy locking (or more generalised, missing 'resource' release) has caused problems for perfectly safe BPF programs. For example, see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1033398 and the fix https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

It is not, programs that are accepted are proved to terminate. Large and more complex programs are accepted by BPF as of now, which might give the impression that it's now Turing complete, when it is definitely not the case.

It’s easy to miss this in large codebases. Having to check every single struct initalisation whenever a field is added is not practical. Some folks have mentioned that linters exist to catch implicit initialisation but I would argue this shouldn’t require a 3rd party project which is completely opt-in to install and run.

Personally I’m not a fan of Go’s default zero-initialisation. I’ve seen many bugs caused by adding a new field, forgetting to update constructors to intialise these fields to “non-zero” values which caused bugs. I prefer Rust’s approach where one has to be explicit.

That being said it’s way less complex than C++’s rules and that’s welcomef.

This looks incredibly comprehensive, thanks for sharing!

Should have added that I read this book in 2016, and the first edition is even older, so there’s naturally been lots of new (and exciting) developments in this area!

For anybody interested in this topic, “Release it!” is a pretty good read. (Not affiliated in any way, just enjoyed reading it)

https://www.oreilly.com/library/view/release-it/978168050026...

Overall, I am for frame pointers, but after some years working in this space, I thought I would share some thoughts:

* Many frame pointer unwinders don't account for a problem they have that DWARF unwind info doesn't have: the fact that the frame set-up is not atomic, it's done in two instructions, `push $rbp` and `mov $rsp $rbp`, and if when a snapshot is taken we are in the `push`, we'll miss the parent frame. I think this might be able to be fired by inspecting the code, but I think this might only be as good as a heuristic as there could be other `push %rbp` unrelated to the stack frame. I would love to hear if there's a better approach!

* I developed the solution Brendan mentions which allows faster, in-kernel unwinding without frame pointers using BPF [0]. This doesn't use DWARF CFI (the unwind info) as-is but converts it into a random-access format that we can use in BPF. He mentions not supporting JVM languages, and while it's true that right now it only supports JIT sections that have frame pointers, I planned to implement a full JVM interpreter unwinder. I have left Polar Signals since and shifted priorities but it's feasible to get a JVM unwinder to work in lockstep with the native unwinder.

* In an ideal world, enabling frame pointers should be done on a case-by-case. Benchmarking is key, and the tradeoffs that you make might change a lot depending on the industry you are in, and what your software is doing. In the past I have seen large projects enabling/disabling frame pointers not doing an in-depth assessment of losses/gains of performance, observability, and how they connect to business metrics. The Fedora folks have done a superb and rigorous job here.

* Related to the previous point, having a build system that enables you to change this system-wide, including libraries your software depends on can be awesome to not only test these changes but also put them in production.

* Lastly, I am quite excited about SFrame that Indu is working on. It's going to solve a lot of the problems we are facing right now while letting users decide whether they use frame pointers. I can't wait for it, but I am afraid it might take several years until all the infrastructure is in place and everybody upgrades to it.

- [0]: https://web.archive.org/web/20231222054207/https://www.polar...

There's always room for improvement, for example, Samply [0] is a wonderful profiler that uses the same APIs that `perf` uses, but unwinds the stacks as they come rather than dumping them all to disk and then having to process them in bulk.

Samply unwinds significantly faster than `perf` because it caches unwind information.

That being said, this approach still has some limitations, such as that very deep stacks won't be unwound, as the size of the process stack the kernel sends is quite limited.

- [0]: https://github.com/mstange/samply

Inlined functions can be symbolized using DWARF line information[0] while unwinding requires DWARF unwind information (CFI), which the x86_64 ABI mandates in every single ELF in the `.eh_frame` section

- [0] This line information might or might not be present in an executable but luckily there's debuginfod (https://sourceware.org/elfutils/Debuginfod.html)

Curious to hear more about this. Full disclosure: I designed and implemented .eh_frame unwinding when I worked at Polar Signals.

Had completely missed that line — great point!

Once it’s loaded in memory, if Kernel Samepage Merging is enabled it might not be as bad, but would love to hear of somebody has any thoughts https://docs.kernel.org/admin-guide/mm/ksm.html

Haven’t checked the source code yet, wondering if profiling of code without frame pointes is supported. Curious on their approach.

There’s Kvrocks. It uses the Redis protocol and it’s built on RocksDB https://github.com/apache/kvrocks

FWIW, I never advocated for not explicitly calling Close(). Brought up the finalizer in File, because it seems to have performance implications and it’s called for every file handle.

From the docs (https://pkg.go.dev/os#File.Fd):

> Fd returns the integer Unix file descriptor referencing the open file. If f is closed, the file descriptor becomes invalid. If f is garbage collected, a finalizer may close the file descriptor, making it invalid; see runtime.SetFinalizer for more information on when a finalizer might be run.

To give more context on how I found this out, we were missing a Close for a couple of files in our codebase. As soon as I realised I added them, but checked in production for file descriptor leaks and there were none. Checking Go’s source code led me to the finalizer and this doc confirmed what I saw in the code.