HNHacker News
TopNewBestAskShowJobs

nickelpro

2,445 karma · joined December 21, 2019

https://vito.nyc
submissionscomments
nickelpro··on C++ patterns for low-latency applications including high-frequency trading
Of course, time spans are different. But read the paper, it's dealing with incredibly basic techniques. When we're speaking on the level of "data should be in cache" and "cold code should be far away from the hot path" the fields are practically identical.

You're entirely correct that quant work is dealing with pure latency in situations where game engines might amortize throughput over multiple frames. But a question about "how do I get started in performance/latency work?" isn't usefully answered by "get a Hudson River Trading internship".

nickelpro··on C++ patterns for low-latency applications including high-frequency trading
On the software side I don't think HFT is as special a space as this paper makes it out to be.[1] Each year at cppcon there's another half-dozen talks going in depth on different elements of performance that cover more ground collectively than any single paper will.

Similarly, there's an immense amount of formal literature and textbooks out of the game development space that can be very useful to newcomers looking for structural approaches to high performance compute and IO loops. Games care a lot about local and network latency, the problem spaces aren't that far apart (and writing games is a very fun way to learn).

I don't have specific recommendations for holistic introductions to the field. I learn new techniques primarily through building things, watching conference talks, reading source code of other low latency projects, and discussion with coworkers.

[1]: HFT is quite special on the hardware side, which is discussed in the paper. The NICs, network stacks, and extensive integration of FPGAs do heavily differentiate the industry and I don't want to insinuate otherwise.

You will not find a lot of SystemVerilog programmers at a typical video game studio.

nickelpro··on C++ patterns for low-latency applications including high-frequency trading
Fairly trivial base introduction to the subject.

In my experience teaching undergrads they mostly get this stuff already. Their CompArch class has taught them the basics of branch prediction, cache coherence, and instruction caches; the trivial elements of performance.

I'm somewhat surprised the piece doesn't deal at all with a classic performance killer, false sharing, although it seems mostly concerned with single-threaded latency. The total lack of "free" optimization tricks like fat LTO, PGO, or even the standardized hinting attributes ([[likely]], [[unlikely]]) for optimizing icache layout was also surprising.

Neither this piece, nor my undergraduates, deal with the more nitty-gritty elements of performance. These mostly get into the usage specifics of particular IO APIs, synchronization primitives, IPC mechanisms, and some of the more esoteric compiler builtins.

Besides all that, what the nascent low-latency programmer almost always lacks, and the hardest thing to instill in them, is a certain paranoia. A genuine fear, hate, and anger, towards unnecessary allocations, copies, and other performance killers. A creeping feeling that causes them to compulsively run the benchmarks through callgrind looking for calls into the object cache that miss and go to an allocator in the middle of the hot loop.

I think a formative moment for me was when I was writing a low-latency server and I realized that constructing a vector I/O operation ended up being overall slower than just copying the small objects I was dealing with into a contiguous buffer and performing a single write. There's no such thing as a free copy, and that includes fat pointers.

nickelpro··on No more boot loader: Please use the kernel instead
The reason is very simple, you only get to call ExitBootServices() once (absent hacks that hook the function).

If you want to be able to do anything prior to calling ExitBootServices(), such as choose what EFI application you want to use and options you want to pass it, you need a service built to provide you that interface which itself does not call ExitBootServices().

The name of that service is the bootloader.

nickelpro··on GCC's new fortification level: The gains and costs (2022)
I hadn't thought about the `sizeof(T)` point before, and I can think of a handful of exceptions, but it's a great and expressive rule of thumb.
nickelpro··on GCC's new fortification level: The gains and costs (2022)
It's not about querying the size, it's about what the allocator has available without having to ask the operating system for new pages.

It's got a contiguous block of 6371 bytes? Cool I'll take that, the specific size didn't matter that much.

nickelpro··on Own Constant Folder in C/C++
The existence of a theoretically pure floating-point behavior doesn't have any impact on the reality of the implementations we program for.
nickelpro··on Own Constant Folder in C/C++
The existence of situations where the optimization is not appropriate, such as multiplayer input replay, does not invalidate the many situations where it is appropriate.
nickelpro··on Own Constant Folder in C/C++
If you're using floating point at all you have declared you don't care about determinism or absolute precision across platforms.

Fast math is simply saying "I care even less than IEEE"

This is perfectly appropriate in many settings, but _especially_ video games where such deterministic results are completely irrelevant.

nickelpro··on Experts vs. Imitators
In my career I've at times been a leading expert on, as is the way of these things, a couple niche technologies. I'm speaking from that perspective.

While certainly there are those who struggle to connect with layman audiences, especially in academia, a complete inability to communicate the challenges and successes of a technology, technique, or theory to laymen is a huge demerit against a claim of expertise.

There's a huge difference between walking through technical and theoretical mechanical specifics, and being able to communicate about a subject at different levels of abstraction. The greatest experts can often walk with a laymen right down to and rub against the jargon-filled specifics in a way that leaves no doubt they're able to step over that line without problem.

This is important because we don't perform ideation, architecture, or structural problem solving in the terms of white papers and technical specifications. If an expert does not have an abstract internal narrative with which to navigate the mechanics of the subject, they are likely not an expert at all.

nickelpro··on What Is PID 0?
xv6 and its many forks are what I'm thinking about

You address this somewhat in the post:

> Going back to the Wikipedia article, it seems the author of that edit wanted to write “swapping”, in the classic Unix V5 sense of swapping out whole processes as a consequence of scheduling. But the edit didn’t clarify that “swapping” was being used in an archaic sense that was likely to confuse the modern reader.

> context switches only the page table directory pointer

Swapping out the the PTD pointer is exactly what I'm thinking of. I'm wrong, because I didn't have the common colloquial meaning of "swapping" (paging out memory to disk) in my mind

I think it's a little strange such a meaning has come to dominate, at least in a classroom setting it is still fairly common to discuss the operation of the scheduler as "swapping pages".

nickelpro··on What Is PID 0?
The x86 version did, here's NYU's fork:

Scheduler: https://github.com/AnubisLMS/xv6/blob/main/kernel/proc.c#L26...

Page Table Swap: https://github.com/AnubisLMS/xv6/blob/main/kernel/vm.c#L151-...

nickelpro··on What Is PID 0?
PID 0 on most academic Unixs used for teaching operating system design still swap out the entire process, and call into the memory subsystem ("paging") to do so.

Linux is not the owner of the concept of PID 0. Saying that PID 0 frequently is involved with paging in and out memory is not incorrect.

nickelpro··on Run0, a systemd based alternative to sudo, announced
None, it's a nonsense "hack"
nickelpro··on Stop Acting Like You're Famous
I don't understand the difference in what I and the OP are talking about.

The OP said, "In the end, find something you enjoy doing and just do it because you enjoy it. If you have to, make some goals for yourself, but never for your 'audience'"

Presenting to the audience is my goal. If you're telling jokes, your goal is to make your "audience" (friends/coworkers/family/etc) laugh. The only way you can get funnier is to try jokes out and see what gets a response, the measure of quality is external, not internal.

I don't tell jokes to myself in an empty room. I don't take photos just to look at them myself in Lightroom. I don't write open source code just to put it on a thumbdrive and throw it in the bin.

None of the activities I, personally, enjoy work without an audience. The audience is entirely the point. Without the audience I wouldn't enjoy them.

Saying "do things for yourself without an audience", period, without caveat, is wrong for tons of stuff.

nickelpro··on Stop Acting Like You're Famous
Hard disagree with this as a polemic

When I take photographs of my friends, it is incredibly important to me that they be in focus, sharp, with a good depth of field bokeh that brings out their face and presents an attractive image. I take a huge amount of pride when a photo I shot ends up as a profile picture or widely shared. That's a large part of why I take them, to share with others.

Got my current job because of my public code and the quality of articles I have written on subjects relevant to the employer. When I interviewed they largely skipped the technical parts and focused on cultural fit and the kinds of projects I wanted to be involved with, because my publicly demonstrated track record left no question about the quality of my work.

It is fine to have some activities you enjoy without perfectionism, but there is a world of advantages that can come from a focus on quality.

nickelpro··on Verified curl
All of this is nice but none of it stops the style of hack that happened to xz besides the fact that it's very unlikely Dan is going to be bullied into handing over maintainership to someone else.

Every other element on the list can be attacked if the maintainer themselves is the malicious party carrying out the attack and it is being performed with the level of sophistication in the xz attack.

So ya, I'm not saying it's security theater in all contexts, but if the context is the attack vector used in the xz attack, it's security theater.

nickelpro··on Autoconf makes me think we stopped evolving too soon
vcpkg, conan, and Fetch_Content fix all the issues you discuss in that blog with the added benefit of not being insane. You dismiss package managers out of hand and you shouldn't.
nickelpro··on Google suspends romance author's account for writing sexually explicit content
Google's policy doesn't say anything about using their services to create sexually explicit content, only using their services for distribution.

> Do not distribute content that contains sexually explicit material, such as nudity, graphic sex acts, and pornographic material. This includes driving traffic to commercial pornography sites. We allow nudity for educational, documentary, scientific, or artistic purposes.

This squares with what happened to the authors here, who were only subject to enforcement actions when they shared the content via Google Docs.

But yes I think the basic idea of being unable to "trust" a free service which has no contractual obligations to you as a user is a given under any circumstances.

nickelpro··on Google suspends romance author's account for writing sexually explicit content
Things are very unclear. The author claims to have been able to download their own work prior to filing the appeal, so perhaps that's exactly what happened.

We're following a bread crumb trail of discord screenshots and instagram posts, no one is being straightforward about what is going on.

Google of course is a black box about enforcement actions on the foundation that transparency benefits spammers and doesn't help their bottom line, which is likely also true.

nickelpro··on Google suspends romance author's account for writing sexually explicit content
Yes, both things are reasonable, neither makes the other untrue.
nickelpro··on Google suspends romance author's account for writing sexually explicit content
Misleading title, none of the authors involved are claiming they're locked out of the accounts. The exact quote from the OP's mastadon is "An author lost access to her explicit writing when she shared it with alpha and beta readers."

Google says you may not use its services to distribute sexually explicit content. This is a very common and reasonable policy. The authors violated that rule, and while none of them are being straightforward about exactly what happened what seems to be the case is the violating content has been removed/locked.

nickelpro··on The Reddits
It's only about 5% of users, which is sizable but not unkillable by any means. The third-party mobile apps had way, way more than that and they proved very killable.
nickelpro··on Importance of Open-Source EDA Tools for Academia
Which of those "projects" do you imagine is an EDA tool? Cocotb is at least a tool, not a newsletter or a mailing list, and used in the hardware space, it probably comes the closest.
nickelpro··on Importance of Open-Source EDA Tools for Academia
A classic divide in European and American approaches to open source. Europe signs a petition and loudly declares a need, while the US has quietly funded all the major open source work in the field since 2018 [1]

[1]: https://theopenroadproject.org/

nickelpro··on What Luddites can teach us about resisting an automated future
There's no distinction, besides time period and the particular piece of automation at issue. Classical luddites hated looms putting textile workers out of work and modern luddites hate calculators putting accountants out of work.

The luddites of the early 19th century didn't argue for profit sharing or a protected monopoly over the textile industry. They didn't argue for job placement or pensions.

They said the "obnoxious" and "offensive" machines needed to be destroyed. They engaged in many other, more common labor practices, but those are not distinct. What makes luddism a distinctive practice, the ideology of the 19th century luddites that separated them from their contemporaries in other industries, is machine destruction.

If neo-luddism is in anyway separated from "classical" luddism in more than time period and devices at issue it's that the neo-luddites want to destroy new technology for more diverse reasons than merely job security.

No one calls themselves neo-luddites, but the groups the label is attached to often take issue with things like GMO foods not because a disease-free tomato put them out of work, but because they have a deep personal phobia of disease-free tomatoes. So there's some evolution there maybe but not in a way that I think is worth much meditation. They're still basically irrational, anti-progress, self-serving destructive forces. In other words, a boogeyman, a joke.

nickelpro··on What Luddites can teach us about resisting an automated future
It's not interesting! If you argue I should light my pants on fire, or shoot myself out of a canon, or dunk my head in tar bucket, what am I supposed to respond with?

"That's nuts!"

It's not a lack of curiosity, the position the opponent is arguing doesn't have legs and that's readily apparent.

"Renounce your sinful ways of sewing machines and egg beaters! RETVRN TO TRADITION!"

Like, what? That doesn't deserve a dignified response, there's no argument there. If anything, engaging grants it a kind of credibility that is more harmful than just saying "Move along sweetie, leave the crazy man be".

nickelpro··on What Luddites can teach us about resisting an automated future
You're arguing for a social safety net, that's in no way what luddism means or advocates for. This is why it is a parody.

Luddism doesn't argue for a social safety net, that people displaced by advances in automation and technology should be taken care of by the state. It argues that we shouldn't need a social safety net to take care of such laborers, because we should prevent automation from displacing them in the first place. This is exactly what the OP argues here.

Could the luddites have been satisfied by such an arrangement? Maybe, but that's not what they did. They threw clogs in the looms, and luddism as an ideology describes that mechanism and no other.

That particular nature is what makes luddism distinct from other labor movements. Luddism is not a synonym for "labor movement", it is a very specific set of prescriptions, and a laughable set of prescriptions at that.

nickelpro··on What Luddites can teach us about resisting an automated future
Sure, as a debate strategy doing the "Yes" meme is interesting, but only if you mean to gain something by it.

When the Republicans call Bernie a socialist, and Bernie says, "yes, and?" that's because Bernie is a socialist, and he's defusing the nature of the charge. Socialism is a valid, defensible position. Being gay is valid. Opposing a North Korean dictatorship is valid.

If I call you a luddite, and you say "yes", then I say "great you're a crazy person, bring in the men with the white coats." You didn't win anything, luddism isn't defensible. Arguing that we should regress to a agrarian society with a "restriction of automation to only the most dangerous forms of work" isn't sane or defensible or interesting, it's just crazy person talk.

nickelpro··on What Luddites can teach us about resisting an automated future
> It's not anti-progress to say that automation must not come at the expense of the lives of the people it is replacing.

Yes it is, we shouldn't deny 10000 people shirts or cars or fresh bread to save 100 jobs, but that's almost besides the point. This piece is arguing a far more straightforward and hilarious version of Luddism than you are.

The piece argues that military drones and homeless-harassing robotics are bad not because of their nature, but because they are taking human jobs. It advocates for "restriction of automation to only the most dangerous forms of work" (hope you like churning your own butter). It is such a ridiculous piece of work that it almost seems like a false flag, a piece intentionally made to portray "moderate" luddism like the degrowth people (already extremists) seem even more insane than they already are.

← PreviousPage 7 of 21Next →