HNHacker News
TopNewBestAskShowJobs

davidmr

2,328 karma · joined April 1, 2012

David Ressman hpc, linux, and all things i/o nerd davidr@ressman.org / https://blog.ressman.org
submissionscomments
davidmr··on Org-Mode for Visual Studio Code
It's easier said than done. I decided to switch about a month ago, and after playing around with evil mode for a week, I gave it up in favor of emacs-std bindings. The core editor evil emulation was flawless. All my 20-year muscle memory commands just worked without a second thought.

When going into other packages like org, magit, etc. I noticed that all their default bindings were emacsey as well (for obvious reasons). I realized that I would be forever mapping evil bindings into new packages every time I wanted to use something new, so I gave it up as a bad job and just bit the bullet and switched to stock emacs without evil. I'm like halfway through the learning curve (I hope!) and it's tough. I've always had the caps lock key mapped to Ctrl anyway, but I still feel like my pinkie is about to fall off. After using vi/elvis/vim for almost 25 years, I do admit that I like emacs, but "just switch" still means a period of remarkably reduced productivity to many.

(Yes, I know that for any popular package someone has published evil mode bindings and use-package makes them trivial, but it's not the point.)

davidmr··on Org-Mode for Visual Studio Code
That's a statement I hear a lot. A good friend wrote his physics dissertation entirely in org mode because of the TeX integration. You can even bring in the custom TeX templates every University's thesis publishing offices unaccountably seem to have.
davidmr··on Declassified documents show three Mexican presidents worked for the CIA
This has actually been declassified for quite a while[0].

Coincidentally, I started reading Narcoland: The Mexican Drug Lords And Their Godfathers by Anabel Hernández last week, and although the writing is a bit scattershot, it’s quite an eye-opener. I’m only 1/4 through it, but so far, the level of detail is shocking about precisely how deep rooted the corruption is (or at least was until the mid nineties, which is where I’m at now) inside the Mexican government and simultaneously how invasive the interference is from the US government.

[0] https://nsarchive2.gwu.edu/NSAEBB/NSAEBB204/

davidmr··on Why People Dislike Really Smart Leaders
I’m glad someone else felt the need to ask. I did work on a computer with 480V into the rack and the electrical training they made us sit through scared the bejesus out of all of us. It was basically 4 hours of watching people get electrocuted with 480v+. A 480V arc flash is no joke[0]. The training had the desired effect: we played it absolutely by the book, the CPR-trained buddy system, the PPE, everything.

[0] https://youtu.be/-iClXrd50Z8 the helpful “ding” is the best part. At any rate, imagine take that on your arms or face.

davidmr··on Ex-CIA Officer Suspected of Compromising Chinese Informants Is Arrested
A lot of the Manhattan Project scientists were naturalized citizens, including many of the greats: Enrico Fermi, Leo Szilard, Hans Bethe, James Franck, etc.
davidmr··on Ask HN: What influenced your personal growth the most?
Have you read any of the other Odyssey translations? I'm very curious Wilson's translation. I read the Fagles, Latimore, and Fitzgerald translations in college and did a few weeks of translation myself in our Greek class (I went to one of those hippy-dippy "Great Books" schools).

I think I was in the minority, but I found the Fagles translation to be far and away the best. I definitely subscribe more to the school of poetic translation than literal translation, so I was intrigued by the Wilson translation. What were your thoughts?

davidmr··on German Engineering Yields New Warship That Isn’t Fit for Sea
Do you think that missile batteries on eight ships are what’s stopping Russia from invading Germany?
davidmr··on German Engineering Yields New Warship That Isn’t Fit for Sea
Okay, I get that the computers don’t work right and there are some other engineering-specific problems of varying severity, but the primary objection from the WSJ’s point seems to be that it isn’t designed to hold its own in a shooting war with Russia. (And secondarily that Germany’s defense industry is no longer capable of building complicated programs like this.)

I know next to nothing about strategic planning, but it doesn’t seem that any missile battery whatever would be a better deterrent to Russia than the great big German flag flying on the top of the thing. How much effort should Germany really be trying to put into challenging the Russians?

davidmr··on The Sloan Digital Sky Survey
I worked with probably a dozen or so scientists and engineers who were building the SDSS telescope and the associated data analysis infrastructure (full disclosure: my role was very nearly entirely nothing compared to everyone else).

It's really a technological marvel, and some truly brilliant engineers and scientists were working on it. I'm intensely gratified that the survey has been such a smashing success; they all deserved it.

The part I was always most impressed with was they way they took the spectra. As mentioned in the article, the survey was twofold: the image survey over the vast swaths of the sky and somewhere around 2,000,000 spectra taken of distant objects.

Every single one of those spectra was taken by use of a fiber optic cable at the focus point of the telescope. They would take a metal plate, drill holes in it where the objects would appear, and plug the holes by hand with the cables: http://blog.sdss.org/wp-content/uploads/2014/05/twitter5.jpg

2 million times!

davidmr··on Ask HN: What did you work on in 2017?
I’m very glad they have such a good support network. A close friend of mine got a job out of law school prosecuting child sex crimes, and there was absolutely no support. After a year, he couldn’t take it any more and left public service altogether. Even a decade later, you can tell he’s still bothered by it. I have an incredible amount of respect for people who can do that work; I’m absolutely certain that I don’t have what it takes.
davidmr··on Fedora Modularity
I don’t believe this page has been updated to note that it won’t be released in F27 as hoped due to systemic problems in the beta: https://communityblog.fedoraproject.org/modularity-dead-long...

I’m looking forward to seeing what they’re going to do with it next; this is a super tough problem to get right in a supportable way. I have a lot of faith in the Fedora team to put together something awesome.

davidmr··on Uber Accused of Espionage, Bribery, Hacking, in Former Employee's Letter
It’s pretty rare, but it does happen every now and then. Arthur Andersen was convicted of criminal obstruction of justice. Because their entire business model was predicated on being trustworthy, they imploded really quickly. (Ironically the damage was mostly done before their conviction was overturned, but they were still guilty as hell.)

For Uber, since their business model seems to be completely unrelated to their inherent untrustworthiness, it’s unlikely that a criminal conviction would have such dramatic consequences. The FBI doesn’t fuck around with bribing foreign officials though, so it wouldn’t surprise me at all to see someone go to the pokey over that.

You never know, though. That might be enough to get local governments to ban them.

Also some states can apparently sue to have corporations dissolved. https://www.wired.com/story/equifax-deserves-the-corporate-d...

davidmr··on The Bitcoin Whales: 1,000 People Who Own 40 Percent of the Market
Is there any currency or commodity or derivative for which that statement is not true? It seems self-evident that if you own a lot of something like gold, dollars, BTC, sterling, etc. and you sell it, its value relative to what you're selling it for declines.
davidmr··on US government approves 'killer' mosquitoes to fight disease
When’s the last time anyone died of smallpox? As a species, we seem demonstrably better off for having arrogantly removed it from the planet.
davidmr··on New Research on How Ride-Hailing Impacts Travel Behavior
The cities included in the study are Boston, Chicago, Los Angeles, New York, San Francisco/Bay Area, Seattle, and Washington, D.C.

I haven't really spent huge amounts of time in a couple of these cities, but I wonder how uniformly over the cities the data factored into the decision. My impression is that NYC, Chicago, and DC have much more comprehensive public transit systems than do the others in the list (except Boston, about which I know next to nothing.)

Is my impression just wrong? Public Transit just didn't seem to be nearly as big a thing in LA, SF, or Seattle in the time I spent there.

davidmr··on NTSB: Tesla’s Autopilot UX a “major role” in fatal Model S crash
The NTSB and most of other highly developed nations ' equivalents really are quite amazing. For whatever reason, I read most of the full NTSB aviation incident reports. Aside from the thoroughness of them and the engineering geekery, they are truly master classes in understanding how to do a proper postmortem.

The NTSB's ability to remain apolitical and engineering-focused is really remarkable. They way they were portrayed in "Sully" is shameful and was uncalled for.

davidmr··on What's So Bad About Posix I/O?
Rather "into an I/O-bound problem". Apologies for ruining Ken's great joke.
davidmr··on What's So Bad About Posix I/O?
They do, more or less. It's called MPI-IO, but it's still tough.
davidmr··on What's So Bad About Posix I/O?
I don't think so; HPC is a different beast There are very few if any organizations on earth that have a database that can write a sustained 1TByte/second+ from as few as 300-400 node to 100s of PB of stateful media for years on end.

Ken Batcher (maybe of OSU?) wrote a quote that the rest of us have been using for years: "Supercomputers are a tool for converting a CPU-bound problem into a HPC-bound problem."

The filesystems start to look more like databases over time, but it's not like they can throw down a nice Cassandra cluster and have it pick up the slack. I'm not saying it will never happen, but I don't think it's am option at the moment.

davidmr··on What's So Bad About Posix I/O?
I’ve spent most of my career specifically dealing with HPC I/O from a systems and applications perspective. I worked in academia for a while, then in the DOE (almost indistinguishable from academia except worse bureaucracy; shocking, I know), and finally in private industry.

I don’t think I’ve ever read a more spot-on description of the problem. Once I left the DOE, I realized that the problem was far more acute than I had thought. At least on the real big systems, most of our work was with a small number of research groups basically doing the same workflow: start your job, read in some data, crunch, every 15 minutes or something slam the entire contents of system memory (100s of TB) out to spinning disk, crunch, slam, your job gets killed when your time slice expires, get scheduled again, load up the last checkpoint, rinse, repeat. Because of the sheer amount of data, it’s an interesting problem, but you could generally work with the researchers to impose good I/O behavior that gets around the POSIX constraints peculiarities of the particular filesystems. You want 100,000,000 cpu hours on a $200M computer? You can do the work to make the filesystem writes easier on the system.

Coming into private industry was a real eye-opener. You’re in-house staff and you don’t get to say who can use the computer. People use the filesystems for IPC, store 100M files of 200B each, read() and write() terabytes of data 1B at a time, you name it. If I had $100 for every job in which I saw 10,000 cores running a stat() in a while loop waiting for some data to get written to it by one process that had long since died, I’d be retired on a beach somewhere.

The problem with POSIX I/O is that it’s so, so easy and it almost always works when you expect it to. GPFS (what I’m most familiar with) is amazing at enforcing the consistency. I’ve seen parallel filesystems and disk break in every imaginable way and in a lot of ways that aren’t, but I’ve never seen GPFS present data inconsistently across time where some write call was finished and it’s data didn’t show up to a read() started after the write got its lock or a situation where some process opened a file after the unlink was acknowledged. For a developer who hasn’t ever worked with parallel computing and whose boss just wants them to make it work, the filesystems is an amazing tool. I honestly can’t blame a developer who makes it work for 1000 cores and then gets upset with me when it blows up at 1500. I get grouchy with them, but I don’t blame them. (There’s a difference!)

But as the filesystems get bigger, the amount of work the filesystems have to do to maintain that consistency isn’t scaling. The amount of lock traffic flying back and forth between all the nodes is a lot of complexity to keep up with, and if you have the tiniest issue with your network even on some edge somewhere, you’re going to have a really unpleasant day.

One of the things that GCE and AWS have done so well is to just abandon the concept of the shared POSIX filesystem, and produce good tooling to help people deal with the IPC and workflow data processing without it. It’s a hell of a lot of work to go from an on-site HPC environment to GCE though. There’s a ton of money to be made for someone who can make that transition easier and cheaper (if you’ve got it figured out, you know, call me. I want to on it!), but people have sunk so much money into their parallel filesystems and disk that it’s a tough ask for the C-suite. Hypothetically speaking, someone I know really well who’s a lot like me was recently leading a project to do exactly this that got shut down basically because they couldn’t prove it would be cheaper in 3 years.

davidmr··on Silicon Valley is erasing individuality
Wildly off-tooic, but I recently visited Amsterdam for the first time and was absolutely astonished at the lack of curtains. The jury seems to be out on whether it's a vestige of Dutch Calvinism or if it's just somehow cultural, but it was fascinating.
davidmr··on Using GNU Stow to manage dotfiles (2013)
I was going to say that I do something similar with mine[1], but I looked at your makefile and saw that it's about 10x more sophisticated than mine, although I've never really had sufficient problems to warrant learning the makefile black magic. I think I'm now thankful that my dotfile needs are so basic!

1. https://github.com/davidr/dotfiles

davidmr··on Google Critic Ousted from Think Tank Funded by the Tech Giant
I absolutely do. I probably would have worded it differently.
davidmr··on Google Critic Ousted from Think Tank Funded by the Tech Giant
The comma in the second element allows—I would even say requires—one to separate elements of a list with semicolons. I don’t have a copy of the NYT Style Guide, but I just consulted my Chicago Style guide, and it is explicitly permitted.

Grammarians have spoken!

davidmr··on Opioid makers made payments to one in 12 U.S. doctors
I'm not sure I understand what you're trying to say. Is it your contention that, if anyone could get opiates without any regulation or prescription, fewer people would take them than they do now? Why would scrutiny of any kind by anyone lead to reduced usage in a case like this if that scrutiny had no regulatory effect on who could obtain narcotic painkillers?

I understand (though disagree with) the libertarian philosophy that you should be free to take anything you please, but I fail to see how such a situation would lead to decreased use.

davidmr··on Euclid's Elements (1997)
I spent a year in a "Great Books" college, where there are no textbooks, no lectures, etc., just primary sources progressing roughly chronologically.

For first year math, you go through Elements almost entirely front-to-back. I have mixed feelings about the Great Books programs in general, but the Euclid class was remarkable. It tends not to be a math-heavy group of students, but even those who think they're bad at math can still follow (for the most part) the geometric proofs. It gave all of the students--those of us who were mathematically inclined and those who weren't--a shared vocabulary and methodology for talking about actual mathematics in a way that any of the other math classes I took later after transferring to another university would not have.

If anyone wants to go through them by yourself, do yourself a favor and get the Heath-annotated copies. Those annotations can be a life-saver when you get stuck.

davidmr··on Is Handwriting History?
As others mentioned, that's certainly not my experience, but the tactile nature of taking noted by hand still helps me remember better.

I've been half-heartedly considering learning shorthand. I still like taking notes by hand, but I often realllllly wish I had a mostly verbatim record of some meetings.

davidmr··on The System Bottleneck Shifts to PCI-Express
I certainly don't have much to add on any of the technical discussion here about the host side of things, but speaking as someone who in a previous life had to run miles and miles of IB cable under the floors, rip out mysteriously broken cables and run their replacements back in, these high-capacity ports are a godsend for switch interconnects. I can use more ports on my edge switches and run way fewer shitty $1500 Mellanox ISL cables if I just want my hosts running FDR IB.
davidmr··on Safety lapses hobble Los Alamos National Lab’s work on U.S. nuclear warheads
As a former DOE lab employee, I can't even begin to imagine the hassle of changing operating contracts, although LANL has really been putting forth extraordinary effort to get the DOE to try.

What's most irritating is that the DOE use a broad brush and go overboard where it's easy and clearly don't do enough where it's hard. I had to attend ladder training before I was allowed to climb down under the 4' floor in my computer room and the rule was made that we had to use ladders all the time. You can't even fit the ladders in the tile holes, but the rule was on the books because it was "safe!". There were so many of those boneheaded rules, but these boneheads are allowed near plutonium to photograph it for no scientific purpose? Grrrrr

davidmr··on TensorFlow-World: Simple and ready-to-use tutorials for TensorFlow
Unfortunately not. The docs on the Dataset API are quite good though.
← PreviousPage 5 of 10Next →