HNHacker News
TopNewBestAskShowJobs

jpgvm

10,011 karma · joined March 14, 2013

web: jpg.dev email: jpg@jpg.id.au twitter: @jpgvm github: josephglanville

Principal Software Engineer @ NVIDIA

Opinions are mine and not that of my employer, etc.

submissionscomments
jpgvm··on Docker Model Runner
In our case some machines would need to access less than 1% of the image size but being able to have an image with the entire model weights as a single artifact is an important feature in and of itself. In our specific scenario even if eStargz would be slow by filesystem standards it's competing with network transfer anyway so if it's the same order of magnitude as rsync that will do.

I don't have any perf numbers I can share but I can say we see ~30% compression with eStargz which is already a small win atleast heh.

jpgvm··on Docker Model Runner
Not too hard. If you happen to be on CRI-O this has been implemented for a bit but if you are like us and on containerd then you need the new 2.1 beta release. That does most of the heavy lifting, implementing a CSI driver that mounted these as PVs wouldn't be super hard I don't think and you could borrow liberally from the volume source implementation.
jpgvm··on Docker Model Runner
That is exactly the feature we are using, right now you need to be on a beta release of containerd but before long it should be pretty widespread. In combination with lazy pull (eStargz) it's a pretty compelling implementation.
jpgvm··on Docker Model Runner
I'm looking at using OCI at $DAY_JOB for model distribution for fleets of machines also so it's good to see it's getting some traction elsewhere.

OCI has some benefits over other systems, namely that tiered caching/pull-through is already pretty battle-tested as is signing etc, beating more naive distribution methods for reliability, performance and trust.

If combined with eStargz or zstd::chunked it's also pretty nice for distributed systems as long as you can slice things up into files in such a way that not every machine needs to pull the full model weights.

Failing that there are P2P distribution mechanisms for OCI (Dragonfly etc) that can lessen the burden without resorting to DIY on Bittorrent or similar.

jpgvm··on Pentagon to terminate $5.1B in IT contracts with Accenture, Deloitte
Finally they actually cut something that should be cut.
jpgvm··on A decision to eject from a failing F-35B fighter and the betrayal in its wake
Even if he held the glide without visibility (harder than it sounds) he would have had less than 60s before eating dirt (rate of descent was ~800ft/min).
jpgvm··on A decision to eject from a failing F-35B fighter and the betrayal in its wake
Losing all instruments with no visibility is still ejectable even if he thought the engine was still running. He was disorientated and relying on his instruments, when flying under IFR (instrument flight rules) loss of instruments is tantamount to loss of control. The likely outcome in those situations is controlled flight into terrain at 350+mph.

With low altitude being an aggravating factor he was always 100% correct in ejecting and whatever the plane did afterwards is largely irrelevant.

jpgvm··on Crabtime: Zig’s Comptime in Rust
This is cursed in the most wonderful way, kudos.
jpgvm··on Monster Cables picked the wrong guy to threaten (2008)
I wish HDMI would die as a standard. The TV folk won't that happen though, controlling HDMI allows them to control the ecosystem by extension.

Would be nice if the EU could step in an make DisplayPort the required connection and protocol like they did for the USB-C port for charging.

jpgvm··on The Pain That Is GitHub Actions
This is the way.

My personal favourite solution is Bazel specifically because it can be so isolated from those layers.

No need for Docker (or Docker in Docker as many of these solutions end up requiring) or other exotic stuff, can produce OCI image artifacts with `rules_oci` directly.

By requiring so little of the runner you really don't care for runner features, you can then restrict your CI/CD runner selection to just reliability, cost, performance and ease of integration.

jpgvm··on US Ends Support For Ukrainian F-16s
... and just like that, America cedes arms export leadership.

China will laugh all the way to the bank.

jpgvm··on Layoffs Don't Work
I find layoffs very context sensitive.

Atleast 2 of the fast growing companies I have been a part of have had serious layoffs that were very successful in cutting dead weight that had accumulated because of fast and loose hiring and poor middle management. So it can be done well.

By and large though they are a sign to jump from the soon to be sinking ship... so it's very important to know what kind it is and act accordingly.

jpgvm··on Tailscale is pretty useful
We recently did a pretty big rollout of Tailscale and tbh I am presently surprised with how well it works. Between subnet routing to our bare metal stuff and the Kubernetes operator, especially the ability to expose services to the Tailnet has been a big win.

I was a doubter a bit as to how it would work at a bigger org but so far rock solid, easy to setup and great user experience.

jpgvm··on Zelensky leaves White House after angry meeting
1. Is not about Ukraine. It's about the US and Europe. They benefit massively while Ukraine suffers the losses on their behalf.

Think about it. They support Ukraine by sending them surplus or aged weapon systems while modernising and funnelling money to their military industrial complex (with public support!). In return Ukraine uses those weapons systems to keep Russia tied up in a war, exposing Russia's most modern weapon systems and providing a huge amount of information on how to fight a drone war.

This is a great investment independent of what happens in or to Ukraine, basically they turn weapons they needed to replace or get rid of anyway into intel, new weapons, political brownie points at home and a weakened adversary.

China doesn't mind it that much either, though they enjoy it less because they had invested fairly heavily into Ukrainian infrastructure that is being destroyed. Keeping Russia isolated makes it easy for them to extra cheap oil & gas, push for pipelines that flow to China instead of towards Europe and also offload a bunch of their industrial surplus now that their requirements are winding down and the rest of the world is trying to tariff them for dumping.

So basically the problem is 1. is good for almost everyone except Ukraine and Russia. Trump for whatever reason can't understand that and just wants to throw a tantrum as is befitting for someone of his intelligence.

jpgvm··on Fire-Flyer File System from DeepSeek
Ahh I missed that. Yes prefix caching and RAG are 2 cases were you will want something like this during inference time.
jpgvm··on Fire-Flyer File System from DeepSeek
Well current popular systems are pretty much limited to Lustre and the new kid Weka, mostly Lustre though tbh.

You can try to use "standard" options like MinIO/Ceph(RADOS)/SeaweedFS but you will very quickly learn those systems aren't remotely fast enough for these usecases.

AI training is what this is used for, not inference (which has absolutely no need for any filesystem at all). What makes the workload somewhat special is that it's entirely random read and not cacheable at all as most reads are one and done.

Would Lustre be perfectly fine at 6TiB/s? Yes. Is it a huge pain in the ass to operate and make remotely highly available? Also yes. If this thing is capable of the throughput but easier to operate and generally more modern and less baroque it's probably an improvement. TLDR is Lustre is fast but that is literally it's only redeeming quality. I have lost far too many hours of my life to the Lustre gods.

jpgvm··on Fire-Flyer File System from DeepSeek
None of those are close to fast enough.

The only competitors in the parallel FS space that are useful for this are Lustre and Weka.

Otherwise if you don't need a single namespace a bunch of fat AF NFSv4 servers w/NFS over RDMA will also get you to 6TiB/s.

The "surefire" way though is still Lustre, it's the big daddy of distributed parallel filesystems still but it's an absolute beast to setup and operate.

jpgvm··on Plane Crashes Near Washington DC after midair collision with helicopter
Rough year+ for aviation at this point...
jpgvm··on Migrating Away from Bcachefs
A file system is an emotional investment of a decade+ so I can understand Kent's desire to defend it the way he does.

That said the kernel is the bigger picture, if he wants it to be successful and remain in mainline he needs to adapt to how Linus expects maintainers to operate. That probably means delaying getting fixes to mainline users until he can show they have been adequately tested. Until that trust is built bcachefs is always a few bad cycles away from being dropped from mainline.

jpgvm··on Garmin's –$40B Pivot
Margin isn't everything, it's a very stable business with a very wide technological and regulatory moat. Add to that fierce brand loyalty among pilots and it's a great business IMO.

Personal electronics (unless you are Apple basically) is a pretty fickle business and fortunes can change very rapidly. Aviation? Not so much, unless you chuck a Boeing.

jpgvm··on I'll think twice before using GitHub Actions again
Use Bazel.

GHA/Gitlab CI/Buildkite/whatever else floats your boat then just builds a bunch of Bazel targets, naively, in-order etc. Just lean on Bazel fine-grained caching until that isn't enough anymore and stir in remote build execution for more parallelism when you need it.

This works up until ~10M+ lines of code or ~10ish reasonably large services. After that you need to do a bit more work to only build graph of targets that have been changed by the diff. That will get you far enough that you will have a whole team that works on these problems.

Allowing the CI tools to do any orchestration or dictate how your projects are built is insanity. Expressing dependencies etc in YAML is is the path to darkness and is only really justifiable for very small projects.

jpgvm··on So you want to build your own data center
Netbox is crap unless you are trying to manage a small but very heterogeneous environment. For anything big, very homogeneous etc you really don't want it.

It feels more like an OSS tool for managing university campus scale infra, which is completely fine if that is the problem you have but for commercial scale infrastructure unfortunately there isn't a good OOTB DCIM option right now.

jpgvm··on Waymo achieves 92% reduction in bodily injury claims compared to human drivers
I'm not sure it's possible to solve for that with additional prowess.

If you are wronged by a machine you are always going to be emotional about it, i.e having your email account unceremoniously terminated by Google without recourse by an essentially automated process.

The thing to solve for is the lack of recourse. Very very difficult in American society given how skewed the relationship is between corporations and people (or the government for that matter).

If instead when a Waymo demonstratively caused an accident the payout was 10-100x higher than if a human was involved I suspect many peoples qualms would go away.

While I think just the "knowledge" (assuming Americans would accept facts...) that the driver is X% better than the best drivers on the roads would provide little comfort to most.

I think in societies outside of America these may be more successful actually. i.e most parts of Asia.

jpgvm··on Waymo achieves 92% reduction in bodily injury claims compared to human drivers
Why? We aren't looking at introducing autonomous ambulances any time soon.

They should be compared to taxi/Uber/Lyft drivers that is what is being replaced by this and as such where the net harm increase or reduction will occur.

jpgvm··on Nyxt: The Hacker's Browser
Not sure if anyone else remembers uzbl but I'm sad that didn't catch on. Will give this a go I guess.
jpgvm··on Llama-3.3-70B-Instruct
I suggest joining the Discord and enquiring there or depending on what you need emailing sales@.
jpgvm··on Llama-3.3-70B-Instruct
https://twitter.com/sundeep/status/1865094783234142240

Enjoy!

Disclaimer: I work at Groq.

jpgvm··on Kubernetes on Hetzner: cutting my infra bill by 75%
You really don't need all 10 people on-call to know k8s to that level. They just need to know enough as to when to wake someone else up.

Everywhere I have worked where we have run clusters in the 100s to 1000s of nodes we have rarely had a team larger than 4-5 of true k8s folks and even then it's been a split between folks that are very hardware provisioning/network/etc focused and more higher level k8s folk which also take on a large portion of CI/CD work also.

At smaller scale (in the $1M/yr ballpark) I have done all the k8s bare metal ops myself along with all CI/CD and been responsible for a ton of the backend programming too. This is feasible because with distros like Talos etc it doesn't take a lot of manpower once it's setup and upgrades aren't too painful at small scale if you aren't running stateful services.

So tbh no, you just need ideally 2 folks at around ~200k/yr each that are competent and have done it before. The rest of the folks on the on-call rotation are just the rest of your engineers (and if you are at $1m/yr cloud spend you have more than 10 of those).

jpgvm··on Kubernetes on Hetzner: cutting my infra bill by 75%
If you prefer no bullshit communications they are great. They are to the point, terse and very German. I find this both refreshing and exactly what I want/need out of support. The few times I have needed to contact them it's been HW related. One was a SSD that was clearly having issues even though SMART reported nothing wrong, I sent them blktrace output and they said yup that checks out, scheduled disk replacement right away. The other time was a network related problem with their transit, I had some ASNs that I was trying to talk to suddenly getting some pretty damn cursed paths and a big increase in latency as a result, they sorted out the path weights super fast and everything has been great since.

The only other time I have received better support was from Aussie ISPs. Back in the day when you called Internode the guy who answered the phone was a bona-fide network engineer and would go as far as getting a shell on the DSLAM to check out what is going on. To me that is peak support, live debugging of the problem!

Similarly I called into Aussie Broadband to do my first NBN setup, explained I did "BYO" modem because I was going to initiate the PPPoE session with my Linux router and they said no problem. She even offered to send me a cookie cutter pppd config along with the info to set it up myself. Easily the some of the most knowledgeable and "can do" attitude for first layer support I have encountered.

Needless to say when I encounter damn good support I stay even when it costs more.

jpgvm··on Kubernetes on Hetzner: cutting my infra bill by 75%
In my experience noone bothers unless they are using GPUs or they are already at 100k/mo.

I do think 100k/mo is the tipping point actually, that is $1.2M/yr.

It costs around $400k/yr in engineering salaries to reasonably support a sophisticated bare metal deployment (though such people can generally do that AND provide a lot of value elsewhere in the business, so really it's actual cost is lower than this) and about $100k/yr in DC commitments, HW amortisation, and BW roughly. So you save around $700k a year which is great but the benefit becomes much greater when your equiv cloud spend is even bigger than that.

← PreviousPage 5 of 34Next →