Hacker News Reader

zackangeloon Alternative clouds are booming as companies seek cheaper access to GPUs

Yes we’re running our own inference server because we do nonstandard things in the decoder loop (think logit masking, other types of sampling, etc).

zackangeloon Alternative clouds are booming as companies seek cheaper access to GPUs

Would like to take a moment to recommend fly.io for GPU workloads.

I've been building a prototype using them for the last couple of weeks and it's been great to use. I didn't have to jump through any hoops or apply for any quota adjustments to get started. And I especially appreciate how easy they make it to automatically scale your GPU instances to zero based on traffic.

zackangeloon Gradient AI Releases 1M Context Llama 3 8B

The table at the bottom says they initialized the 65K version from "LLaMA-3 7B"? (Assuming the 7B is a typo and they meant 8B.)

And each successive version with a larger window was initialized on the previous smaller one (65K -> 262K -> 524k -> 1048k).

zackangeloon SableDb – Fast, persistent database supporting Redis API

If anyone is interested in a Rust-based on-disk KV store, I've come across sled[0] a few times, seems interesting. The author's also built a lot of other cool concurrency primitives for Rust as well.

[0] https://github.com/spacejam/sled

zackangeloon Intel One Mono

I like Berkeley Mono [0].

[0] https://berkeleygraphics.com/typefaces/berkeley-mono

zackangeloon The new science on what ultra-processed food does to the brain

He's referring to Robert Lustig, in case anyone else was curious.

zackangeloon Ex-Google engineer charged with stealing trade secrets

That is satire.

zackangeloon Expert sounds alarm on new wave of US opioids crisis

Prescription drugs are dispensed by a pharmacist for a reason. In the US, they are legally required to report any unusually large, frequent or suspicious orders to the government. Many pharmacies were found to have neglected that responsibility so they could sell more drugs.

zackangeloon Common Rust Lifetime Misconceptions (2020)

Yes but Send + Sync is required everywhere regardless.

zackangeloon Writing a Compiler is Surprisingly Easy (part 1)

My first introduction to this subject was “Let’s Build a Compiler” by Jack Crenshaw. Even though it’s quite old now, I still highly recommend if you’re looking for a gentle, no-frills introduction: https://compilers.iecc.com/crenshaw/

zackangeloon The TextFX Project

Don't miss the YouTube video with Lupe Fiasco [0]. The cynic in me says that this might be all marketing; I would be curious to know if he's actually using LLMs regularly to write songs.

So many people think AI will entirely replace artists, writers, creatives, etc. but I increasingly believe that (for the foreseeable future) we're just giving these folks a new set of superior tools. Taste still matters. Being able to ask the right questions still matters.

[0] https://www.youtube.com/watch?v=yYp18JAvKkQ

zackangeloon Attention Is Off By One

He’s not trying or claiming to improve attention. He’s trying to reduce outliers to improve the ability to quantize the parameters.

zackangeloon IntelliJ Rust

I have a similar experience when using rust-analyzer and copilot in vscode.

zackangeloon Why SQLite is so great for the edge

I think the read-only use case you mention is a compelling one. I’m working on a developer product that makes it easy to prematerialize data in the way you describe.

It works by consuming data out of upstream APIs and then publishes a new, faster version of that API in edge locations. The underlying data there is stored in SQLite (plus some custom bits), but that’s just an implementation detail.

zackangeloon Deep sleep may be the best defense against Alzheimer’s

The Eight mattress will measure sleep stages without you having to wear anything.

zackangeloon Uber attempting to reduce office space in SF / 31% of office leases open

Every time stories like this get posted, the first question is typically "when will they convert these buildings to residential?". And, like clockwork, there's always someone there to point out that residential conversion is prohibitively expensive and it would be cheaper to just tear the building down and start over.

My question is: is it possible to design high rise buildings in the future that can flex between residential and commercial? What's the extra upfront cost in plumbing, etc to build in this optionality from the start?

zackangeloon Study finds elevated levels of toxic metals in some juices and soft drinks

They’ve since reduced PFAS levels by half but still not great: https://www.consumerreports.org/bottled-water/topo-chico-cut...

zackangeloon Lightweight SQLite Editor for Windows

This also brought back memories for me of writing Win32 apps. If you're looking for a good book, checkout Programming Windows by Charles Petzold. It really demystified a lot of things for me and I bet it's still relevant if you want to write stuff like this.

zackangeloon Show HN: IronBoy, a highly accurate GameBoy emulator written in Rust, runs WASM

"runs WASM" made me wonder: is transpiling a ROM to WASM an effective strategy for emulating something like a GameBoy? You'd still have many issues to solve, not least of which mapping the hardware onto a runtime layer, but you'd benefit from all of the innovation happening in the WASM ecosystem.

zackangeloon Spotting and avoiding heap fragmentation in Rust applications

If you're doing memory-intensive tasks that are clearly bounded (e.g., handling an http request), I wonder if it's worth looking at something like an arena allocator [0] that can free the entire task's memory at once.

[0] https://docs.rs/bumpalo/latest/bumpalo/

zackangeloon Japan skiers’ smartphones making unnecessary emergency calls

I stopped wearing my Apple Watch while surfing because this feature would trigger almost every time. It’s made worse by the fact that the touch screen is inoperable when the phone senses that it’s wet, so it’s difficult to cancel.

zackangeloon What should I know about garbage collection as a Java developer?

Most JVMs take advantage of a thread local "bump" allocator[1] as well to avoid having to cross JVM or kernel boundaries to allocate memory, which can result in huge speedups for memory-intensive use cases.

[1] https://shipilev.net/jvm/anatomy-quarks/4-tlab-allocation/

zackangeloon Data-driven performance optimization with Rust and Miri

An often overlooked option for profiling Rust is Apple's Instruments.app. It's amazing and usually the first thing I reach for when I need a profiler on Mac OS X.

zackangeloon Deno raises $21M

Surprised this is the case because I use their Rust v8 bindings on an ARM laptop almost every day. Have you tried building it from source?

zackangeloon Memray: a memory profiler for Python

Do you mean specifically for Python? There's this for async Rust:

https://tokio.rs/blog/2021-12-announcing-tokio-console

zackangeloon Scala at scale at Databricks

People often compare them because they're the two best supported languages for interacting with Spark (and its ecosystem).

zackangeloon Fleet, a Lightweight IDE from JetBrains

Have you tried using rust-analyzer with VS Code? I find it to be a little rough around the edges but mostly pretty great. I used the Rust plugin for IntelliJ a couple of years ago and remembered being disappointed.