HNHacker News
TopNewBestAskShowJobs

jhgg

3,448 karma · joined November 4, 2013

currently: Lead of Supercomputing at SpaceXAI, building the biggest AI supercomputers in the world. https://grok.com, https://x.ai

formerly: Principal Engineer of "Core Technologies" at Discord. https://discord.com/

[ my public key: https://keybase.io/jhgg; my proof: https://keybase.io/jhgg/sigs/NfC1mAm4fGJgKsLzYcshaSlYAfvMnxgLUZgee3nWqzI ]

submissionscomments
jhgg··on How Discord stores trillions of messages (2023)
Nope. Didn't change the schema, mainly added read coalescing and used ICS. I think the big thing is when Scylla is processing a bunch of tombstones it's able to do so in a way that doesn't choke the whole server. Latest Scylla version also can send back partial/empty pages to the client to limit the amount of work per query that is run.
jhgg··on How Discord stores trillions of messages (2023)
There are ballpark of a few hundred million discord servers... do you really want to run that many Postgres instances? And even so what do you do about DM/GDMs? Easier to just run one big mega cluster for messages.
jhgg··on How Discord stores trillions of messages (2023)
Fwiw the benchmarked numbers are for writing very small rows. When doing the messages migration, with no read traffic, and the cluster/compaction settings tuned for writes we only managed approx 3m inserts/sec while fully saturating the Scylla cluster.
jhgg··on How Discord stores trillions of messages (2023)
Yes, we wanted to migrate all our data stores away from Cassandra due to stability and performance issues. Moving to something that didn't have those issues (or at least had a different set of less severe issues) while also not having to rewrite a bunch of code was a positive.
jhgg··on Discord Reduced WebSocket Traffic by 40%
One thing to note is that on a given gateway server there are potentially 100k other compression contexts active, and given each connection is transmitting a trickle of small data in an unpredictable way, from different CPU cores as the processes are scheduled by the erlang VM, chances are the CPU caches are absolutely being thrashed. I imagine this contributes to some level of fixed overhead here too, especially when you're measuring these timings on a machine serving actual production traffic as opposed to simply running a bunch of small payloads through a single compressor.
jhgg··on Discord Reduced WebSocket Traffic by 40%
It was a bug
jhgg··on Discord Reduced WebSocket Traffic by 40%
The 2mb case is pathological - an account on MANY servers with no local cache state (the READY payload works to only send data that's changed between when you've reconnected by having the client send hashes of data it knows.)
jhgg··on Discord Reduced WebSocket Traffic by 40%
I think one thing this blog post did not mention was the historical context of moving from uncompressed to compressed traffic (using zlib), something I worked on in 2017. IIRC, the bandwidth savings were massive (75%). It did use more server side CPU, and negligible client side CPU, so we went for it anyways as bandwidth is a very precious thing to optimize for especially with cloud bandwidth costs.

Either way the incremental improvements here are great - and it's important to consider optimization both from transport level (compression, encoding) and also from a protocol level (the messages actually sent over the wire.)

Also one thing not mentioned is client side decompression on desktop used to use a JS implementation of zlib (pako) to a native implementation, that's exposed to the client via napi bindings.

jhgg··on Mistral releases Pixtral 12B, its first multimodal model
Try out https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct
jhgg··on An unordered list of things I miss in Go
> Is Go adding a random seed to each hash map?

Yes: https://github.com/golang/go/blob/27093581b2828a2752a6d2711d...

jhgg··on An unordered list of things I miss in Go
https://go.dev/doc/go1#iteration

> The old language specification did not define the order of iteration for maps, and in practice it differed across hardware platforms. This caused tests that iterated over maps to be fragile and non-portable, with the unpleasant property that a test might always pass on one machine but break on another.

> In Go 1, the order in which elements are visited when iterating over a map using a for range statement is defined to be unpredictable, even if the same loop is run multiple times with the same map. Code should not assume that the elements are visited in any particular order.

> This change means that code that depends on iteration order is very likely to break early and be fixed long before it becomes a problem. Just as important, it allows the map implementation to ensure better map balancing even when programs are using range loops to select an element from a mapl.

jhgg··on Jepsen: Jetcd 0.8.2
Look at the code for the watcher client[1] and lease management[2].

[1]: https://github.com/etcd-io/etcd/blob/main/client/v3/watch.go

[2]: https://github.com/etcd-io/etcd/blob/main/client/v3/lease.go

jhgg··on Jepsen: Jetcd 0.8.2
I'm currently working on a Rust v3 client, and have been reading the Go v3 source code, and the code definitely is hard to follow so I would be unsurprised if there were issues lurking.
jhgg··on Structured Outputs in the API
Valid JSON, sure, but that key does not conform to the schema provided in the example. The LLM must generate valid JSON that also conforms to the provided schema.
jhgg··on Dumb website only lets one user in at a time
Cloudflare offers this as a service: https://www.cloudflare.com/application-services/products/wai...
jhgg··on Our great database migration
How exactly would writing this in elixir have avoided async pains? Can you elaborate.
jhgg··on OpenAI and Microsoft Azure to deprecate GPT-4 32K
Because the state of mind is derived from the input tokens.
jhgg··on How/Why to Sweep Async Tasks Under a Postgres Table [video]
Set a reasonable maximum for your auto-scaler to avoid this issue.
jhgg··on Discord Applying Forced Arbitration - opt-out before it is too late!
FYI, the forced arbitration clause has existed in the ToS since 2018. This is not new.

Blog post from 2018: https://discord.com/safety/terms-of-service-feedback-and-cha... Archived version of ToS from may 2020: https://discord.com/terms/terms-of-service-may-2020

jhgg··on You receive a call on your phone. The caller says they're from your bank
Because it's significantly easier to just ask the phone to make a call, than it is to build VOIP calls into the app.

As well, you get a lot of things for free:

- user is familiar with the phone UI, and knows how to mute/use speaker phone.

- put the call on hold if another incoming call comes in,

- able to call someone and make it into a conference call, - more reliable on shitty internet.

jhgg··on You receive a call on your phone. The caller says they're from your bank
Large banks like chase are motivated to make social engineering attacks like this more difficult. However, they need to do so while also considering the friction and potential frustration that additional security/verification measures might cause for their customers.

One idea that would mitigate against this, while also not imposing significant burden to the customer is to have the procedure be that the app simply instructs you to hang up your call, and then after dials a randomly selected number from a large pool of numbers the bank controls, that has been wired up to (given your phone number) connect you back to the bank and directly to the agent that has called you. If your phone number dials any other number from the pool during this time (the attacker is trying to race you to call back the bank while spoofing your number), then the account flagged as being potentially under attack.

jhgg··on The V8 Sandbox
V8 isolate termination is more akin to throwing an uncatchable exception, versus killing a thread abruptly. When you ask v8 to terminate an isolate, it does take some time for it to terminate depending on what code is running.
jhgg··on Tell HN: GitHub is blocking search unless you are logged in
So register a throwaway for work and your phone. Seems easily solvable in a few seconds!
jhgg··on Cyclist hit by driverless Waymo car in San Francisco, police say
Recaptcha has not been used to train any neural network to detect anything in a very long time.

In fact, visual challenges like this are a mostly solved problem, and the real magic is the classification that happens behind the scenes.

jhgg··on Ask HN: Is Apple's R1 a Discrete GPU?
Saying that the input latency is 12ms does not mean that it can only update once every 12ms. It just says that there's a 12ms delay from what happens in the real world to what your eyes see through the device.
jhgg··on Self-hosting a high-availability Postgres cluster on Kubernetes
I would consider our scale pretty large here, and it works just fine.
jhgg··on Self-hosting a high-availability Postgres cluster on Kubernetes
You must be. It is very easy. We just let kubernetes scheduler take care of everything. We use anti affinities to disallow scheduling nodes from the same ES cluster on the same K8S node. We use the ES operator which will apply the appropriate PDBs to ensure that operations done to the k8s cluster as a whole doesn't end up turning off too many things, and threatening availability.

Things literally have just worked.

We are about to do a scale up operation of about 6 nodes each for 16 clusters (total of 96 ES data nodes being added.) This was editing a variable in a config file, and pushing that config to kubernetes. The operation will probably take a day to complete as we throttle the speed at which data transfers to new nodes to not adversely affect latency or ingest rate.

The amount of "human time" to kick off this operation and begin the scale up is measured in minutes, then it's just letting it do its thing.

jhgg··on Self-hosting a high-availability Postgres cluster on Kubernetes
On the other hand, at a certain scale (running hundreds of ES nodes across 80 or so ES clusters), Kubernetes actually does make a lot of sense.

At work, we moved from hosting elastic search on bare VMs to kubernetes. By leveraging scheduler policies we are able to pack / over-provision ES node pods of different clusters onto the same Kubernetes nodes, allowing for far greater resource efficiency, while being able to handle node failure while maintaining availability across all clusters. Additionally, this simplified operations significantly as we can now leverage the operator to do cluster wide operations (e.g rolling restarts, node OS upgrades, ES version upgrades, etc...) fairly easily.

We did, however, go 6 years (and several hundred million users and trillions of documents indexed) without needing to use Kubernetes!

We will blog about this at some point this year.

jhgg··on Discord is down
we are on it
jhgg··on OpenAI Shoves a Data Journalist and Violates Federal Law
Right, but the penalty for non compliance here is a $20/day fine. No where does the law state that you're granted license to enter the building if they refuse you entry or deny your request. You made your request, they refused and asked you to leave. Your only valid recourse here is to file a complaint with the IRS and let them do their thing. Alternatively return after the holiday break when the office has the staff there that can fulfill your request.
← PreviousPage 2 of 17Next →