HNHacker News
TopNewBestAskShowJobs

eieio

3,173 karma · joined December 24, 2011

I build unusual games with big constraints (such as https://en.wikipedia.org/wiki/One_Million_Checkboxes). You can see all my work at https://eieio.games

I'm based in Brooklyn. You can reach me at eieiogames at gmail dot com

alum Jane Street Capital, Recurse Center (F23, W25)

submissionscomments
eieio··on Running a million-board chess MMO in a single process
FWIW (I'm the author) my creative output was ~0 while I was working a 'normal' job. I worked really hard and didn't have much energy for tech stuff outside of work (especially since I wanted to live a life that included non-tech things!)

I think it's totally fine to not make stuff outside of work, and it's so impressive to me that some of my friends manage to make creative stuff in their free time while working a day job.

eieio··on Running a million-board chess MMO in a single process
One million radio buttons exists! https://omrb.olivia.website/

I think the research on progress bars and what makes users feel good is super interesting. But I also think "basically instant" is a good thing to aim for when you can.

eieio··on Running a million-board chess MMO in a single process
Ha, yes, I was absolutely in the red a decade ago when I got sober.

A huge congratulations to you - 11+ years is incredible (I'll be at 11 in 5 months!). It really is crazy how things can turn around.

eieio··on Running a million-board chess MMO in a single process
I thought about not pushing snapshot/move data over websockets - one of the systems-y friends I ran my architecture by brought this up while I was speccing the site out.

You can't really put move batches on disk and have clients grab them (afaik), since the set of moves you want to send to an individual client depends on their position (and you don't want to send every move to every client).

But you could do this by not sending move batches at all, and instead having clients poll for the entire current state of the board.

The thing is, for them to get realtime-ish move updates they'd have to poll constantly. Cloudflare also has a min TTL of 1 second so there'd be more latency, and also if I screwed something up or saw more cache misses than anticipated I could end up unintentionally hammering my server.

Also if I'd had 100x more traffic (which would be crazy and well beyond what I prepared for!) I think I'd owe like $95 or so for bandwidth with my current setup. So the benefits to reducing bandwidth even more were a little marginal!

eieio··on Running a million-board chess MMO in a single process
Hi! I'm the guy.

I have the savings to not worry about monetizing anything for a while. So I don't monetize my stuff. It's freeing and kinda fun!

eieio··on Show HN: Tattoy – a text-based terminal compositor
looks good for me on mobile now!
eieio··on Show HN: Tattoy – a text-based terminal compositor
I think you just need to add "playsinline" to your videos so that they play inline on mobile devices instead of fullscreening.

    <video autoplay loop muted playsinline>
instead of your current

    <video autoplay loop muted>
cool project :)
eieio··on One Million Chessboards
I'm using a moderately beefy server but I don't think I've hit double-digit CPU usage; this could have run on a much smaller box
eieio··on One Million Chessboards
The rate limits aren't that aggressive and have a decent amount of burst, you can get about 10 moves done in 1 second before you hit them and start getting throttled[1]. And of course you can run multiple clients (I account for this too, but I'm not that aggressive because I don't want to punish many people NAT'd behind a single IP)

[1] down to something like 3/sec?

eieio··on One Million Chessboards
these days I mostly use vscode / cursor, although I still really like vim and use it for languages that I know really well (mostly python these days) and quick edits.

I spent much of my professional career at Jane Street Capital, which means that I spent a long time just writing OCaml and some bash (and a tiny bit of C). I'm very comfortable with Python, and over the last year I've gotten pretty comfortable with frontend javascript. And now golang!

I could probably write semi-reasonable java, ruby, or perl if you gave me a few days to brush up on them. And it'd take me a while before I was happy putting C on the internet. Not sure otherwise.

eieio··on One Million Chessboards
Hah, yes, but for testing I removed all my rate limits so I pushed 1 million moves in 2 or 3 seconds, whereas now I think I rate limit people to like 3 or 4 moves a second (which is beyond what I can achieve on a trackpad going as fast as I can!) so the test isn't quite comparable!

I definitely learned a lot here. Most of my projects like this are basically just "give the internet access to my computer's memory but with rules." And now I think I've got a really good framework for doing that performantly in golang, which should make the next set of projects like this much quicker to implement.

I also just...know how to write go now. Which I did not 6 weeks ago. So that's nice.

eieio··on One Million Chessboards
oh this is a pretty fun idea! If I decide to keep this one around for a bit maybe I'll play with doing this after wiping the board
eieio··on One Million Chessboards
I didn't like the idea of a queen on one board capturing the king on another board as her first move. And then I tried this rule and thought it created really fun counterplay when you're trying to capture a piece someone else is controlling, since you can move to a new board to be safe (and can lay traps this way).

I'm sorry you don't like that decision! But I think that I stand by it.

eieio··on One Million Chessboards
fwiw I think the biggest single trick there is to group IPV6 addresses at the /48 or /64 level before applying rate limits (you can rate limit IPV4s on a per-ip basis).

It's kind of annoying and expensive to get a bunch of IPv4s to evade limits, but it's really easy to get a TON of IPv6s.

The other Big Trick I know is to persist rate limits after a client disconnects so that they can't disconnect -> reconnect to refresh their limits.

eieio··on One Million Chessboards
I have had some success on Hacker News with similar games before[1]! But I'll certainly do a proper writeup of the tech behind this one because I think it's pretty neat.

[1] https://news.ycombinator.com/item?id=40800869 for example

eieio··on One Million Chessboards
Ah thank you dang! I meant to submit a Show HN for this one with a little context/a few technical details, but someone beat me to submitting the link. So this is perfect :)

(and thanks, I'm in no rush!!)

eieio··on One Million Chessboards
So sometimes I don't test these projects that much but I did this time. Here are a few thoughts:

My biggest goal was "make sure that my bottleneck is serialization or syscalls for sending to the client." Those are both things I can parallelize really well, so I could (probably) scale my way out of them vertically in a pinch.

So I tried to pick an architecture that would make that true; I evaluated a ton of different options but eventually did some napkin math and decided that a 64-million uint64 array with a single mutex was probably ok[1].

To validate that I made a script that spins up ~600 bots, has 100 of them slam 1,000,000 moves through the server as fast as possible, and has the other 500 request lots of reads. This is NOT a perfect simulation of load, but it let me take profiles of my server under a reasonable amount of load and gave me a decent sense of my bottlenecks, whether changes were good for speed, etc.

I had a plan to move from a single RWMutex to a row-locking approach with 8,000 of them. I didn't want to do this because it's more complicated and I might mess it up. So instead I just measure the number of nanos that I hold my mutex for and send that to a loki instance. This was helpful during testing (at one point my read lock time went up 10x!) but more importantly gave me a plan for what to do if prod was slow - I can look at that metric and only tweak the mutex if it's actually a problem.

I also took some free wins like using protobufs instead of JSON for websockets. I was worried about connection overhead so I moved to GET polling behind Cloudflare's cache for global resources instead of pushing them over websockets.

And then I got comfortable with the fact that I might miss something! There are plenty more measurements I could have taken (if there was money on the line I would have measured some things like "number of TCP connections sending 0 moves this server can support" but I was lazy) but...some of the joy of projects like this is the firefighting :). So I was just ready for that.

Oh and finally I consulted with some very talented systems/performance engineer friends and ran some numbers by them as a sanity check.

It looks like this was way more work than I needed to do! I think I could comfortable 25x the current load and my server would be ok. But I learned a lot and this should all make the next project faster to make :)

[1] I originally did my math wrong and modeled the 100x100 snapshots I send to clients as 10,000 reads from main memory instead of 100 copies of 100 uint64s, which lead me down a very different path... I'm not used to thinking about this stuff!

eieio··on One Million Chessboards
Ah yes good question! Here's some context for you. First off, the way moves work:

(edit: I realized I didn't answer your question. If we receive a captured for a piece we're optimistically tracking that always takes precedence, since once a piece is captured it can't move anymore!)

    * clients send a token with each move
    * they either receive a cancel or accept for each token, depending on if the move is valid. If they receive an accept, it comes with the sequence number of the move (everything has a seqnum) and the ID of the piece they captured, if applicable
    * clients receive batches of captures and moves. if a move captured a piece, it's guaranteed that that capture shows up in the same batch as the move
So when you make a move we:

    * Write down all impacted squares for the move (2 most of the time, 4 if you castle)
    * Write down its move token
    * If you moved a piece that is already tracked optimistically from a prior not-yet-acked-or-canceled move, we note that dependency as well
We maintain this state separate from our ground truth from the server and overlay it on top.

When we receive a new move, we compare it with our optimistic state. If the move occupies the same square as a piece that we've optimistically moved, we ask "is it possible that we inadvertently captured this piece?" That requires that the piece is of the opposite color and that we made a move that could have captured a piece (for example, if you moved a pawn up that is not a valid capturing move).

If there's a conflict - if you moved a piece to a square that is now occupied by a piece of the same color, for example - we back out your optimistically applied move. We then look for any moves that depended on it - moves that touch the same squares or share the same move token (because you optimistically moved a piece twice before receiving a response).

So concretely, imagine you have this state:

    _ _ _ _
    K B _ R
You move the bishop out of the way, and then you castle

    _ _ B _
    _ R K _
Then a piece of the same color moves to where your bishop was! We notice that, revert the bishop move, notice that it used the same square as your castle, and revert that too.

There's some more bookkeeping here. For example, we also have to track the IDs of the pieces that you moved (if you move a bishop and then we receive another move for the same bishop, that move takes precedence).

Returning the captured piece ID from the server ack is essential, because we potentially simulate after-the-fact captures (you move a bishop to a square, a rook of the opposite color moves to that square, we decide you probably captured that rook and don't revert your move). We track that and when we receive our ack, compare that state with the ID of the piece we actually captured.

I think that's most of it? It was a real headache but very satisfying once I got it working.

eieio··on One Million Chessboards
Yes exactly!

My goal with the board architecture was "just be fast enough that I'm limited by serialization and syscalls for sending back to clients" and go made that really easy to do; I spend a few hundred nanos holding the write lock and ~15k nanos holding the read lock (but obviously I can do that from many readers at once) and that was enough for me.

I definitely have some qualms with it, but after this experience it's hard to imagine using something else for a backend with this shape.

eieio··on One Million Chessboards
Ah hello! I made this :)

My blog describing it is pretty sparse, sorry about that. Happy to answer any questions that folks have about the architecture.

Not that it was necessary, but I got really into building this out as a single process that could handle many (10k+/sec) moves for thousands of concurrent clients. I learned a whole lot! And I found golang to be a really good fit for this, since you mostly want to give tons and tons of threads concurrent access to a little bit of shared memory.

eieio··on One Million Chessboards
I'll certainly open source the code! I just want the flexibility to change my rate limiting logic in the short term to counteract abuse. Happy to answer questions though!
eieio··on One Million Chessboards
oh huh, I'm not sure why this one made it to the front page and not the link to my site!

Anyway, yeah, I guess I could have gone with turns here but I thought that building a more realtime MMO thing where pieces could cross boards would be a little more interesting and novel. I also didn't feel like a version of this that was turn based would ever complete.

certainly a queen can go wipe out a whole board, but the game tries to place you next to other active players when you join, which hopefully promotes some interesting counterplay to that. And I think playing chess in realtime like this against someone is pretty fun. But I understand why it might not be for everyone!

eieio··on Blue Prince is a roguelike puzzle masterpiece
FWIW Adrian Hon (the writer) is easily the game reviewer that I trust the most! I am quite sensitive to false positives with games - it really sucks to spend a few hours on a game that you drop - and my false positive rate with his reviews is very low.

(I have no affiliation with this post beyond being a fan of Adrian's writing and work + haven't played Blue Prince yet, although I'm very likely to play it because of this review)

eieio··on Twitch limiting uploads to 100 hours, deleting the rest starting April 19th
It is! You can see the pool here: https://news.ycombinator.com/pool

And this submission shows up there (it's on the second page at the time that I write this comment)

eieio··on Running Pong in 240 browser tabs
ah thanks adam! fun to see you here :)
eieio··on Running Pong in 240 browser tabs
Sorry yeah I'm talking very specifically about the stuttering that I first saw when making a square move into the tab bar.

I was doing something inefficient (updating all my favicons many times a second), but I didn't really understand why that would cause stuttering in my foregrounded tab. My thought process was that the favicon stuff was happening in different tabs in different threads, and so it didn't make sense to me that the thread in charge of the foregrounded animation should suffer (unless my entire computer had slowed to a crawl and everything was getting slower - but it hadn't).

And my friend's point was just that i was probably GPU-bound and not CPU-bound (or at least I could be) and so I was not thinking about the original problem correctly.

eieio··on Running Pong in 240 browser tabs
Matt's work is a huge inspiration for me!

I definitely had his URL stuff in mind when making this, but it'd been a while since I had actually read through his blog / watched his video. It was really fun to come back to it mid-project and remember that he went in the same "stack windows to get 2 dimensions" direction.

eieio··on Running Pong in 240 browser tabs
what’s your pong game? would love to see it
eieio··on Running Pong in 240 browser tabs
Doom is kind of a pain because:

    * afaik I'm capped at 4 FPS (just how often you can update tabs)
    * more of the screen is likely to update on each frame, which requires way more "turning canvases into data urls to update favicons," which is expensive with my current implementation (although I wasn't aware of offscreencanvas, which might help here)
eieio··on Running Pong in 240 browser tabs
ah hi! I made this! was wondering if this one would appeal to the HN crowd :)

Happy to answer any questions! And here are a couple of scattered thoughts:

    * I'm really interested in what this looks like if you use animations (firefox supports animated favicons) - I could, for example, anticipate future ball positions and create animated SVGs to get a much nicer framerate.
    * A friend pointed out offline that canvas rasterization (typically) is on the GPU, which is probably why my performance intuitions with my stuttering animation were so wrong
    * I am only moderately confident that chrome caps favicon updates to 4 a second; I know there are a lot of different ways to update favicons and I could have missed something here!
← PreviousPage 2 of 10Next →