for a while I thought I had an off by one error, which was pretty funny to think about in the context of trillions of items. but in fact I was just doing some bad react state management.
3,173 karma · joined December 24, 2011
I'm based in Brooklyn. You can reach me at eieiogames at gmail dot com
alum Jane Street Capital, Recurse Center (F23, W25)
for a while I thought I had an off by one error, which was pretty funny to think about in the context of trillions of items. but in fact I was just doing some bad react state management.
(someone actually made an issue for this: https://github.com/nolenroyalty/paccam/issues/1)
1. I heard this a few times during playtesting, but I thought that having walls would just make the game too challenging; it's hard enough to move around as it is, and getting stuck on a wall would be no fun. I considered sticking people to more of a "grid" but didn't really see the benefit of that without having walls, felt like turning your head "up" and still moving to the right would be disorienting, and thought it would add some complexity to my engine, so I didn't.
2. The bots are playing with exactly the same parameters as you! In fact, on average they should be slightly slower than a human that opens and closes their mouth at the "optimal" speed. The bot logic[1] is only capable of opening/closing a bot's mouth or changing the direction that they're facing, and then the same rules apply to them as a human.
That said, the bots are...pretty good, at least until you get pretty used to the game. Maybe this is a mistake? I didn't have bots until a few days before launch; this was originally just a local multiplayer game, and I added bots to make it easier for my friends to test. Then I realized that bots were a straightforwardly good addition. But it might have made sense to spend some time adding bot difficulty levels or something - the fact that they can react quickly to information and that they don't struggle to turn their face at the right time makes them pretty good.
That said I promise they're beatable :)
[1] https://github.com/nolenroyalty/paccam/blob/main/src/BotStat...
The game does need to download a model[1] to do the face tracking, and right now that does happen after you enable your webcam[2]. but once that model is downloaded the game runs fine with wifi disabled
alternatively the code is open-source (and linked on the page), so if you really wanted to you could check it out and then build it yourself: https://github.com/nolenroyalty/paccam
[1] it uses mediapipe, an easy way to do face tracking in the browser https://ai.google.dev/edge/mediapipe/solutions/guide
[2] it's a reasonable heavy download, didn't really want to put it on page load, although in retrospect it'd have given me a much more satisfactory response here!
They had some notion of the squares they were watching and what their values needed to be (to draw the images). And then they subscribed to the full state of the grid (which my API made really easy to do) and would respond by sending websocket requests to check boxes when they got toggled to the "wrong" value.
I'm sure people did other things - lots of people wrote scripts in the browser console. But I think the above is basically what people in the discord did.
I had some basic protections in place. There's some client-side rate limiting that prevents the really trivial "run the script in the console stuff" and then some server-side per-IP limiting to prevent really aggressive botting. Plus my server got slow at some points which inadvertently slowed people down.
The big difficulty with the server-side stuff is that you don't want to be too aggressive with IP based limiting because sometimes a large group of people resides behind a single IP (like at a school) and you want to make sure that all of those people can play. So it's a balancing act.
there were a bunch of bots run by different people, but for the most part they coordinated in the discord so that they weren't fighting over the same "territory"
at one point there was a group not in the discord trying to draw in the same place as the people in the discord, which lead to the bots constantly changing checkboxes to try two different images! but it was mostly cooperative
* More than anything, I think it's good for things to end! I figured interest in the site would die off over time (and it started to), and I thought it was better to close things out providing a special experience for the people that used it than to keep it up to get a few more users
* Costs started adding up; donations stopped matching them. I coulda figured out how to lower my costs but I wasn't excited about it.
* While the site was up I felt an obligation to make sure someone hadn't found some trivial workaround to deface the thing and I didn't want to do that anymore.
I'm very pro ephemeral stuff! So I feel good about the decision. But it's a good question.this is my favorite story from running the site, and possibly the best story I've ever been a part of. I'm not a big crier but I have cried so many times thinking and trying to write about it over the past 2 months. And of course, the process of discovery (and going from panic to excitement) was pretty crazy too.
One of my favorite things about this is that it validated one of the core beliefs I have when making these things - that you need constraints for the small group of people that are jerks, but that for the most part those constraints are fodder for the largely-good and very creative folks that play around on the internet.
Happy to answer any questions folks have!
There's probably 0 overlap in the codebase - 100% (or close to it) of the code is from Neal (the only code I wrote for this was a very early prototype that cropped out just the eyes, instead of the bar you see now). I sent him a couple of snippets from my codebase early on but mostly just gave pointers on my tech stack / chatted about some of the problems I ran into. Given the size of Neal's audience the default scale for a project he launches is pretty big!
Anyway, mostly wanted to clarify that the similarities between this and stranger video are intentional :)
[1] this is where the idea for one million checkboxes actually came from!
We were originally going to collaborate more on eyechat but then One Million Checkboxes[1] blew up and I had to bow out
But that's based on my skillset and the timeframe of the project; I'm not disputing that the project is bounded by Redis's performance.
My takeaway from this experience is that I have no idea which of my sites is going to be popular - most of my projects are not nearly this successful - and that I should be optimizing for speed of creation, not speed of an individual project. My comparative advantage is "writing ok code and making ok decisions very quickly" and I'm going to continue to lean into that.
Oh sure, yeah, I think we just agree then!
Painful to use managed redis when I was debugging (let me log in! let me change stuff! give me the IP!! ahhhhh!!!!) but really nice otherwise. A little painful to think about giving that up, although I coulda run a really hefty redis setup on hertzner for very little money!
The entire timeframe of this project was 2 weeks, and the critical period (most activity / new eyes) was a couple of days.
FWIW I certainly hit storage space in some boring ways - I didn't have a good logrotate setup so I almost ran out of disk, and I sent my box-check logs to Redis and had to set something up to offload old logs to disk to not break Redis. But neither of those were very big deals - pretty interesting to have a problem like this where storage just wasn't a meaningful problem! That's a new one for me.
And yeah, thinking about bandwidth was such a headache. I was on edge for like 2 days, constantly checking outbound bytes on my nic and redoing the math - not having a hard cap is just really scary. And that's with Digital Ocean, which has pretty sane pricing! I haven't used the popular serverless stuff at all, but my understanding is that you get really gouged on bandwidth there.
(also yes, lua-in-redis is really incredible and lets you skip sooo many hard/racey problems as long as you're ok with a little performance hit, it was a joy to work with)
I don't think it's unscalable at all - it's just that if you do this there's not a great story for adding a second machine if the first one can't handle the load (but ofc a beefy machine with a fast implementation could handle a lot of load).
When we did the go rewrite we considered just getting one beefier box and doing everything there (and maybe moving Redis state in memory), but it felt easier and safer to do a drop-in rewrite.
If I was planning to keep the site up for long I would have moved, but in this case I knew it was temporary and so toughing it out on DO seemed like a better choice.
Sorry that some of the stuff went over your head! I wanted to include longer descriptions of the tech I was using but the post was already suuuper long and I felt like I couldn't add more.
Very happy to answer any questions that you've got here!
I'm not sure how you'd simplify the architecture in a major way to be honest. There are certainly services you could use for stuff like this, but I think there you're offloading the complexity to someone else? Ultimately you need:
* A database that tracks which boxes are checked (that's Redis)
* A choice about how to put your data in your database (I chose to just store 1 million bits for simplicity)
* A way to tell your clients what the current state is (I chose to send them all 1 million bits - it's nice that this is not that much data)
* A way for clients to tell you when they check a box + update your state (that's Flask + the websocket)
* A way to tell your clients when a box is checked/unchecked (that's also Flask + websockets. I chose to send both updates about individual boxes and also updates about all 1 million boxes)
* A way to avoid rendering 1 million dom elements all the time (react-window)
The other stuff (nginx for static content + a reverse proxy) is mostly just to make things easier to scale; you could implement this solution without those details and the site would work fine, it just wouldn't be able to handle the same load.Think the total cost was about $850, which was (almost) matched by donations.
I made a mistake and never really spun down any infrastructure after moving to go, and also could have retired the second redis Replica I spun up; think I could have cut those costs in half if I had been focused on it. But given that donations were matching costs and there was so much else going on I wasn't super focused on that.
I kept the infra up for a while after I shut down the site (to prepare graphs etc) which burned a little more money, so I'm slightly in the hole at this point but not in a major way.
Thanks for the heads up! Guess it's not time for me to revisit Godot yet :(
From looking through the TODOs here, I think the one thing that would stop me from playing with this soon is signal support. I can't quite tell how non-existent the signaling support is here: can I just Obj.magic (unsafe cast) stuff and get it to work, albeit without type safety? Signals in Godot are great and I'm already used to them not being type safe (they're just strings in Godot 3! Gross!) so that'd be tolerable.
Anyway, very cool stuff.
[1] I've been meaning to go back at try out C#, which might be more my taste. I held off partially because I don't know C# yet and partially because examples for C# in Godot seemed a little lacking when I last looked - but also because Godot 4 web exports were pretty busted for a while, which irritated me enough to make me use something else. I think there's been some progress here though!
My level design skills are pretty weak. Being able to partially compensate for that by getting built-in nice character movement, enemy design, etc for free was really really nice. If I wanted to do more 3D stuff right now I'd almost certainly be using something like Mario Builder (I think stuff like Doom custom maps get at a similar idea).
If Recurse sounds fun and magical to you - it is! And you should consider applying :). You can always defer your acceptance until the timing works out. It's free (they make money via job placements, but when I applied I was clear that I wasn't looking for work). I've been meaning to write a blog about my time there - here's what I would want to say:
The best thing that Recurse (RC) did for me was help me get in touch with my own taste. When I arrived I was making games that were pretty "normal" - Flash-style games with high scores and weak themes. While there I got weird. One of my first projects was an abuse of the OpenSearch spec to make a version of Wordle that ran in the Firefox address bar[1].
It was the perfect place to build something like this - folks were encouraging and supportive and interested; it made me realize that people were interested in the type of thing I actually wanted to do. I think I shipped 8 games during my 12 weeks at RC. I wouldn't have started making the types of things I now make without RC. I think plenty of folks have similar experiences across all sorts of techy things, not just games. Kinda like finding product-market fit but for your own interests.
RC also connected me with a bunch of folks that were doing similar things. The community is huge and kind and weird in the best way. It is like a writers' retreat but for people that want to program (and become better programmers).
I am probably coming off as shilling RC hard and I suppose I am (although I'm not being paid for this - I'm just a grateful alum). I'm literally typing this from the RC space right now (I occasionally stop by to chat with people in batch and work).
[1] https://eieio.games/nonsense/implementing-wordle-in-the-fire...