HNHacker News
TopNewBestAskShowJobs

Darkphibre

679 karma · joined December 11, 2018

Decades of engineering / architectural work. 20 years at Microsoft. 5 years SQL Server Analysis Services. 7 years at Xbox Advanced Technology Group. 10 years at 343 architecting realtime telemetry pipelines used by every game from Gears to Forza to Halo to Minecraft to Solitaire. I'm now a Principal Data Solutions Architect for a billion dollar startup, helping transition the organization to leverage AI and guide our organization on event-driven design for scale.
submissionscomments
Darkphibre··on Probiotics can support the effect of antidepressants
No lie, I've done exactly this before.

Something was coming up that'd wipe out my intestinal flora, and I'd been feeling extremely healthy lately. So I preserved a chilled sample of the 'good stuff.'

Usually I'd have all sorts of problems as the gut bacteria sorted itself out, but this time it was... smooth sailing.

Sure, it's... WEIRD in the extreme to actually perform. But I can't argue with the results.

Darkphibre··on SpaceX said to fire employees involved in letter rebuking Elon Musk
It's fascinating to me how many comments seem to be defending the workplace culture at SpaceX, and outright dismissing the letter in a hostile manner.
Darkphibre··on A mystery of the ketogenic diet: benevolent pseudo-diabetes in mice (2019)
Love the anecdote, wish it applied to me!! I constantly drift upwards, even when eating crazy healthy. My "caloric rheostat" is just dialed in a bit high, so I have to accept I need to eat less than I think is correct... so I have to watch my macros religiously.

But, that's what healthy living habits are: Finding what works for you!

Darkphibre··on The moon’s permanent shadows are coming to light
I mean, the James Webb makes use of such to drop to, what, 20 Kelvin? https://ttu-ir.tdl.org/bitstream/handle/2346/67554/ICES_2016...
Darkphibre··on MCC (Master Chief Collection) Server Incident Summary
The thing with using cloud services is that sometimes the hosting provider can make changes to configurations that have impacts (such as monitoring software that steals precious CPU at seemingly random moments, or network topology hardware that impacts connectivity)... without it necessarily being under your own control. I've had a few high-visibility incidents where, after investigation, the buck stopped with us even though technically the studio could have shifted blame to an unanticipated change from the hosting provider.

Disclaimer: I currently work at Microsoft Game Studios, though this comment reflects my own opinions based on experiences unrelated to this incident.

Darkphibre··on MCC (Master Chief Collection) Server Incident Summary
Along these lines is the venerable TRIBES Engine Networking Model whitepaper. It was so good, it was shipped as part of the XDK for a decade. I believe Bungie even leaned on it quite a bit when creating their networking stack. https://www.gamedevs.org/uploads/tribes-networking-model.pdf

Disclaimer: I work at Microsoft Game Studios, but this comment reflects my own opinions.

Darkphibre··on I built a receipt printer for GitHub issues
Heh, that's in the article under _Wrapping up and next steps_

> For the tickets themselves, a QR code could be added to link directly to the issue on GitHub. You could also add in more details from the issue itself like tags and severity.

Darkphibre··on Almost 37 years after its launch, someone found an Easter egg in Windows 1.0
The first time I realized the danger of Easter eggs was one that was triggered by Alt+Ctrl+Shift+PERIOD or some such. Tested it out and sure enough it would paste an email exchange about needing to add an easter egg into a comment field.

Then I was demoing the software to leadership in a surprise visit, and it triggered. Turns out, I'd ORed the keys together! I tried to Ctrl-A+DEL but... the email was longer than the comment field and triggered a state in which any keypress was rejected with an alert, even one that would shorten the comment (bug #2). And I couldn't leave the form, as the early prototype was designed to save-on-exit. So I had to kill the app... lesson learned.

My last easter egg was simply the name of "Lightish Red" for one of the AI colors in Halo Infinite. Much safer (once it got through localization, hah).

Darkphibre··on Show HN: SHA-256 explained step-by-step visually
Man, this is amazing. I had to hand-unroll bit packing in a binary encoding scheme we used in a game. Rare enough that making a tool wasn't worth it, but damn I love your visualizations! Doing something like that would have helped others understand how I was "seeing the matrix."
Darkphibre··on Slime molds’ complex ability to detect mass from a distance
> They found that, even more than the total mass of the objects, the slime mold reacted to how much of the horizon the objects occupied. It repeatedly preferred objects spread along the horizon as opposed to those at a single, stacked point
Darkphibre··on The Value of In-House Expertise
Yup! I'd told them last year as the divorce was starting that I'd need to be promoted within 18 months, or I'd start to become bitter and perform less well... and I'd leave before that happened.

Just put in for a GDC talk, hopefully that gets approved. Otherwise I'll stick around to give a talk at the Game Analytics Summit (big networking event for my niche of a field), and spin up some conversations. Also have contacts in most major studios... or I may just strike out and start doing consulting work. It was always a dream of mine to get back into global consulting work once the kids were grown up. They have been, but the ex was wanting to stick around for the grandkids.

Me, I'll be a free woman to make her mark on the world (once I can extract myself from the leech).

Darkphibre··on The Value of In-House Expertise
Oh, I'm sure that's the case. That, and they know I love the product and would (usually) be willing to put up with lower pay to keep working on it. But, losing assets in the divorce, I'm going to need to take more risk to get back on my feet. Only have a few decades to retirement, and need to rebuild that nest egg. So my priorities are changing, and I'm less willing to put up with people that don't appreciate me.
Darkphibre··on The Value of In-House Expertise
Please tell me you found better pastures! A place where you were compensated and appreciated. :)

I've been at the current job for 10 years, receiving the highest level of performance review for all but the last year (the global pandemic was one of four life upheavals... I'm glad I pulled off a 'great' review vs. 'exceptional'), doing similar style work of solving problems people can't seem to comprehend... yet I just can't get seem to break into the next job title tier because "We don't need a principal."

...

I just found out my lead was just promoted to principal. Once the divorce is off my plate, I plan on taking more risk and jumping ship.

Darkphibre··on Building a new vector based storage model
Oh, this is fascinating. Seven years ago I architected a true-realtime telemetry pipeline with end-to-end sequential guarantees (with roundtrip times <200ms excluding network latencies, and cloud processing times <20ms, leveraging BOND/ProtocolBuffer over AMQP over Websocket). It's still used by every 1st-party game for a large publisher.

It allowed for non-windowed event sequence analytics, enabling realtime feedback (think achievements that have multiple conditions).

And then the requirement was dropped, and (as you've found), everyone just uses it like a standard telemetry stream and is OK with 5-15min bins. :P

I still have a passion for the space, will definitely be reading up on this. I firmly believe this is the future of telemetry analytics; Congratulations on your efforts seeing the light of day!!

Disclaimer, I currently work for Microsoft, all words here are my own and do not necessarily reflect those of my employer, etc. ;)

Darkphibre··on All my servers have an 8 GB empty file on disk
Oh, fascinating! My first vehicle was the family's 3/4-ton Diesel '84 Chevy Pickup from the farm, and I'd forgotten it had an Aux fuel tank! This makes a lot of sense.
Darkphibre··on Only Google is really allowed to crawl the web
Hooray! Looks like I'm one of today's lucky 10,000. :)

https://xkcd.com/1053/

Darkphibre··on ISO-8601 date format reference not publicly available
Hmm? This timestamp is known to be local-time, used for measuring durations-between-events on a single session, and not used for cross-session reporting. Time would be compared from one event to the next.

Cross-session reporting (such as calculating DAUs or MAUs) reports on the timestamp from the ingestion service, which is applied at the batch level before a batch of events is placed on the event hub for routing (events are batched and transmitted every 16ms off the client). Typically you can get away with treating everything that happened in the same simulation frame as having happened instantaneously... though we use a monotonically incrementing guaranteed-unique sequence identifier for processing using a high-water-mark (one stream is at-least-once sequentially-guaranteed with retries, while the other stream is lower priority and at-most-once sequentially-guarantied without retries).

No amount of ISO spec will help with client clocks being completely arbitrary. :D The ingestion service clocks are NTP synced to datacenter-hosted atomic clocks, but we still see a few-second skew here and there.

I've even done research into how often they drift backwards in time (but that doesn't really happen, since NTP will just slew the system time by delaying, or adjust the clock massively on startup before the game launches).

Darkphibre··on ISO-8601 date format reference not publicly available
"Funny" story about a binary 8601 implementation.

I was architecting a true-realtime telemetry pipeline for a AAA videogame studio (later used by a dozen or so franchises for all titles by the publisher). It had a requirement for subsecond client notification of per-user aggregated statistics, including complicated event interactions (such as ensuring the grenade that you threw that killed someone, would only be attributed to you if a prior event hadn't already occured), which resulted in sequentiality guarantees. But with that, we could get rid of windowed event processing (and those inherent latencies), and instead treat them as a true stream of events. Keep in mind we could support up to 400ms one-way latency, so we only had 200ms for processing/routing in the client and in the cloud. I measured client code in μs.

Annnyways. The principal in our org insisted that time not be be transmitted as an epoch. I argued for weeks, but he was insistent... "No magical offsets to arbitrary dates. ISO8601. Do it." He came from and worked most heavily with the web services group. Apparently some point in the past, services had picked the wrong epoch and there was confusion (I still can't understand how that wouldn't be caught right away, there's only a few standard epochs that are quite different in resulting time)? The events in my design were bit-packed using a competing protocol to Protocol Buffers... there's no way I was doubling the payloads just to store a timestamp as a string.

So I read and re-read ISO8601. And it dawned on me: ISO-8601 never specifies the encoding. It specifies the order in which information is conveyed, and if in text, the delimiters to use. One example clue is in the definition of basic format, and the call-out that most of the document leverages plain text to communicate their ideas.

basic format format of a date and time representation or date and time format representation comprising the minimum number of time elements necessary for the accuracy required. The basic format should be avoided in plain text.

I ended up proposing the most terrible hack I've had to live with. It involved bit-packed 64-bit integer that stored: 12 bits for year, 4 bits for month, 5 bits for date, 5 bits for hours, 6 bits for minutes, 6 bits for seconds, and the remaining bits for an integer that stores the sub-second fraction. We lost a few orders of magnitude resolution on that end due to the inefficiencies of the earlier packing, but... it worked. And it was in the right order. And there were no "magic" epochs to dissuade the principal engineer.

Most importantly, it sailed through the review process.

Still makes my skin crawl a decade later. And I secretly suspect you technically have to have a T between date and time to be ISO8601 compliant, it's been a long while since I read the spec. But there you go.

Darkphibre··on Physicists have filmed the oscillation of a time crystal
I don't know... atomic clocks are approaching precision that can measure the effects of time dilation due to gravity on the order of inches. They have to take into account daily tectonic tides.

> The precision of current optical clocks is astounding. You may have heard that time goes more slowly when gravity is strong due to general relativity. Optical clocks are so sensitive they can measure the different flows of time 2cm apart in height. If I lay a book on the table, the bottom of the book is slightly closer to the center of the Earth than the top, so experiences slightly stronger gravity. This difference is measurable with an optical clock. Optical clocks are so sensitive we can no longer average the time of multiple clocks together—the ground you or a clock are sitting on typically rises and falls by ~5cm a day due to land tides. The seismic motion of the ground currently limits our ability to measure time. https://arstechnica.com/science/2021/01/a-curious-observers-...

Darkphibre··on Parsing can become accidentally quadratic because of sscanf
I guess my point was a curiosity that these control codes had become obscure, when text format interchange is so prevalent throughout computing history.
Darkphibre··on Lotus 1-2-3 arbitrary resolution
Heck, that reminded me of The Well! Looks like they're still going strong. :D

https://people.well.com/conf/inkwell.vue/topics/510/State-of...

Darkphibre··on Parsing can become accidentally quadratic because of sscanf
You know, why don't we use 0x1F (Unit Separator)? Seems like we have had a hierarchical delimiters since ASCII / punch cards, yet I never hear about them. https://en.wikipedia.org/wiki/C0_and_C1_control_codes#Field_...

CSVs are such a pain if you have freeform text (in which you have to handle newlines and escaped quotes).

It seems like using FS/GS/RS/US from the C0 control codes would be divine (provided there was implicit support for viewing/editing them in a simple text editor). I get that it's a non-printable control character, and thus not traditionally rendered... but that latter point could have been done ages ago as a standard convention in editors (enter VIM et. al., as they will indeed allow you to see them).

Darkphibre··on Mars becomes the 2nd planet that has more computers running Linux than Windows
As someone that worked at Microsoft on the XAudio 2 for the Xbox 360 & Windows Phone stacks... holy heck that android audio path is painful to read. >.<;;

We went with a 5.33ms quantum on the Xbox 360 (vs. Window's 10ms), and tried to stay out of the way as much as possible to ensure minimal latencies: https://docs.microsoft.com/en-us/windows/win32/xaudio2/xaudi...

Darkphibre··on Mars becomes the 2nd planet that has more computers running Linux than Windows
Fun read, tyvm! I loved this anecdote from the second link:

> On sol 20, the command team sent it the command SHUTDWN_DMT_TIL ("Shutdown Dammit Until") to try to cause it to suspend itself until a given time.

>

> It seemingly ignored the command.

Darkphibre··on Drug for treating obesity cuts body weight by 20%
Yes, obesity is a risk factor when catching COVID.

Or, if you meant to question the truthfulness of their claims (without offense, is that possible?)... A quick internet search shows that the heaviest person recorded was 50% heavier.

So, yes, the weight described is certainly possible.

Darkphibre··on Strategies to Reduce Brain Fog and Improve Clear Thinking
Yup! I've also found what I eat is important.

Carbohydrates to be a source of brain fog as I get older. I've also found cutting them out to 0g/day seems to increase anxiety for me (studies have shown reducing it below a certain amount... 50g?... definitely increases cortisol, a stress hormone). So finding a good balance in the diet is key.

Darkphibre··on Base64.ai – Extract text, data, photos and more from all types of docs
Thanks for the heads up! Will definitely check it out.
Darkphibre··on Base64.ai – Extract text, data, photos and more from all types of docs
Yup, was planning to use ffmpeg (or, more likely, OpenCV), and a subset of the frames.

Thanks so much for the tip on DeepSpeech!

Darkphibre··on Base64.ai – Extract text, data, photos and more from all types of docs
Interesting!

I've been thinking of running OCR on video frames. I'd also like to do speech-to-text extraction for searching my archives later (have about 4TB of video to trawl through, and desire text-based search capabilities). It's an interesting space to explore, but everything's been moving to web-service at a cost-prohibitive model.

Darkphibre··on FTC Settlement with Fertility-Tracking App for Facebook/Google SDK Data Sharing
Open inquiry: Anything out there recommended for Android or PC?
← PreviousPage 2 of 8Next →