Here's a shoutout to all the on-calls who woke up this morning to deal with "someone else's problem". I think everyone who woke up gets to, at least, order a "fancy coffee" and send the bill to Verizon.
Edit/Disclaimer: Yes, this is a joke. I woke up to several serious alarms just as the problem was starting. Luckily, I thought to check Cloudflare’s status page from my phone around the second or third time PagerDuty called me. I saw a preliminary notice from them indicating that they were observing networking issues. At that point, I decided I’d rather watch the world burn from my bed than my computer, so I “scheduled” maintenance for a few hours and went back to bed. Our whole infrastructure had split by that point, but there was nothing I could do about it.
"Hi, I'm Teejmya, and I was on call last night"
If you email me (matthewatcloudflaredotcom) your shirt size, preference for men's or women's cut, and your postal address with the subject line:
"Verizon BGP Leak On Call Support Group"
I'll send you a Cloudflare tshirt. Least we can do.
There’s a silver lining to everything.
I went off-call at 8:30am EST. Then, while the internet burned down, I slept in and played video games.