1,402 karma · joined March 17, 2012
Previously at Shopify, PagerDuty, and Autodesk.
An on-device search engine for everything you need to remember. Index your markdown notes, meeting transcripts, documentation, and knowledge bases. Search with keywords or natural language. Ideal for your agentic flows.
It’s a (small) moving part we now have to maintain. But it’s very much worth the massive cost savings in NATGateway-Bytes.
A big part of OpsLevel is we receive all kinds of event and payload data from prod systems, so as we grew, so did our network costs. fck-nat turned that growing variable cost into an adorably small fixed one.
At the same time, in recent years, I've found that ssh running on top of Wireguard / Tailscale is way more usable than 2013 days. Those latter tools address the roaming IP issues directly at the network layer.
So while there are still issues with ssh / TCP if you're on a really crappy network (heavy packet loss, satellite link, etc), those have been less common in my experience compared to IP changes.
The “killer use case” for Mosh feels a lot less killer now.
Avery (Tailscale CEO) has actually written about IPv6 in the past:
- https://apenwarr.ca/log/20170810 (2017)
- https://tailscale.com/blog/two-internets-both-flakey (2020)
Tailscale, on the other hand, doesn’t need to wait for the Internet to upgrade. Their model sits on top of the existing stack, works through NATs, and focuses on "identity-first networking". They could evolve at the transport or app layer rather than rip and replacing at the network layer. That gives them way more flexibility to innovate without requiring global consensus.
Again, I don’t know what their specific plans are, but if they’re chasing something at that layer, it’s not crazy to think of it more like building a new abstraction on top of TCP/IP vs. trying to replace it.
A few other things:
1. Go-to-market costs
Even with Tailscale's amazing product-led growth, you eventually hit a ceiling. Scaling into enterprise means real sales and marketing spend—think field sales, events, paid acquisition, content, partnerships, etc. These aren't trivial line items.
2. Enterprise sales motion
Selling to large orgs is a different beast. Longer cycles, custom security reviews, procurement bureaucracy... it all requires dedicated teams. Those teams cost money and take time to ramp.
3. Product and infra
Though Tailscale uses a control-plane-only model (which helps with infra cost), there's still significant R&D investment. As the product footprint grows (ACLs, policy routing, audit logging, device management), you need more engineers, PMs, designers, QA, support. Growth adds complexity.
4. Strategic bets
Companies at this stage often use capital to fund moonshots (like rethinking what secure networking looks like when identity is the core primitive instead of IP addresses). I don't know how they're thinking about it, but it may mean building new standards on top of the duct-taped 1980s-era networking stack the modern Internet still runs on. It's not just product evolution, it's protocol-level reinvention. That kind of standardization and stewardship takes a lot of time and a lot of dollars.
$160M is a big number. But scaling a category-defining infrastructure company isn't cheap and it's about more than just paying engineers.
The best thing about Rails is its strong, opinionated defaults for building web applications. It handles HTTP request routing, data marshalling, SQL interactions, authentication/authorization, database migrations, job processing, and more. That means you can focus on business logic instead of wiring up the basics.
Rails isn’t as fast or lightweight as Go, but they solve different problems. For most web apps, the bottleneck is I/O, not CPU. Rails optimizes for developer productivity, not raw performance, and that tradeoff is often worth it, especially when speed of iteration matters more than squeezing out every last cycle.
(btw, big fan of Lost and Founder).
Sounds like your current/previous companies have heavier weight release processes that reduces how often releases go out?
/s
It's a big part of our check evaluation infra at OpsLevel.
I worked with a bunch of smarter-than-me UW grads after graduating.
My “how to write large systems” takeaway from that early point in my career was to focus on the interfaces between various parts. What I’d never thought about until now is that is a very data centric viewpoint.
- What system has what data?
- In what shape?
- What shape does the next system need its data in?
- Are the interfaces between these orthogonal? Shallow? Easy to grok? Tight (as opposed to leaky)?
Great article.
You tell the agent you want to opt out. They scream “Opt out!” at the top of their lungs.
Then another agent takes you through the plexiglass door beside the metal detector.
That agent recites a brief script about how they’ll pat you down and then do so.
Pat down complete, they check their gloves for residues in a machine. Once that’s done, you’re free to go.
It generally adds 5-10 mins to the security line. Sometimes, if they really backed up and you say opt out early enough, it can actually be faster.
You have the right to opt out and can exercise it.
e.g., if you're building an integration with any of the myriad of SaaS tools that fire webhooks, you can test in devlocal and have a URL of whatever.tunnel.com.
There are tools like ngrok and localtunnel that exist to do just this. I'm looking forward to replacing those with TS Funnels.
It's always a tricky balance, but I definitely sympathize with the feeling of just getting started without having to talk to a salesperson.
Backstage is super configurable, but requires that you build your own developer portal. That generally requires a dedicated team of people at your company to think about dev workflows to support, building plugins, plus hosting/maintenance.
At OpsLevel, we're focused on a much more turnkey solution. We want to get you up and running in under an hour, not weeks / months.
100%.
Solving this problem is why I started OpsLevel [1] a few years back. Everyone always starts with a spreadsheet because they're flexible.
But spreadsheets quickly fall apart because they're not complete / up-to-date, automated, or really scalable.
We figured we could automate a whole lot of collecting service info from different places (Git, k8s, other deployment streams) and keeping it up-to-date.
[1] - https://www.opslevel.com
One of our team members who previously ran a Backstage instance at $OldJob wrote about that pain:
- tankless water heater
- if you want motorized blinds, have wiring go right to the window frame. You can do a battery otherwise, but you’ll have to recharge it every month.
- solar cells and battery pack
- home automation system (if you’re into that kind of thing)
- timer switches on bathroom fans
- bathroom fan in your laundry room
- hidden safe that’s built in (eg in wall or in floor)
Not one or two, but any place you think you’ll ever need one. It’s much easier to wire these before you have drywall up.
I did this in a basement reno at my old place. Square room, had one on each wall plus one in the extra bedroom. No wifi dead zones, Xbox and TV had no latency.
(and also https://rsync.samba.org/how-rsync-works.html)
The great innovation with rsync was the idea of having a rolling checksum. It allows:
- quickly identifying which parts of a file have changed
- efficient computation (because the checksums are incremental / rolling)The recognition you point to from other countries, is, at its heart, political. Those other countries are Muslim majority and vehemently anti-Israel. Ignoring that political and religious bias is intellectually dishonest.
But, if you still maintain Palestine is indeed a state, please answer the following:
- when was its independence? - who is the leader? - what is its currency? - who are its main trade partners?
Absolutely, Israel still has more steps to take towards peace (eg, removing settlements).
Historically though, PA leadership has rarely demonstrated any honest desire for compromise or peace. Hamas, currently the government in Gaza, literally calls for the “obliteration” of the State of Israel as part of its charter.
Like it or not, 30% of the world’s countries identify as Muslim majority and part of the Organization of Islamic Cooperation.
None of those countries recognize Israel as a state.
Just because a bunch of countries believe (or don’t believe) something doesn’t make it a fact.
You're right, total rubbish. /s
Perhaps?