HNHacker News
TopNewBestAskShowJobs

trjordan

5,143 karma · joined October 8, 2008

Catch the decisions your agent didn't tell you about https://tern.sh

tr at tern dot sh

submissionscomments
trjordan··on Modern network load balancing and proxying
This is a great introduction to a lot of the problems Envoy is trying to solve. If you're interested in learning more, the rest of the Envoy blog [0] is great. Coincidentally, I also spent most of last week putting compiling [1] all the posts we could find on Envoy, smart proxies, and modern load balancing.

[0] https://blog.envoyproxy.io

[1] https://www.turbinelabs.io/resources

trjordan··on Against the synchronous society
It does happen. For example, Game of Thrones.
trjordan··on Ask HN: Huge enterprise customer wants to see our source code
Another possibility is they're worried you'll go out of business. In that case, they don't need to see the code right now. You can set up a code escrow agreement in case you cease to exist.
trjordan··on How we saved money by replacing Mixpanel with BigQuery and K8S
Just to flesh out the vendor comparison...

Five weeks of, say, three engineers with $150k salaries ($200k fully loaded) is $60k. People say that the majority of software is after initial development. Let's say 20% is development [0]. That means this project costs $300k total and lives for 5 years before the team decides to rewrite it. That brings us to $60k / year.

This is good savings! My general guidance to folks thinking about this sort of project is that you should implement a vendor and see how you like it. The reason being, MixPanel does a LOT more than this tool, and buying a vendor can help you figure out the subset of features that you actually need. Once you have a clear requirement list, you have a clearly scoped project and can hammer it out relatively easily to get that cost savings.

Beware of thinking that you can do this for $X,000 / year on your own, or get it right on the first shot. There's a reason MixPanel (and any other vendor) has so many engineers. You're paying them to have already made the mistakes that you're about to make.

[0] https://stackoverflow.com/questions/3477706/development-cost...

trjordan··on Ask HN: What are the big differences working at a startup, bigco, old smallco?
Other comments have good observations about big vs small.

One of the main markers of a startup is there tends to be One Big Challenge to get the company to The Next Level (generally talked about as a funding round, but that's just a convenience). Initially you're finding product / market fit, then you're figuring out a scalable go-to-market, then you're scaling the product, then you're launching a new product, etc.

It's weird to go through one of these transitions, because you'll need to change your focus pretty drastically. Sometimes you'll feel sidelined, because your department isn't the one primarily on the hook. If you're figuring out how to build a sales team, engineering gets chill and potentially complacent. The most successful startups have leaders that paint a clear picture that gets all departments working autonomously towards the same goal.

This is different than a smallco, because without a war chest of cash, your options are more limited. Because cash flow dictates the cadence of the business, and that's never going to change. This can feel more stable, but does mean you'll have to occasionally make hard choices that delay a valuable project or work on something outside your speciality.

trjordan··on My Apology to Naomi Wu and the Make Community
I'll upvote, because we're all on a journey here. But that weird and offputting feeling a feeling that, more often than not, comes up when we're talking about women. Only women. And that's the problem.

Here's a counterpoint: https://twitter.com/aphyr

I can't speak for all the harassment he may or may not have endured, but it's not on the same level as Wu. He makes fun of his weird pairing of interests in his featured photo. He doesn't have the CEO of a company and de facto leader of his professional association questioning his competence on Twitter. Whatever he's dealt with, it probably hasn't been "this is so offputting that I'm going to question the fact that you even exist."

Honestly, I had the same reaction as you. I don't think it's OK to question people like Naomi in this way, because I'm pretty sure I'm doing it unevenly. It's something I'm trying to figure out if I do elsewhere, and how to improve. This stuff is hard.

trjordan··on Non-Consensual Intimate Image Pilot
There was an excellent Twitter thread on why they didn't do this: https://twitter.com/alexstamos/status/928740488395608065

The salient bit:

> A quick note to everybody who says 'calculate the hash locally':

> A) Photo fingerprinting algorithms are usually not included in clients to prevent the development of circumvention techniques.

> B) Humans need to review to prevent adversarial reporting.

Further down the thread, they discuss further that PhotoDNA and similar algorithms are, in particular, not resistant to being tricked. Until they are, it's another layer to prevent adversaries from fully decomposing how they're hashing it, which is a good thing.

It's a trade-off, for sure, but this is a hard problem with no cut and dry answer. Facebook already has the images in question, because you have to be sending them to somebody in order to mark them as NCII. Asking yet another Facebook service to look at them without storing them seems reasonable to me. You may disagree.

trjordan··on Ask HN: What are top promising technologies of tomorrow must get involved today?
Kubernetes -> Envoy -> Istio.

Yeah, yeah, it's not rockets to mars or whatever, but I think we've spend DECADES trying to figure out how to abstract away the lower parts of the stack. The last big leap was AWS, and it forced a choice for a lot of people: do you want to work at AWS? or do you want to work on applications that run on AWS? The API is so strong that you can't do both.

There's a similar thing around operating systems, networking tools, and containers. Heroku was a cool PaaS, but ultimately is a little too inflexible to work for everybody. Companies keep rebuilding these internal PaaS systems, and I think Kubernetes has finally nailed it. It's lacking the primitives to do traffic management and introspection, and Envoy / Istio provide those primitives.

Many people see these technologies as an endgame to themselves, but the Istio community has started talking about how they're a platform themselves, and that feels correct. Like AWS, you can abstract away a TON of this stuff, and you will divide people into those that keep a cluster running vs. those that build apps on top of it. The specialization gains will be on the same level as dynamic infrastructure, imho.

Disclaimer: I work at a company that wants to build on top of this new world, but I joined them because I believe this, not the other way around.

trjordan··on Ask HN: How did you significantly reduce your AWS cost?
Tactically: spot instances, reserved instances, auto-scaling groups, deleting data that we didn't need.

Strategically: I've only ever see it work by making it somebody's job to save money on Amazon, ahead of feature velocity. Find a tool that analyses where you're spending [0], pick the biggest bucket of spend you don't really understand, and drill down until you're sick of looking at it. Make some optimizations around how you're spending that money, rinse, repeat.

Every org I know has done a first pass that's pretty effective, where you buy a bunch of reserved instances for the compute power you need. A couple of the companies I've worked with stalled out at that point. The others figured out real cost / benefit models for real projects and did things like move data from S3 to Glacier, or from hosted MySQL to RDS, or from Cassandra to DynamoDB.

There are only so many free-lunch cuts you can make, so it's worth your while start down a path that allows you to consider trade-offs, which includes empowering somebody to lobby for them.

[0] Seems nice, haven't used it: https://github.com/Teevity/ice. I know the folks at http://cloudhealthtech.com, they're building a pretty solid business if you're willing to spend money to save money.

trjordan··on Ask HN: How can I get my co-founder to stop coding?
There is probably something else here that you need to talk to him about.

Why does he feel the need to code up a feature right now, without talking to you about it?

Why is he having trouble keeping everything about the product in his head, with looking in the code as the only fallback?

Why sort of gaps are there in the demo that looking at the code is the best way to address it?

This is not a problem with him, though it's also not a problem with you. It's a problem with the business where the founder who talks to customers feels he has to come back and bang out features. Worst case, he's just bad at his job, but the likely reason for this is that there's a mismatch between you two in terms of stability, urgency, or prioritization.

You're adults. Talk it out.

trjordan··on Don’t Build a Startup, Build a Movement
I saw a great talk on this topic by Dharmesh Shah, founder of Hubspot.

He says that movements are great, but they're necessarily harder to build. Convincing people to change the way they approach their job is _hard_, even if you're right. Then you have to sell them software, which isn't trivial.

Their competition, Marketo, didn't build a movement. They latched on to the existing trend of Marketing Automation, and they built a huge business, at least as successful as HubSpot.

In both cases, the software and ideas were new, but as a company, you don't have to be the standard-bearer for your movement. You just have to support it.

As others in this thread have argued, you don't even have to have a movement to sell software. But fundamentally, if your software doesn't enable people and companies to work differently, you haven't built something deeply valuable. Outside of the early adopter SF-product-hunt folks, nobody wants to buy new software just for kicks. People buy software to change and upgrade their workflow. You can call it a movement or a trend or a wave, but it's always about change. If you want to build something enduring or big or valuable, you need the wind at your back, bringing a useful change to your customers.

trjordan··on Are you built to be an employee?
Startups are wonderful, freeing experiences. You can build exactly the company and culture you want, balancing so many of the things that you didn't have control over at a bigger company.

Then you get customers.

Customers don't care if you're burnt out.

Customers don't care if you're transparent and fair to your colleagues.

You get grace periods when you're first starting out, or first signed a couple contracts, or after delivering a few key efforts.

But at the end of the day, customers are where the pressure comes from, because they are the sustaining force of your business.

trjordan··on Why Isn’t It a Crime to Kill a Cyclist with a Car?
As pointed out in the article, it can be a crime, depending on jurisdiction and local laws. The problem is prosecuting that crime in the same way you'd process a murder with a knife, because the public doesn't see them the same way.

Honestly, I get that. Driving a bike on a road with cars seems dangerous. I do it, and I don't want to die, but it's not the same as walking on the sidewalk or sitting in my yard. If there were bad choices made by the driver, then sure, let's prosecute. But passing a biker going 15mph on a 45mph road doesn't seem crazy. Going 72 in a 65 doesn't seem crazy. I like to think I'm an aware driver, and I haven't hit any bikers yet, but I've seen myself be two distracted moments away from hitting one. It's not crazy that I would be checking my maps app at the same time a police siren comes into view, leading me to miss the biker who'd just turned into the lane next to me.

The problem seems to me that we've built both physical and social structures around roads being for cars. It needs to feel comfortable to bike while taking up a whole lane, both for bikers and cars. ("Drive at the speed bikers go" isn't reasonable, imho.) Bike lanes help a lot, and having a lot of bikers around helps a lot. But we can't just start with "killing a biker is deliberate murder", because it's simply not. It's old attitudes being applied to a new world, and I'd rather find a way to bring people into the new world than punish a few of them for making a mistake.

trjordan··on iOS Privacy: Easily get a user's Apple ID password, just by asking
The point is the OS manages your fingerprint, and even if you have it, you can't easily send a stolen fingerprint to the OS.

If you have access to the hardware, that's a whole different attack vector.

trjordan··on Facebook must kill its news feed. Democracy depends on it
OK, so Facebook kills the home feed tomorrow. What replaces this?

Human-curated news? Either it'll be fast enough that people get sloppy with facts, or it'll be slow enough that people move to faster sources. Clickbait wins.

Another site? Most other sites (Google News, Quora, whatever) use clicks as an important measure of interestingness. Clickbait wins.

Broadcast TV? Have you seen what the networks do when the facts are scarce and the topic is urgent?

Eventually this all gets sorted. In the moment, fake news always wins, especially if there are organizations that are invested in producing it. That's fine. The trick is to help real, verifiable news and level-headed analysis propagate as it comes out as well.

It's a solved problem via reputation, but we've collectively decided that reputation isn't that important in the last 15 years. It's an open but in-progress question for ML / AI. There are probably other models, but we'd first need to change the consumption norms to make them appealing.

Clickbait is always appealing, so it wins when people are trying to find new information before they start to sort through it.

trjordan··on Google Pixel 2 and 2 XL announced
Yep, dropped my phone (totally my fault), got a Pixel XL for $89. Pretty OK, would buy the extended warranty again.
trjordan··on [dead]
It's easy to be glib about this, but this is actually pretty interesting to me!

This trope shows up in so much sci-fi because it's a pretty quick way to undermine any philosophical questions. What's my purpose? To help run this simulation. Maybe I'll live long enough to find out when the outside computer shuts us down. It's never bothered me personally, because there's not a lot to be done about it. On some level, though, it's a though experiment that I didn't have a good answer to.

Of course, it's still unprovable: the outside simulation might have oodles more atoms than our universe, or different physics, or magic! But that's fundamentally less interesting than knowing there's a "higher" universe with roughly the same physics. That looks much less likely at this point.

It's really cool to read an argument that so strongly disproves something I thought was untestable!

trjordan··on A successful Git branching model (2010)
I like feature flags, but it's crazy to suggest that they totally mitigate your risk. Some features will end up with more lines of code hiding the feature than actually implementing it. After it's released, you then have to unwind all that code. That code churn isn't trivial, and "tests pass, ship it" isn't really a good model if you're shipping blocks of work that you don't want to expose to customers.

If your releases processes are perfect, then it's OK, but CI tends to treat release as "if the tests pass, it's perfect" and ships it to all customers. At the very least, you need to be able to compare releases that "don't change anything" to make sure you're not breaking something because some edge of your feature didn't get hidden behind the flag.

trjordan··on Three Paths in the Tech Industry: Founder, Executive, or Employee
I did that, sort of. But I was an exec(ish) before I founded a company.

I think founding a company helped my career, but not necessarily faster than working on a similar exec-level track for 2 years.

Reading this article was interesting to me. I really bristled at the descriptions of founder and employee, but exec feels comfortable. I really enjoy owning a set of responsibilities that's not redundant with anybody else in the organization, and I enjoy working on internal relationships in order to help everybody get their work done better ("politics").

Maybe at some point I'll found another company, because that pressure really does force you to confront the weakest parts of your professional skillset. I learned a ton about myself by doing it.

trjordan··on HAProxy vs. Nginx for load balancing (2016)
There are 4 Discovery Service APIs that you can implement: service (SDS), cluster (CDS), route (RDS), and listener (LDS). Colloquially called xDS for all of them.

https://envoyproxy.github.io/envoy/intro/arch_overview/dynam...

trjordan··on HAProxy vs. Nginx for load balancing (2016)
You don't call Envoy to make changes. You either update the config file, or you implement an API server with a specific set of endpoints, then have it serve the updated values back. It's a bit tricky, but entirely doable. https://envoyproxy.github.io/envoy/intro/arch_overview/dynam...

(You can do this in nginx, too, and it's similarly annoying. This is a big part of what we do at Turbine Labs -- a UI with a slider between blue and green, and a place to host all those stats, so you don't have to manage it!)

trjordan··on HAProxy vs. Nginx for load balancing (2016)
turbinelabs.io is the whole of everything we're building on top of nginx/envoy. Our primary goal is to provide more powerful ways to release software like incremental blue/green deploys or canaries or production previews. That requires a solid data plane (nginx/envoy) and management plane (missing in nginx/envoy xDS APIs) underneath it to work. Nobody runs nginx because it's fun -- one of the logical use cases for a better nginx is better releases, which is what we work on.

The transition has been good! There's certainly a lot more complexity to implement with the Envoy APIs, but that's because there's a lot of power. Working with Matt Klein and the rest of the community has been really positive. We're excited about all the other patterns that unlocks, like circuit breaking and tracing. Those sorts of features are on the wrong side of the cost/benefit equation for most companies, but if you get them for free because you deployed Envoy as ingress and service mesh, they're great to have.

trjordan··on HAProxy vs. Nginx for load balancing (2016)
No kidding! I'm glad it exists, but Envoy seems like a much more legit / stable choice.
trjordan··on HAProxy vs. Nginx for load balancing (2016)
Do you know if it's been deployed at scale anywhere?
trjordan··on HAProxy vs. Nginx for load balancing (2016)
I'm glad this has 2016 in the title! It's now 2017, and we have better options.

Nginx has been the best routing and traffic serving system for a while now, but it's showing its age. In addition to hiding metrics, HTTP 2 support is difficult, there's no clear path to grpc, and (as mentioned in tfa) there is no way to get stats out of it.

Envoy (envoyproxy.github.io) seems like the next thing. Nginx won because of the predominant patterns at the time, and it's still totally fine for serving lots of HTTP traffic. The problem is that if you're thinking microservices or rely on metrics to do your job or have elastic infrastructure, Envoy's feature set is the only real way to handle traffic routing in that world.

Disclaimer: I work for turbinelabs.io, and we're moving from nginx to Envoy to take advantage of a lot of these features to power our product. We've put a lot of effort into that decision, and believe it's the right one.

trjordan··on HAProxy vs. Nginx for load balancing (2016)
We at Turbine Labs have been contributing to Envoy recently, and it now builds on MacOS!

https://github.com/envoyproxy/envoy/issues/128

trjordan··on Giving you more characters
Good.

Twitter needs to keep moving forward. There have been several features that feel like the core of their platform, but based on their revenue growth, they may be holding them back. Personally, eh, 280-length tweets feel weird and long and I don't see the need, but I'll probably get over it. If they can change how Twitter is used open up new audiences and conversations, this is good.

They are trying something big, and that is Good.

trjordan··on Let me introduce: __slots__
Since I don't think it was explicitly mentioned, this is because __slots__ is effectively a named tuple. The size reduction comes from dropping the keys in the dictionary (they're stored once on the class, not every time on the instances), and the speed reduction come from it being an array lookup instead of a dict.

It's also a great way to introduce some static typing. No more setting mistyped class attributes without errors!

trjordan··on Painless NGINX Ingress
I think this blog post is one turn of the crank away from a truth we're all about to learn: don't hand roll your own Kubernetes ingress.

Dealing with the traffic handling between your users and your code is not a trivial problem. Like all good ops problems, you can fix it with good tools, deep knowledge of those tools, fine-grained observability, and smart people running all that.

This has been the recipe for a couple of really successful SaaS offerings. Individual servers? Datadog. CDN? Akamai / Fastly.

Disclaimer: I work at one of those companies, Turbine Labs, and we're trying to make ingress better. Here's a presentation from our CEO on Kubernetes ingress, and why the specification creates the problems that this blog post is trying to fix. https://www.slideshare.net/mobile/MarkMcBride11/beyond-ingre...

trjordan··on Who Owns Work, and Its Future?
One of the dangers of writing structural critiques like this is that it's easy to feel like the problems are structural in a way that makes individual effort irrelevant. On this particular topic, I think there is a huge amount of individual latitude to attack the problem.

I'm lucky enough to work on problems that I find interesting, important, and personally challenging. I optimize for this at the expense of salary and stability. One of the biggest things that's helped me in a cultural safety net of believing that this is OK, with a network of people that help me out. I think there's also enormous potential for individuals to reinforce the values they think will define work in the right way.

- Can you support your friends when their companies attack their work/life balance? Can you help them come up with ways to fight back without hurting their careers?

- Can you share information that's not typically public, in order to help others make better decisions? Salary to peers is an easy place to start. Harder is telling younger folks what Sales Engineers and Product Managers do (and how it can be really fun, fulfilling, and lucrative!).

- Can you normalize behaviors that benefit workers at the expense of companies? Unions used to be great, but I personally think they're the wrong tool for the problems of 2017. Working from home 2 days a week shouldn't break any team, but it's still a cultural taboo.

I'll admit that these are pretty 1st-world problems and I can't do much for people outside of Bay Area tech, but Bay Area tech is where I live and work. We can't change where we are right now, but there's a pile of small problems that are right on your doorstep that you can shove in the right direction. I think there's more ground to be won there than trying to overhaul the whole system.

We overestimate what we can do in a year and underestimate what we can do in a decade ;)

Aside: I disagree somewhat with the framing as a Wicked Problem. That method seems to deliberately conflate multiple problems, in order to make the discussion more sweeping. I think there's value to breaking down "Work" into smaller, more tractable problems. That aside, there's a lot of meat on this bone, so it's good to acknowledge that.

← PreviousPage 7 of 20Next →