HNHacker News
TopNewBestAskShowJobs

bhntr3

897 karma · joined May 28, 2012

submissionscomments
bhntr3··on Bored Lonely Angry Stupid
Thank you! Yeah, I think Dunbar's number has been proven experimentally. Of course I'm probably guilty of applying it too liberally outside of the limited contexts in which the science has proven it (the pitfall of pop psychology/sociology.)

But I agree it's super useful and interesting to think about quantifying and enumerating the limits of human capability where possible. Maybe that's somewhere else we differ from the attitudes of the 18th century romantics from TFA.

bhntr3··on Bored Lonely Angry Stupid
I wrote this paper for a class at university in 2002 that relates to some of what you're saying: https://web.stanford.edu/class/symbsys205/facetoface.html

It's funny to read now because it predated social media for the most part. When I say social networks in the paper it's a theoretical concept which we might call a "social graph" now.

I didn't mention it in the paper but that class also introduced me to the concept of Dunbar's number (https://en.wikipedia.org/wiki/Dunbar%27s_number) I guess I'd TLDR it this way now: Internet connections are weaker than in person ones but they probably still take up a slot in your social graph. That graph is probably capped somewhere around 150. So it's not super surprising that when you freed up some space by disconnecting from the internet communities, you felt compelled to fill it back in with people in physical proximity with you. It's a cool personal experiment and it would be interesting if it could be replicated more scientifically.

bhntr3··on Why can’t a bot tick the 'I'm not a robot' box?
Great points. I agree the critical and biggest innovation is building a secure environment inside the browser. When I explored keyboard/mouse biometrics it was for detecting account theft which is a bit different.

If they have a way to create a secure, immovable cookie across browser sessions even in incognito mode then they don't need biometrics. In the absence of persistence, biometrics could serve as the cookie. Even with a naive approach in a hackathon, a member of my team was able to get very high precision identifying users based on a small sample of keyboard and mouse movements. I'm sure Google can do better.

So it's not really about attackers being able to look like any human. It's about being able to look like a specific human. Which is much harder.

But maybe you have more experience? We abandoned it seemed intrusive and because we knew we couldn't invest in the secure environment. Without that it doesn't matter. And with it maybe there's an easier solution. But I figured that given Google made it that they would be using keyboard/mouse movements for user identification.

bhntr3··on Why can’t a bot tick the 'I'm not a robot' box?
I haven't seen any mention of keystroke (or mouse) biometrics (https://en.m.wikipedia.org/wiki/Keystroke_dynamics) When the checkbox appeared I assumed that was what it was.

Keep a history of the biometrics of the devices either on the account or in a cookie. Use a sufficiently secure and obfuscated language to capture and upload the data (maybe using steganography in an image, explaining the weird image uploads.) Prompt when the biometrics don't match a known user of the device/account (according to ml). Or if the tracking data didn't exist (incognito) then prompt using a backup model trained in the difference between the biometrics of known bots and known humans. Keep a very loose threshold here.

That's how I'd do it if I were Google (and no one else because only Google can afford that.) All the browser fingerprinting stuff mentioned is great but doesn't really work as much as you'd hope in practice.

bhntr3··on Bill Gates promises to add his own billions if Congress does nuclear power push
> the public is skeptical regarding fission for good reason: history has not been kind to the kind of disasters fission reactors have wrought.

It's not just that! I was pro-fission until someone educated me how horribly irresponsible we've been about our nuclear waste disposal. And how dangerous it is.

Deep waste storage has been deadlocked in government since the 80s so waste is just hanging out by each reactor. If this funding and nuclear push doesn't include just finishing Yucca mountain, then imo it's a bad idea.

It looks like some progress was made recently but given that it took 40 years to vet yucca mountain I hate the idea of more nuclear power concurrent with a restarted site selection process.

http://sitn.hms.harvard.edu/flash/2018/looking-trash-can-nuc...

bhntr3··on How to Be Successful
https://www.theguardian.com/education/2018/jun/01/famed-impu...

Unfortunately the marshmallow test as a predictor of later success is debunked. It's part of the replication crisis in psychology. Half of the psychology we learned may be wrong. People just aren't that straightforward.

https://www.theatlantic.com/amp/article/576223

bhntr3··on Au Revoir
> There is nothing inherently special about someone who runs a small vs. big company, except for experience.

I think the question is, given that someone can do one of those things really well, how likely is it that the same person can do the other really well too?

It seems obvious that the kind of traditional business ceo that investors often want to find to replace a founder is not going to be successful as a starter. Let's just accept the common assumption on that one.

I'd argue that what makes a great startup founder often makes a terrible leader for large corporations. I think every great founder has a reality distortion field (https://en.wikipedia.org/wiki/Reality_distortion_field) which projects their vision, keeps them pursuing seemingly bad ideas in the face of insurmountable odds and allows them to overcome the biases and assumptions that dominate so many business decisions.

Some leaders expand and adapt this to build great, crazy companies: Steve Jobs, Elon Musk, Zuckerberg (maybe). Often they also create a cult-like, maybe toxic, certainly messy corporate culture that still somehow succeeds.

Others can't do this: Travis Kalanick, Elizabeth Holmes, Parker Conrad. They let their "startup founder, growth at all costs, thumb your nose at the rules" attitude destroy their company culture and embroil them in ridiculous, easily avoidable legal issues.

In my experience, a corporation's culture is the manifestation of the most extreme personality traits of its leaders. There is a time, when crossing the chasm, that the exact traits that made the company grow rapidly and the founder extremely successful are the ones you want to suppress to create a stable, efficient and effective collaborative endeavor involving thousands of people: a corporation.

I think (and this is due to some reflection on my own ideas) that if you reject the idea that there comes a time when a company should abandon "burning vision" in favor of stability, process, predictability, and broad appeal, you're actually rejecting the idea of a corporation as a hierarchical system of power deployed to efficiently organize the work of thousands of people. And I'd probably agree with that. The traditional corporation needs a traditional ceo but it's a damn shame that every revolutionary startup has to end up as a traditional corporation.

bhntr3··on The Design of Software is a Thing Apart

  import cPickle
  def transform(data):
    with open('my.pkl', 'r') as f:
      model = cPickle.load(f)
      return model.predict(data)
bhntr3··on F.C.C. Repeals Net Neutrality Rules
About the wireless approach, Monkeybrains (https://www.monkeybrains.net/residential.php) in SF seems to do this. I believe I read somewhere that they're planning on replacing the wireless stuff with fiber for areas where they have high customer density. They're awesome, the service is good (except when it rains. Thanks california), it's cheap ($35) and I've been pretty happy with it.

I wonder if that could work to bootstap an isp. Obviously the wireless thing won't work in less dense areas and is subject to weather but maybe a successful business in the city could provide enough capital to expand to the suburbs.

bhntr3··on Samsung DeX
Yeah, what's the problem this solves?

Here's one it could (but doesn't quite) solve.

At my office everyone carries around their laptop and is constantly trying to get it to work with the telecomm and video software. Some people have a work phone, a work laptop and a personal phone too. I even know some people who add a tablet to that mix. Sometimes I see them bouncing around between all those devices.

For most employees (not necessarily coders) it probably makes sense to just have one device: a phone that you can edit excel and powerpoint on. A universal dock should exist in every meeting room and on every desk.

The major impediments to this are currently the power of the device, the ability to use the os for actually making things, and the existence of a decent dock.

With this dock and the power of the latest generation of phones, we're just stuck on the fact that no one has been able to make an os that actually works well on both mobile devices and as a keyboard/mouse driven system for creating things.

Yeah, I'm looking straight at you, Windows 8.

bhntr3··on Is AI Riding a One-Trick Pony?
Is life riding a one trick pony?! (Evolution)

The title is just clickbait. Finding the simplest formula or equation for a process or phenomenon is the goal of a lot of scientists.

Backprop may not be that simplest equation. But actually finding that "one weird trick" to intelligence will in no way be a bad thing when it happens.

bhntr3··on A new kind of map: it’s about time
Yeah. This is showy and really cool. But it seems kind of unnecessary to make it "a map"

In terms of solving real problems, I don't see it as functionally any better than a list ordered by distance with a small compass showing directionality. Basically what you see on the left when you search google maps for "pizza in san francisco".

The benefit of that list is that it uses space much more efficiently and can show things like ratings and thumbnails. Whereas in the article's ui, you have to click on every circle to see what it is.

Put another way, maps are incredibly information dense but when you remove most of the information from them, they're just dense.

bhntr3··on Let's Remove the Global Interpreter Lock
Yeah. Python is pretty standard now. To implement high throughput scoring on models written in python, I have to run multiple processes with one copy of the python model for each process. For large models like random forests, this can eat up a lot of memory.

Ideally, it would be a single model in memory with access from multiple threads. But that won't work right now cause GIL.

bhntr3··on OpenAI at the Dota 2 World Championships
This was fun. It's our intern's last day on the ML infra team here. And he happens to be a competitive collegiate DOTA player. We were all crowded around watching the screen, shouting. Couldn't have planned a better send off. Nice job, OpenAI!
bhntr3··on Why We Chose Typescript
https://danluu.com/empirical-pl/
bhntr3··on The Holder Report on Uber
Yep. I asked about work life balance when evaluating offers and my future manager told me "Just look at the delta between when breakfast ends and dinner starts. For us, it's 8 hours. For Uber, it's something like 11."

It might be an inconsequential recommendation. But it is representative of the broader culture.

And yes, it's absurd that silicon valley tech companies feed us three meals a day. That HBO show Silicon Valley is a documentary not a comedy.

bhntr3··on The Minimally-Nice Open Source Software Maintainer
Best blog I've read in a long time! <3

> By consistently exhibiting a few simple behaviors, one can at least look like a kind and decent person. Maybe someday we all actually will be.

The best thing about this is that we will! Mindset follows behavior.

P.S. Am I doing it right? I'm actually terrible at this. But I wish I weren't.

bhntr3··on Why Japan’s Rail Workers Point at Things
You're getting teased a bit but I think this is a very smart thing to do.

The automatic processes of our brain dominate the majority of our behavior. If you're a deep thinker who often ties up his attentive thread thinking through a tough technical problem or something, you need to prime your automatic process with some surprise that triggers the association.

I think Kahneman's Thinking Fast and Slow does a good job explaining this. But I also like the idea of the "elephant and the rider" from Jonathan Haidt. This quora post (https://www.quora.com/What-is-the-rider-and-the-elephant-met...) gives a nice summary but I highly recommend his book.

Bringing it back to TFA, I think the Japanese as a culture might have a better relationship to the frailty of human rationality. The enlightenment was a great period in Western history but it cemented a belief in the power of reason and the universality of truth.

The Japanese have a longer history of training the mind through repeated action, visualization and behavior. Western cultures just want to power through it with rationality. So, we don't point at stuff repetitively and repeat things out loud to ourselves even if it would save lives. Kind of a bummer.

I dig your upside down tomato can, man.

bhntr3··on Why Japan’s Rail Workers Point at Things
People sometimes tease me for reviewing my own pull requests.

But I am a big believer in roles. The reviewer role is different than the creator role. When I mentally switch, through the priming mechanism of my code review tool, I am often able to see the defects in my own design or code.

I sometimes do the same thing with design docs or proposals by imagining myself as the person who has the most to lose by my proposal succeeding and then commenting on my doc from their perspective.

bhntr3··on Trolls are winning the internet, technologists say
Yeah, the meaning has shifted. It used to be that if people knew you were trolling, you weren't a good troll.

Now a troll is just an asshole on the internet.

bhntr3··on Oh, you’re with them? (2016)
Option 3 is to re-train. Every time we see a woman of color think to ourselves "She's probably a powerful executive" and every time we see a white man in a suit we think "he's probably an executive assistant." Like a visualization technique.

The reality is that we're all biased but we're also malleable. If we understand how our biases manifest, we also understand that biases are pretty resilient to reason and goals. They need to be re-trained or manipulated (like through priming and other bias inducing mechanisms.)

This is what bothers me about unconscious bias training. They show that everyone is biased. But the solutions fail to acknowledge that the system can't change just through positive intention.

bhntr3··on Ask HN: What does your production machine learning pipeline look like?
When I joined Airbnb, we did something similar with PMML on the risk team. The issue I had with PMML is that sometimes the scores would differ in weird ways due to the differences in the model implementation. Also, it limited what models we could use.

I moved to our ML Infra team and built a service to allow inference on the pickled sklearn model or serialized TensorFlow model. This also worked with our ml framework: Aerosolve. (If you're using TensorFlow only then you can use TensorFlow serving.) One of the primary goals I had was to make everything config driven so data scientists could launch models without engineering help. This has worked pretty well. We've put about 20 models into production on this service in the last year.

For batch scoring, the solution is similar. We use Airflow for scheduling and can just do inference on the model in the airflow worker. It's easy enough when it's python or spark because Airflow plays nicely with those.

Some complications occur in:

1) Dependency management. We let users package up their own virtualenvs and use them in both the real time scoring and batch scoring. But it's a bit clunky in certain cases and we're moving to docker.

2) Heterogenous resource requirements. If you need a lot of memory or gpu access then it can be a bit tricky to do that with Airflow. We're working on integrating it with docker and kubernetes to solve this at the moment. Typically we use yarn for problems like this but our ML use cases require some things that are just hard to do on our hadoop cluster. I'm not a big fan of yarn for ML. No one has needed GPUs for real time prediction yet so I'll cross that bridge when we get to it.

3) Online / offline consistency. Sometimes we'd end up re-implementing transform logic in Java so we could run it in production. This led to lots of sadness when things didn't quite match up. Now, we try to make it easy to deploy the whole sklearn pipeline including custom python transformations. That way we have some confidence we're running the exact same code at training and inference time. Similarly, we built a framework to compute offline aggregations in hive and upload the results to a KV store for use in online scoring. We have the ability to augment this with streaming data to reduce the data delay on the aggregation. This introduced some lambda architecture requirements that were kind of interesting. We actually ran into quite a few scaling issues here so we've gone through a couple iterations as new projects come in. I'm currently working on the third generation of this framework.

These are some of the issues we tackled early on with the ML Infra team. We're now focusing on improving the quality of our offline training tools. Particularly we're trying to make:

1) A seamless process for going from a notebook to production that integrates all these tools.

2) A good notebook environment that includes simple provisioning of resources for large training jobs.

3) Sample notebooks for common ML tasks that encode best practices and make it easier for people to get their initial model quickly.

4) A system for making it easier to do complex training data backfills.

Outside the ML Infra team we have a few ML focused product teams that own their own infrastructure and optimize it for very specific needs they have.

bhntr3··on Why Databases Are Not for Docker Containers
Hadoop and HBase are very different from mysql. Those run on yarn and are designed for containers.

> Any regular relational database should be pretty close to automated failover

In my experience, most people who work with mysql would not enable automated failover. And I believe the concerns in the article are valid and important if considering a container for mysql.

Edit: Though I do think it conflates containers and things like kubernetes or mesos in an awkward way. The good arguments are more about running relational dbs in containers on some sort of cluster orchestration system.

bhntr3··on Lessons from 3,000 technical interviews
I wonder if "took courses" could be a stand in for "prepared heavily". It seems like people with all the other attributes might think they didn't need to study. People without them might think they did and took courses to "catch up". In my experience, preparation is the key driver of performance in these types of interviews.

It seems reasonable that a person who took a MOOC might have prepared in other ways as well while people who didn't probably didn't prepare much at all (since watching a few Algo lectures seems the most accessible refresher.)

bhntr3··on Building Jarvis
> a grey t-shirt cannon would require hardware modifications to work

This almost felt like satire but in the best way possible.

bhntr3··on Waymo: Google's self-driving car company
What happened to worst case analysis? Don't we all know the flaws of best case?

I think (and I could be wrong) that there are many more cases where conservative driving is safer than aggressive driving.

In a lot of the cases mentioned, doing nothing or stopping are the safest choices. What's safe to drive over? Just assume everything is unsafe and only drive over it if it puts others in danger to avoid it.

If the road is blocked and you need to communicate to navigate, don't. Wait.

If the road conditions are changed due to construction in ways the car can't understand, stop. Find a new route.

Most humans wouldn't do these things. The nice thing about the autonomous car is that it can drive extremely cautiously.

bhntr3··on Waymo: Google's self-driving car company
My scuba instructor shared this piece of wisdom: "You don't have to swim faster than the shark. You just need to swim faster than your buddy."

People get this stuff wrong all the time. Especially when they're drunk, tired or texting. These cars won't be perfect but I think pretty good and conservative under uncertainty will be better than human, drunk and overconfident.

bhntr3··on Data Wrangling at Slack
I've looked at the code and messed around with arrow. It seems like a performance optimization that solves a small sliver of the problem. It could help with the parquet/thrift version issues they mentioned. But I don't see any guarantee it won't introduce its own version and compatibility problems. If the initial implementations are buggy like described in TFA it could actually be a lot worse.

In general, I've learned to be skeptical of any new big data solution. Hadoop and hive are clumsy but as someone on my team said "they've found and fixed the tens of thousands of bugs".

It seems to take five years before any significant new solution is stable and reliable enough to be used on large, complex workloads.

Which makes me really uncertain how we get out of this situation. Maybe something like arrow is a silver bullet that fixes everything with minimal complexity and thus few bugs. But I'm skeptical.

bhntr3··on Data Wrangling at Slack
Seems like a pretty typical set of problems. Dependency conflicts hard. Schema evolution hard. Upgrades hard.

The big data space still feels like an overengineered, fractured, buggy mess to me. I was hoping spark would simplify the user experience but it's as much of a clusterf*ck as anything else.

How hard can fast, reliable distributed computation and storage for petabytes of data be? He said ironically.

bhntr3··on It Takes 6 Days to Change 1 Line of Code (2015)
But will anyone change it again? How many times would they have to change it for this to be better? Maybe the developer should build a UI so the business owners can change it themselves. After all, business people editing config files directly on production machines is dangerous. Clearly the conf file should be in source control and subject to code review. And so on and so forth. Reasonable changes accrete.

The real question is: What is this policy and process protecting against? Does it work?

We're all risk averse. Bad things happen. We come up with processes and systems to prevent them. We write our postmortems and say "How could this have been prevented?" and we say "Heavyweight process X and automated, difficult to maintain, high overhead system Y" and everyone nods their head sagely and makes plans to implement X and Y. Then we make a new mistake and postmortem and build process Z and system W.

But you know that old saying "Lightning never strikes the same place twice"? Well, it's wrong. Sometimes it does. Especially over a long enough timeline. But it's pretty rare. We're the soldiers preparing to fight the last war and losing the next one.

Personally, when I see stuff like this I always think of https://medium.com/@webseanhickey/the-evolution-of-a-softwar...

It's a cartoon. But I'm getting old and angry. 15 years in, I think that most of this stuff that we do, these pedantic code reviews, processes, policies, best practices and test plans we're always beating each other over the head with, they don't always pay for themselves. We still write bad software. We still make stupid human mistakes.

There is no answer but good judgment, trust and respect. You need as much of it as possible. I think it's easier if you believe Clayton Christensen and build protected small teams that can operate in a limited bureaucracy and limit the connectivity between them and groups viewing them as "others". But again, trust, respect, autonomy and good judgment.

Or just make shittons of money. With lots of money, you can afford to hire lots of smart people to do stupid stuff and take forever doing it.

← PreviousPage 3 of 6Next →