Hacker News Reader

dmuinoon The chess bot on Delta Air Lines will destroy you (2024) [video]

That's true, I'm 2050-2100 lichess, around 1800 on chess.com. Never played a rated tournament but played some rated players who were 1400-1500 rated USCF, and they were roughly my strength, maybe a bit better. Still the Delta bot, easy mode, was much, much better than me.

dmuinoon The chess bot on Delta Air Lines will destroy you (2024) [video]

I'm 2100 rapid on lichess, 2050 blitz and bullet. I got destroyed every single time I played the easy mode version on Delta. It knew opening theory. It did not blunder a single time in the middle game. I never made it to an end game.

dmuinoon After 24 years, Doom II's final secret has been found

Wil is a very good audiobook narrator and does a fine job reading this audiobook. I listened to it a year or so ago and it was extremely entertaining.

dmuinoon Yahoo and FreeBSD (1997)

Note that we didn't run stock freebsd. It was a custom freebsd4 with a custom gcc 2.95 toolchain. I don't remember all the mods, but migrating from 4 to 6 was a nice undertaking, and it was done in parts. People who worked there will remember the term "4 on 6". After the migration, the kernel was 6 (and you got the latest drivers) but we were still running those weird 4 user-land binaries.

Inktomi (YST) was acquired in 2003, later Overture (YSM) and both were linux shops. YST was the biggest property by far and we didn't have any kernel developers. We were doing just fine with stock debian, then stock RHEL. (I did a few very minor patches for 32-bit, like splitting user-space/kernel to 3.5/0.5G instead of 3/1G, and some caching improvements for /proc, but I wasn't a kernel developer). Linux just worked. Drivers were available and supported for the HW we needed. Oprofile, systemtap worked fine for the most part. We later hired one kernel developer to help Mail move to linux, but we always had more freebsd kernel developers.

For freebsd you needed to write the drivers and the tools. That took time and money. The community was also smaller and it was harder to hire people with experience. Acquisitions were running on linux too.

The decision was not easy. Y! already had great freebsd engineers. By announcing that linux would be the supported platform going forward it was a given that many of them wouldn't be happy and leave. Fortunately not all of them left. For example Peter W. still works there.

In any case I don't think there was really an alternative.

dmuinoon Yahoo and FreeBSD (1997)

Even when Rick was at Y! the push to move to linux was there. Rick had written most of the base libraries for freebsd originally (mdbm, ylock, ylock_kern, yfor, etc.), and tools like the package manager yinst and ysar, but being the great engineer he is, he ported them to linux and got the same or better performance. (In the yinst case he delegated the maintenance to a very capable engineer)

Note that Yahoo! was employing a few very bright freebsd kernel engineers but when the decision was announced that linux was the future many of them decided to leave. That made the path forward even clearer. It wasn't just a matter of integrating new acquisitions easier, and the fact that it was easier to hire people with linux experience than with freebsd experience. That was around 2008 IIRC. I left in 2010, and Rick less than a year later.

dmuinoon Working at Netflix

Can you share the name of the recruiter? Just curious if that person is still working at Netflix.

I've been a happy Netflix employee for over 4 years and have never experienced a 'large layoff' event. All the layoffs I've witnessed were not a surprise for the people affected.

Senior engineers usually make more money than their direct managers, but I fail to see how that's an issue.

I'm guessing RayVR met with Patty who left Netflix 2 years ago. My interview with her was different but I guess it would depend on your background and what she was trying to determine. In my case it was mostly to see if I was going to be a good cultural fit since I was at the time working for Yahoo which had a completely different culture. To me that was an entirely reasonable thing to do. Actually the whole interview process was great, and it was one of the major reasons I chose Netflix over other companies.

dmuinoon Letter From A Psychopath

I've listened to this audiobook and I was hooked very early in the story. Jon Ronson is a very good story teller and he narrates his book. He's quite funny and engaging. "A Journey Through the Madness Industry" is the subtitle, and "journey" is indeed a very adequate description of what you'll experience with this book. Jon wanders from place to place where he'll meet psychopaths and people who work in the "madness industry" and relates his experiences and thinking.

It's not a definitive treatise on psychopathy, just the adventures of the author as he dives into this fascinating world. Definitely worth getting it at this price.

dmuinoon Hack your way through Stripe's Capture the Flag

gdb works fine.

  (gdb) x/4x fns

0xffeaa0cc: 0x080485e4 0x08048640 0x0804869c 0x08048719

  (gdb) x run

0x804875b <run>: 0x83e58955

  (gdb) gdb) x/40x (void*)fns-0x40

etc.

dmuinoon Hack your way through Stripe's Capture the Flag

You can write to /tmp. But since most people are also doing that, /tmp/date gets overriden frequently. I'd recommend mkdir /tmp/CZ-18; PATH=/tmp/CZ-18:$PATH; And then you can figure it out :)

dmuinoon MongoDB rocks my world

I agree that the biggest issue is #1. Another problems I've run into are:

- Operations are not automatically aborted if the client closes the connection. Suppose a case where the client is sending many queries that are queued by the server (usually waiting for a lock to be released). Client times out, re-establishes a connection, retries, etc. Mongo will execute the operations even if the client is no longer waiting for the response. Operations are not automatically aborted.

- Sending slaveOk() queries need to be sent to the slaves explicitly by the driver, and I haven't seen it work reliably. Ideally you'd send them to mongos and let it send slaveOk() queries automatically to the slaves. All the drivers would be simpler and just work. But this is not the case. You can't do it unless sharding is enabled. Each driver needs to implement this functionality.

But I must say that overall my experience has been very positive. It's very developer friendly. You can go from not knowing anything about it to having a working prototype in an hour or so. The flexibility in querying the system is awesome.

You just need to understand the limitations of the system. Once you start pushing the limits it gets significantly more complicated.

dmuinoon Netflix kills Qwikster

I'm a Netflix employee. If you ask me I'll tell you in no uncertain terms that it's not a culture of fear. I've been here for one year and have only seen one person removed for performance reasons. Fear is never a driving factor in any of my decisions.

Employees ask tough questions during company meetings and our executives are incredibly open. I guess we've become complacent because their decisions had been bold and right before these past couple of months. I believe this is why there was not a stronger push back internally.

dmuinoon Ask HN: Who is Hiring? (September 2011)

Los Gatos, CA - Netflix

I'm looking for a frontend developer for my team. Your main job would be to help develop a very interactive infrastructure monitoring system. We have lots and lots of metrics about our systems. We'd like to be able to present them in a useful format, make it easy to troubleshoot production issues, aggregate on different dimensions, slice and dice them, zoom in on interesting events, expand clusters into individual nodes, save your work into your custom dashboard, etc.

I'm mostly looking for a strong developer who knows javascript well (or a language that compiles to javascript) and is familiar with tools or frameworks that will make your job easier (think Backbone, Knockoutjs, Spine, JavascriptMVC, etc.).

Netflix is a great environment for engineers where the emphasis is on agility and there are no rules about what tools or technologies you use to get your job done. You know what's best and we trust you.

If this sounds interesting to you please email me at dmuino @ netflix.com

dmuinoon Donut math - How donut.c works

  <resume>
    http://a1k0n.net/
  </resume>

dmuinoon The Netflix Simian Army

That's a very good point. Unfortunately is not as simple as having discrete states: 'healthy', 'degraded'. For example a tier might be deployed to an auto-scaling-group (ASG) running 100 instances. One of them is failing the health-check. Common sense dictates you can take action (kill the offending instance). What's the overall state of the ASG after this? Are you allowed to kill another instance that's having issues? How many after that? In general you want to put some rules into place. (Percentage of the ASG that's safe to kill, latency boundaries for the service, time to wait before taking the next action, etc.)

dmuinoon The Netflix Simian Army

It wasn't one of the monkeys. Most monkeys only run when we have developers who could notice and fix problems. It also happens that our peak usage is while we're home and our quiet time is while we're at the office. That means, in general, the monkeys don't run on a Sunday night.

Unfortunately we're not running 100% on the cloud today. We're working on it, and we could use more help. The latest outage was caused by a component that still runs in our legacy infrastructure where we have no monkeys :)

dmuinoon ITerm2 1.0 Released

I use it with tmux. The main reason I use it is because of its full screen support and the unixy behavior for focus follows mouse, copy on select, middle button paste.

dmuinoon Cassandra 0.8.0 has been released

The email announcing the release has some high level information:

http://mail-archives.apache.org/mod_mbox/cassandra-user/2011...

The complete list of changes:

https://svn.apache.org/repos/asf/cassandra/tags/cassandra-0....

dmuinoon Netflix Now The Largest Single Source of Internet Traffic In North America

As I mentioned elsewhere Netflix does use Level3, Akamai, and Limelight. So the traffic is mainly coming from CDNs.

I'm not sure if by "content-aware routers" you mean something like Content Centric Networking:

http://www.ccnx.org/

If that's the case it'd be up to the ISPs to push that technology to their infrastructure. Van Jacobson gave a very informative presentation at Netflix about it a couple of months ago. It seems you can watch a similar presentation he gave at Google on youtube:

http://www.youtube.com/watch?v=8Z685OF-PS8

dmuinoon Netflix Now The Largest Single Source of Internet Traffic In North America

That is mostly correct. Most of the traffic is coming from the CDNs. But we also use Akamai and Limelight in addition to Level3.

dmuinoon Lessons Netflix Learned from the AWS Outage

That's exactly what we do.

dmuinoon Reddit has now been down for 30+ hours

>I don't really see how you can have a functional DR plan for a site the size of Reddit with fewer than 10 people.

You can definitely have a good plan, and be able to execute on that plan, with fewer people. I was lucky enough to work in a team that was running the search backend for a big internet company. Tens of thousands of machines, constant code, config and data pushes, configuration management, os images, kernel, etc. The team was responsible for developing and running the tools needed to keep the site up even when a whole datacenter lost power and you had to shift load from one coast to the other. The size of that team was much lower than 10 people.

Obviously Reddit will have to invest in both infrastructure and human resources to get there, but it can be done.

dmuinoon Reddit has now been down for 30+ hours

You can do high availability deployments using AWS. AWS offers different availability zones within a region, and multiple regions if you're extra paranoid. Unfortunately the way reddit is using the available services is not ideal. IIRC they are using cassandra backed by EBS which is not a best practice. It's much better to run cassandra on local storage and have a backup system that's uploading incremental and full snapshots of your data to S3.

I agree with your overall point that having a business continuity plan is required. They should be able to answer questions like how do we minimize impact when we lose one cassandra node or two, ebs gets corrupted, a whole avail-zone disappears, etc. I'm just saying that AWS provides the building blocks for you to do that.

dmuinoon Why HN was slow and how Rtm fixed it

http://en.wikipedia.org/wiki/Robert_Tappan_Morris

dmuinoon Google Public DNS vs OpenDNS vs Your ISP's DNS: Find out which is fastest

This is not the thing to test. DNS resolution time is important, but even if the google resolver returned results faster than your local ISP resolver it would still make your browsing in the general case slower. The problem is that the IPs for the resolvers have presence in very few locations around the world. I believe this number is just three at this time (Virginia, London, Taiwan). The big problem with this is the effect it has on CDNs and GSLBs. The DNS response you get for large sites, like www.bing.com for example, depends on the location of the machine asking the question. If you use your local ISP as your resolver then you'll get an answer that's close to you. If you use the google resolver you'll get an answer that's close to the google resolver and that will generate additional latency for all the rest of the browsing session. This obviously will affect all big sites that are NOT google. They're obviously aware of the problem since they have special cases for their own sites. If you have access to machines in different countries you can see the responses you get from 8.8.8.8 have good locality properties for google sites. So I'm guessing they're just forwarding the queries to their location-aware DNS servers. Obviously they won't do that for akamai, and others.

End result: you're farther away from the resolver, farther away from your web pages, farther away from your CDN. I don't see how you can make a case that this is an improvement in general.

dmuinoon Ask HN: Who's hiring (take 3)?

If you're interested in cloud computing we have a few openings for developers. We're building a very large scale private cloud and we could use help all over the place. From the low level Xen foundation, linux kernel, and virtual machine setup and deployment, to the high level view of the system we'll present to the users (Yahoo developers).

Location: Sunnyvale, CA

Qualifications: You're a very good developer (in any language)

dmuino at yahoo.com

dmuinoon Amazon Kindle for PC Beta Launched

I leave my kindle at home most of the time. Sometimes, while at work, I wish I could take a look at a particular passage of a book. The iphone app is OK for waiting rooms, but is definitely not ideal for more than casual reading. Having a mac version would be very helpful for me.

dmuinoon Barbara Liskov Wins Turing Award

A history of CLU http://publications.csail.mit.edu/lcs/pubs/pdf/MIT-LCS-TR-56... is a very interesting read. CLU is a classic language and it was the first language that provided linguistic support for abstract data types. It also included iterators, exceptions, and a mechanism for polymorphism.

dmuinoon Lazy Linux: 10 essential tricks for admins

Tip for the author: instead of of typing while [ 1 ]; ..., type while :;

dmuinoon Lazy Linux: 10 essential tricks for admins

instead of 1 append init=/bin/sh to the kernel command line.