Yahoo’s fantasy football debacle – and why it was being compared to Obamacare | Hacker News Reader

Yahoo’s fantasy football debacle – and why it was being compared to Obamacare

washingtonpost.com

33 pointscirca14 comments

I was one of the lead devs on ESPN's "League Manager" Fantasy Football engine rewrite back in 2002.

The Yahoo outage reminds me of that experience. League Manager had no clear project or product management leadership. After four months of 60h weeks, most working 7 days, we ended up in a late-August crapstorm when we launched an incredibly buggy product. Things stabilized in about week three of the NFL season and I took the next three weeks off to recover, telling myself that the Christmas bonus would be worth sacrificing my time, energy, health, and relationships.

My key lessons:

0. My bonus was $1500. Not only was it not worth it, it was a slap in the face.

1. No death march projects like that ever again.

2. The opportunity cost to work on a project like that wasn't worth it, given the tiny rewards. An instructive lesson that helped push me towards entrepreneurship - working hard isn't a problem, but it is when the rewards are controlled by someone else.

3. In retrospect, I'm incredibly thankful that social media didn't exist. The closest we had were user forums. A quick perusal of said forums ensured a crappy day due to the vitriol and hate thrown towards the dev team, who were busting their humps to fix the product. I remember one commenter who sympathized with the dev team -- whenever I was down in the dumps, I would read his comment to trudge on. It motivated me for weeks.

For anybody on the Yahoo Fantasy Football dev team, it gets better. Take some time, reflect, and do your best to integrate the lessons you've learned here as you decide what to work on down the road.

I got majorly screwed in my draft yesterday because of this. What was strange was that it only affected about half of the guys in our league. That still means half of us auto-drafted though.

Yahoo just emailed out their solution to everyone:

"Dear Yahoo Fantasy Football Users,

Yesterday evening we encountered an issue with a portion of our Fantasy Football drafts and one of those affected was yours.

We're so sorry for the inconvenience. To fix your league, the Commissioner needs to use the reset draft tool and reschedule your draft date and time. Please note, the new draft time must be at least two hours out from your original draft.

If you need assistance, please contact customer care

Thanks and again we apologize,

Yahoo Fantasy Football team"

Doesn't really solve anything for anyone. I've been trying to get my league commissioner to re-draft or at least open up a vote for a re-draft but he has deemed that unfair to everyone else. There goes my season.

The bigger problem seems to be that your league commissioner is a tool. I would abandon the league and seek out another.

If it's a pay league, I'd demand a refund. It might be unfair to make everyone that was unaffected redraft, but it is decidedly unfair to have a draft in which members of the league weren't able to draft through no fault of their own.

If it's not a pay league, then it might not be the end of the world. You'll never be able to get the same sense of pride if you do well though. :-\

Maybe you could propose a vote from the other members, as a first step, and hope that they aren't as short-sighted as the commissioner?

It is a pay league, but it's a little tricky because the commish is one of my best friends/former roommates. Since it's fantasy football, I've taken to just talking a whole bunch of shit until action is taken. It's looking like we'll re-draft sometime this week I think, fingers crossed!

Commish is pushing back against a re-draft? Safe to say he wasn't hit with the auto draft bug then right?

Bingo

How is it even fair that half of you had to auto draft then?

Well, it isn't. His logic is flawed.

Yeah, same thing happened in my league. I was auto-drafting and apparently didn't do much to me but we had a vote and it was 7/12 for a re-draft. I'm surprised how many people this effected. You think after all this time they would have it down by now.

Oh god.

I'm a developer for a fantasy soccer, football etc. site and this is my ongoing nightmare. Fantasy sport provides some interesting challenges as a developer, but the fact that you are operating with deadlines and windows outside of your control and server loads that vary by a factor of 5-10 in minutes is... challenging.

If your site stumbles at the wrong moment (and we have on several occasions) you are basically screwed. You can't call FIFA or NFL and ask them to delay the event for a few hours while you put out a fire, or finish developing that new feature.

I can feel my stress level rising even as I'm writing this...

Relax. You are not the person responsible for the site being up. That's Ops, QA, and support's job. All you need to do is provide a working product ahead of time to the other teams to load test it and figure out stopgaps for any holes found.

You can of course expect to be working at launch time, crushing random bugs and pushing fixes to production. But those should be minor issues. If everyone else has done their job, you will never have a Yahoo-esque outage, and you certainly shouldn't be held responsible.

For the downvoters who don't find this comment helpful, I worked for four years in CBS's fantasy sports operations team, handling hundreds of thousands of hits per second (dynamic content) and tens of gigabits of sustained traffic on launch days, and we never blamed the developers for an outage. (We did, however, have them on hand to look at problems)

Well we're a small shop, so I am Ops, QA and support ;)

Well then, i'm sorry to hear that! But still, try not to get stressed if you have problems doing the job of 10 different people. My suggestion is to focus on making the application robust against errors and invalid data, and determining the performance limits of various parts of your application stack, then making sure you allocate headroom for the expected traffic well before the event - none of this automatic scaling business.

Reply on news.ycombinator.com