darrellsilver

Add multiple tabs and I'd never use Safari again. Think Safari on iOS 5 got worse... the top nav bar takes up more room now.

Personally, what grinds is not the repetition, but the realization that

1) What's the point, really? Most all finance is really just ... not useful to anything. As an aside, Margin Call gets this point exactly right.

2) In most parts of finance, technology is a cost-center. Unlike other professions, there are truly sectors where technology is king, and developers impact the world around them.

Don't necessarily agree -- How many airlines are there in Kayak?

I've never asked them, but I suspect the Yipit team, in a few years, won't want their revenue dominated by this research business. But it's a hell of a business right now!

Agreed. There's a lot of skill in picking a job, much of which isn't indicative of your skills once in the job. IMO this often leads to people getting pigeonholed into specific positions if they don't happen upon the right opportunity or otherwise aren't inclined or encouraged to experiment or take more risks.

It's easier early in your career to try different jobs, and when you're younger the world is generally more accepting of letting you try new things.

What I love about this NYT article is that I imagine 6 years ago how it would have sped my departure (3 years ago) as a programmer on Wall St at a HF / quant fund.

Perfectly-timed way to help people move out of Finance. A choice quote:

“If you ever read an article about an investment banker named Jim Moran cutting mega-deals for some finance firm somewhere, please come find me,” he told the man, “and shoot me.”

Fingers crossed! Wish I could just find some docs on this...

Signed up for iTunes Match earlier this week...

Most of the songs in my library are non-drm (mp3) and it still took about 18 hours to do steps 1 & 2 (indexing & ?) across 22k songs. Music is stored on a (slow) network attached drobo.

The 5100 songs (25%? Crazy!) that aren't already up there are taking another 24 hours but I assume this is bandwidth dependent.

Crucially, this is all happening in the background, hasn't required my intervention at all, doesn't disrupt using itunes, and once it started uploading everything that it didn't have to upload was immediately available on other devices.

A pretty solid experience, IMO.

Anyone know what happens when the library hits 25k songs???

Tried to upload music and received this hilarious message: http://yfrog.com/ocygcp

After trying a 2nd time through the non-wizard interface, it turns out all 72 of my songs on this machine are iTunes DRM'd. Humph.

Specifically:

"I just got shoved out of the park by a police officer. I'm now going to explain what I saw."

https://twitter.com/#!/JoshHarkinson/status/1363615690754785...

Two excellent accounts from people on the scene:

- https://twitter.com/#!/JoshHarkinson

- https://twitter.com/#!/brianstelter

That it's the unvested portion definitely makes a difference... But to be ethical you really can't just take unvested shares from employees because they have "too much", which is how the WSJ tells the story.

Sounds like Pincus gave performance-based pay with a pretty transparent system, which should have worked well. Employees must know how management sees their performance, otherwise you're destined to have a bad breakup.

If the story is accurate (hopefully it's not as who'd want to ever work at Zynga after this?) then the lack of stock for new hires was a management issue, and should be resolved from management, the investors and the board's pockets. After all, each investor saw the cap table before putting money in.

Awesome, and awesome looking interface. Reminds me of Our project Norc, (github.com/darrellsilver/norc). Kue has intermediate statusnof the job, which would be a great feature for Norc.

That sounds super interesting! This sentence raises a question:

>You can chain tasks, so say TaskA launches another task. You can retry tasks if they fail.

How does this chaining work? Does each Task define its children? I've found that its cleaner to separate tasks from their place in the tree. For example, a script that downloads a CSV file each hour shouldn't care how that file is used.

Yeah, the terminology isn't well-defined.

Jobs start with any tasks that return True when asked due_to_run(), which can be schedule, parents' status, etc.

Glad I could clear it up!

I've been hearing really good things about RabbitMQ, and haven't been particularly impressed with SQS. It works at scale (we're currently at 10s of thousands a day, but it's probably the same performance at 100 or 1000x that...) but tasks take many seconds (like 10-30) to show up in the queue, which absolutely kills us.

We'll be adding a RabbitMQ plugin to Norc, just like SQS, when we can.

If I understand the question the answer is yes. I think you're asking if you can retry portions of a job, like if a task breaks but doesn't exit with an error, thus causing the rest of the job to continue.

In this case, yes, you can retry a task in the middle of a tree of tasks, and retry all the children tasks as well. This will leave some tasks untouched (the parents and peers) and retry the children in the same order as the first time.

If a task in the middle of the tree fails children tasks will not run until it's been skipped or successfully run.

Retrying children tasks would be a great job for a GUI for Norc because then you could choose which children tasks to retry interactively from a web app or somesuch...

I've heard some great things about Celery, and I think it may be better at handling massive amounts of tasks (10s of thousands) without using SQS like Norc.

How does it handle logs, resources and changing trees?

Gonna have to look into it more!

Comparing Norc to batch is similar to comparing it to cron. Many of the limitations of batch processing come with retrying or auditing complex trees of dependencies (bigger than a handful of tasks).

It's like wrapper scripts. They work well until they don't: managing the logs, getting efficient execution, understanding what happens when all become quite unwieldy, whereas a Norc-like approach proves more self-documenting and easy to manage.

I believe resource management is limited to overall system load. Batch isn't designed for things like managing available licenses.

I'm not a huge expert at batch, so if I've missed something let me know.

Norc isn't meant as a replacement for any queuing system, rather a replacement for Cron.

We use Norc in conjunction with SQS, and find the former is good for scheduling, management, logging, while SQS is better for lots and lots of repetitive tasks that we want processed ASAP.

Perpetually.com crawls, archives and versions any web site on demand, with any repeating schedule. One of the challenges is managing bursts of activity, so what we do is use Norc to manage the timing of archive requests, then SQS to manage farming out the actual archiving to available hosts. Since we hook SQS into Norc we can easily monitor/audit any system delay or outage.

We also use it to handle system backups, sending alerts and other system administration tasks.