pz

i believe its cut off. 'read more' doesn't finish the sentence.

this reminds me of the pancake sort problem, except there you can only flip a contiguous subarray at the beginning. i got this in an interview once and then used it a few times when i had to interview folks. some people seem to get it right away... others... well, they don't

if you want to read a good mathematical exposition, check out chapter 6 in Mackay's (the guy giving the presentation) book "Information Theory, Inference, and Learning Algorithms". It covers stream codes and, specifically, arithmetic coding.

I've recommended this book in other threads... its an amazing read.

its about time this started getting some attention. i am currently selling just enough to get by (living in SF, with a taste for good whiskey) and its a pain not knowing when the money will come in. it took them over 2 months to deliver my december US earnings. and when i contacted them i just got back excerpts from some template. since its in violation of their contract, i wonder if its grounds for a class action suit? seems a bit aggressive, but they don't seem too inclined to correct things

convex optimization covers a lot of different problem types. in general, you are trying to minimize (or maximize) some function (the objective function) over a set of variables subject to a set of constraints on those variables. 'convex optimization' deals specifically with optimization problems where the objective function is... convex. this essentially means you can efficiently find the global minimum (or maximum if the function is concave)

ok, so a simple example: lets say you're on a hacker's budget and you want to maximize your caloric intake given your limited funds. This is your objective function. BUT, if you're like me, you can't live on ramen and mountain dew... i have a basic set of nutritional requirements. These are your constraints. For example, sodium has to be less than A and protein greater than B and so on and so forth. So you are essentially optimizing over ALL possible diets.

My guess is you'd end getting back: kale, potatoes and beans. Now that's a real hacker's diet!

i second this. this is probably my favorite text book behind Mackay's "Information Theory, Inference, and Learning Algorithms".

if you take the time to honestly follow the text you come away with a great understanding of convex optimization... both the theory and the methods.

having it somewhere with beer would be nice. that way the day can devolve into a bare-fisted language war.

pancho villa at 16th & valencia is pretty good and la cumbre around the corner is also pretty good.

i also like the tonayense taco truck on harrison and the al pastor at san jose at 24th & mission is delicious.

if you want something a little less authentic but healthier seeming, try papalotes at 24th and valencia.

see you there

yes, i stand corrected.

objective c is fine, its cocoa i find daunting. i am used to object.method() syntax so the [object message] syntax was a little hard to parse at first.... but you get used to it.

i went from writing a lot of python to writing obj-c so the transition to a strongly typed, compiled-and-linked language was a bit frustrating. not surprisingly its not as fluid a language as python..... but you get used to it

I second the Hillegass recommendation. Between that and example code and a lot of head scratching i think i finally understand the iPhone dev framework. I just put out my first app (iGo) and things are promising. We'll see but my hope its enough to cover rent (in SF mind you) and tacos.

amen! godspeed is a must have.

i definitely prefer repetitive or ambient, non-lyrical music. sometimes i'll even put on sleep-music albums

a couple albums in particular standout: * philip glass & ravi shankar - passages * kawabata makoto (of acid mother temple) - infinite love - i think this album is tuned to my brain's beta wave frequency. its like sonic ritalin * robert rich - somnium

for more energy i usually look to montreal. godspeed and silver mt zion always please. a little bit lesser known is "do make say think" who are one of my desert-island bands.

and sometimes, just sometimes, i just have to listen to dancey trance music. check out onnomon (who is also, by the way, my brother and an old timey hacker)

as a side note, i mostly use rhapsody for online music. its just straightforward and easy... well worth the monthly subscription fee. if only the site was a bit faster.

as far as free services go, i really like the spirit of last.fm but pandora's recommendations are consistently better.

I agree that initially you should dump it to a local filesystem. Since this is an experiment you don't want to get bogged down in DB performance details.

Also, if HD space is a concern, occasionally tar/zip up a bunch of the data. HTML is very redundant and I'd bet you could squeeze 500GB of HTML down to < 50GB, even more if you have a lot of pages from the same site.

Really, a lot of this depends on what resources you have available and how you want to process the data later on. If you are classifying pages independently of one another then why bother pooling them to a centralized DB? Just run your classifier on each node and pool those results instead.

An alternative solution is S3, which I've used for crawling storage before. Its not ideal for data processing since you have to constantly pull data over the network, but its an easy way to get centralized storage.

flippant summary of bayesian stats: normalize the likelihood, multiply by a prior (which you can just set to whatevs).

oh, and you are allowed to assign probabilities to non-repeatable events, i.e. what's the probability there is life on mars?