dododo

defining program boundaries is tricky. here's what the FSF say: http://www.gnu.org/licenses/gpl-faq.html#GPLAndPlugins

it's not nitpicking: someone has given a license to certain rights, and not to others. it's important to honour what has been actually given.

it seems worth noting that the block size of gzip/deflate (http://tools.ietf.org/html/rfc1950) can be up to 32Kb, so i wonder if browsers are smart to enough to start rendering before a complete block has been received?

if not, compression could kill network latency... from the graph it seems maximum compression would require about 21 round trips minimum. i suppose this is why zlib's deflate has a flush argument: http://www.zlib.net/manual.html#Basic

it seems people call these techniques machine learning/AI simply because it sounds cooler: a lot of the techniques have a 30+ year history in statistics. this is the same story as neural networks: neural networks are just nested logistic regression, a technique with much history in statistics.

a lot of high frequency trading is simply linear/logistic regression (on the right features of course). anything more complicated is too slow.

btw, editline is a BSD licensed readline replacement: http://www.thrysoee.dk/editline/

i read some of this book (analytic combinatorics). it's quite nice in that if you can write down something akin to a grammar that constructs objects you want to count, it provides a set of rewrite rules to actually count them. unfortunately it doesn't provide a way to count anything more complex recursively than a tree that i could see.

for understanding the tree results, the key thing to understand appears to be lagrange's inversion theorem: http://en.wikipedia.org/wiki/Lagrange_inversion_theorem (i think it's also in an appendix in the book)

an interesting connection between probability/ai and this: the probability generating function is a normalized form of these generating functions that allows you to specify a probability distribution: http://en.wikipedia.org/wiki/Probability_generating_function

in particular, you can use this to work out the average size of a tree picked at random :-)

they do mean dynamic programming in a more general sense: but dynamic programming was invented originally to solve Bellman's equation. it turns out many other problems have a similar structure which is quite surprising! before bellman's seminal work in 1940s, it wasn't known how to efficiently these problems. indeed, you can often do something like Q-learning for finding approximate solutions to dynamic programming problems even faster.

the general search procedure is just A* search but with the beam search pruning each time you expand the fringe of exploration.

i don't know a good source, i picked this up from a bunch of courses/research.

this is a splendid book that covers the most probabilistic/statistical algorithms: http://www.inference.phy.cam.ac.uk/mackay/itila/

quite a lot of structure amongst these algorithms that the list misses out:

- dynamic programming comes from solving Bellman's equation and Q-learning is an approximate means of doing dynamic programming.

- It's possible to do beam a* search, where beam search, a* search, and best-first (greedy) search are special cases.

- in the continuous domain, greedy search is essentially gradient ascent.

- you can use discrete differentiation to find the gradients for gradient ascent (actually, i would replace the first order method suggestion of this list with newton-raphson or some quasi-newton raphson method like BFGS).

- EM ties some bits together.

- if you replace the max of the viterbi algorithm with a summation, then you get the sum-product algorithm which is essential in the E step of EM. if you use viterbi instead of sum-product, you get something known as zero-temperature EM, which is an approximate form of EM

- the M step of EM is typically just gradient ascent!

- you can use max flow to do a similar thing to viterbi on certain graphs

and so on...

You're right in that you need direction. Each subclass of machine learning has some elementary algorithms worth learning:

For unsupervised learning or probabilistic latent models, the EM algorithm, Metropolis-Hastings sampling, variational methods (variation EM, variational Bayes, expectation propagation).

For supervised learning, linear regression, perceptrons, support vector machines.

For reinforcement learning, Q-learning, E^3, Rmax.

linux + firefox + no silverlight = redirect to: http://ecn.channel9.msdn.com/o9/ch9/4/5/0/2/1/5/C9LecturesDo...

in the sample space i was intending, that would not be the case. i was imagining boy as a random variable that counts instances in an order tuple of genders (the underlying sample space).

you're right, without this explicit construction, it's problematic.

other things not worth filling your mind up with:

1. type promotion rules

2. method resolution order

3. APIs for simple functions, just write your own--much more explicit.

4. order of evaluation in "for (init; cond; inc)" loops and other nuances of syntax/semantics.

isn't this the same kind of "intuitive" reasoning that fails at simpson's paradox? http://en.wikipedia.org/wiki/Simpsons_paradox

conditioning upon more events can lead to a higher probability. p(boy = 2 | boy >= 1) < p(boy = 2 | boy >= 1, tuesday) (or more precisely, conditioning upon more events can yield a distribution with less entropy)

presumably os updates (from google) must have the privileges necessary to do exactly this or maybe this is the interface used for OS updates?

a Turing machine has an infinite tape. a typical computer has finite memory. therefore, it's tape is upper bounded by a constant function. this is a kind of linear bounded automata: http://en.wikipedia.org/wiki/Linear_bounded_automata

linear bounded automata don't have a halting problem. it's perfectly decidable. this doesn't detract from the fact it may take a long time to decide, but it is a far easier problem than halting on a TM.

sophos anti-virus wasn't able to detect compromised binaries or source until nearly 2 years after compromise, just like the rest of us. sounds like software i don't need.

(and that's assuming you get their on-demand scanning working (talpa) which only seems to work with out of date kernels.)

why hash the output then? this is introducing a dependence in the PRNG that you seem not to need (nor want).

if you're going to generate another 80 bits of entropy, then simply use this as the seed for another PRNG. this, sure, i agree, will get you more entropy, because you're adding more entropy to the system from an external source (such as the advertised device).

or programmers in general. i'm baffled why this hasn't happened yet.

are you sure?

the PRNG is a deterministic function of its seed. the hash is a deterministic function of the output. if you do the math and calculate the entropy (this isn't hard) you'll see that the total entropy is still 80 bits because the distributions of the PRNG output given the seed, and of the hash given the output have no entropy. the marginal entropy of the PRNG is 80 bits if you don't observe the seed.

another way of seeing this is via the data processing inequality: http://www.neng.usu.edu/classes/ece/7680/lecture3/node2.html

what are these guarantees you refer to? most pass a battery of tests e.g., diehard: http://www.stat.fsu.edu/pub/diehard/ but if you give a PRNG x bits of entropy, then it can produce at most x bits of entropy over its entire output.

all PRNGs have cycles: that is, they'll produce random numbers x[1], x[2], ... , x[n], then random number x[n+1] = x[1], x[n+2] = x[2], .... often n isn't as big as you'd like, and sometimes there are correlations among x[i] and x[i+k] you really don't want (linear congruent generators have this problem).

oops. you're right. i provided a bad reference. sorry!

call/cc was introduced, as i understand it, as a means of formalising goto in this paper:

Denotational semantics of goto: An exit formulation and its relation to continuations. Cliff B. Jones. http://www.springerlink.com/content/f340274565373810/

this implementation of yield in scheme reminds me of the procedure linkage table in ELF executables, roughly in C: goto *current_entry_point then update current_entry_point on return (in the ELF PLT this is a one shot update).

section 8.7 deals with relational algebra. this is a really nice mathematical underpinning to SQL.

why would you want to be a programmer for more than 5 years if managers view the position with such disdain?

given the $100-125K figure, it seems like the message isn't "lower = better" but "frugal = better". perhaps a frugal CEO makes for an efficient company.

what about 3D or 3D with time in R? this is what python has: http://code.enthought.com/projects/mayavi/

i'm not really sure about far superior for 2D: http://matplotlib.sourceforge.net/users/screenshots.html vs. http://had.co.nz/ggplot2/

it's true that CRAN+Bioconductor have a lot of coverage of statistics. if you need to fit a particular kind of model and it's not using MCMC then perhaps it's the best you can do if someone has already written the code.

we use sage (http://www.sagemath.org/) which includes a lot of the common python numerics: numpy, scipy, matplotlib, networkx. it provides nice interfaces to R and gp/pari. i also use mayavi2 for 3d plots (something i could never get R to do well under linux...) enthought have a lot of nice things for python and scientific computing. there's also pymc which i've not used (i just write the MCMC code directly).

i've used R for several large projects. it sucks in several ways for the projects i worked on:

1. it is extremely slow for numerics. slower than MATLAB. slower than python (with numpy). after talking to several stats people, it seems pretty much everyone ends up writing most of their code in C when using R. (in contrast, i didn't find this necessary in python or MATLAB for similar projects.)

2. its syntax is quite clunky. want to concatenate two strings? paste(string_a, string_b, sep=''). radford neal has a series of blog posts on R's design flaws: http://radfordneal.wordpress.com/2008/09/21/design-flaws-in-...

3. uninformative error reporting. by default, the stack trace isn't printed. even if it is, often the errors don't really tell you what went wrong.

i don't see any advantage to R over python. yield makes a nice replacement for the lazy evaluation of R.

i don't understand why "intelligence"/"smart" is a good thing to optimise/aim for:

1. it's not a well defined quantity.

2. under any definition, it doesn't seem to be anything useful in itself.

can some entity create an entity better at a task that the creator? obviously yes. c.f., compilers, factories, cranes, &c.

i'm pretty sure everywhere you put "base", was "32" before. the only 29 in the code when i read it, was on the checksum.

why did you pick this checksum? what are its coding properties?