Comment by westoncb | Hacker News Reader

Comment by westoncb | Hacker News Reader

Seems like the other book people recommend is "Introduction to Algorithms" by Cormen et al, so that the main problem in choosing an algorithms book is really deciding between that one and this one.

I've spent more time with the Cormen book so far and only a little with "The Algorithm Design Manual", but have to say I've been disappointed with Cormen and very impressed by the ADM.

I think the Cormen book has a reputation for being more serious, and it does give more in-depth mathematical treatments of the algorithms covered—but honestly, I think it's a mistake to consider it more serious for that reason. There's a time where I would have stopped reading this comment and ordered the Cormen book, but I think it's worthwhile to consider criteria for a book being serious. In the realm of software, I'll argue that the most important measure is how much the book improves your ability to create serious software—and under those conditions, The Algorithm Design Manual takes the prize.

In case there's still ambiguity on what 'serious software' might mean, I'll point out that the ADM is not a recipe book—I'm not trying to say, "practically speaking the algorithm problems in software are solved, you just have to copy and paste the source for the required algorithm." For crafting difficult, original algorithms, I still think ADM will be more useful.

I also spent more time with CLRS, I've had my copy for close to a decade now :)

I found ADM to be much less useful. It was mostly a small bit of code per concept, with little explanation or proofs or information about why it worked. If I wanted to copy and paste code, I wouldnt've gotten a book, I'dve searched Google. I'm much more comfortable with the knowledge I obtained from CLRS, because I know not just how to code the things (since there's pseudocode for them) but why they work, and how to debug things when they break. Overall, whereas I'd class CLRS as one of the best purchases I've ever made (in terms of dollars/use of the product), I'd class ADM as one of the worst, and that's even considering it was like $5 used :)

I think this sample exemplifies what ADM is quite good at:

"Why is sorting worth so much attention? There are several reasons: • Sorting is the basic building block that many other algorithms are built around. By understanding sorting, we obtain an amazing amount of power to solve other problems. • Most of the interesting ideas used in the design of algorithms appear in the context of sorting, such as divide-and-conquer, data structures, and randomized algorithms. • Computers have historically spent more time sorting than doing anything else. A quarter of all mainframe cycles were spent sorting data [Knu98]. Sorting remains the most ubiquitous combinatorial algorithm problem in practice. • Sorting is the most thoroughly studied problem in computer science. Literally dozens of different algorithms are known, most of which possess some particular advantage over all other algorithms in certain situations. In this chapter, we will discuss sorting, stressing how sorting can be applied to solving other problems. In this sense, sorting behaves more like a data structure than a problem in its own right. We then give detailed presentations of several fundamental algorithms: heapsort, mergesort, quicksort, and distribution sort as examples of important algorithm design paradigms."

This kind of information on context is much more difficult to find than proofs. The idea that sorting algorithms are good, stripped down exemplars of general purpose algorithm concepts is something I've never heard pointed out elsewhere—and the text is full of these golden, context-related insights. Once you have a firm grasp of the principles involved, then CLRS or google become useful references (though, tbh, it's extraordinarily rare I find myself reaching for CLRS over google). I just tested and it took me less than 15 seconds to find a proof for the running time of mergesort, which is basically all that CLRS offers over ADM.

>This kind of information on context is much more difficult to find

Yes, but its also highly subjective & imho useless information. Emanuel Derman often writes about this problem, and it is something I have observed very frequently at workplaces - this present generation seems to be obsessed with meta-knowledge rather than actual knowledge. He points out its better to know a few things but know them indepth, rather than have an ability to talk about everything without actually knowing what those things really are. When I ask people to implement ordinary least squares from scratch, literally like 5% of the people interviewed manage to do that, & they are very, very good. The remaining 95% "know about ols" but don't "know ols" ie. they will give me lots of mumbo jumbo about how ols is the building block of statistics & how it is affected by outliers & so forth - but can't implement one to save their lives. When it comes to sorting, I vastly prefer somebody who can implement atleast one of O(n) ( postman sort), O(n^2) ( bubblesort) or O(nlogn) ( mergesort, several others ) rather than somebody who can give me all this context & then say I will google the actual algorithm. CLRS teaches you how things actually work. ADM is imho good for acing interviews conducted by managerial types (PMs/EMs) who want you to namedrop & give lots of context & color without actually doing any real work.

I can understand your complaint here, and I agree with you that people can easily go too far in the direction of 'when to implement' versus 'how to implement'—but I'm pretty sure the key is (1) Both are required, (2) it makes more sense for one of these things to precede the other.

However, I think that you’re also conflating two approaches to learning ‘context.’ One gives people an elaborate index for finding implementations when required, so that they become capable ‘solution googlers.’ This is unquestionably an inadequate approach to education. However, there is another approach to context which emphasizes principles. ‘Principles’ can be a vague word itself, but I’ll try to be more precise about it: you have understood something ‘by principles’ when you first understood a more general structure, then, in order to learn the more particular thing, you understood which parameters must be supplied to the more general structure in order to get the more particular one. E.g., you learn principles of programming languages first, then can understand new languages as parameterizations of your prior, more general knowledge. I think the context material in ADM serves as a nice guide for acquiring principles in this sense. The bit I quoted is largely significant because of their emphasis on principles.

It’s interesting that you think ADM would be the book for faking an interview. I have the exact opposite stance. If I wanted to impress a typical software person (e.g. a yet to be determined interviewer), I’d learn a lot of the details from CLRS, rather than going for ADM.

Also, in attempt to be more objective about this divide, I've found it useful to note that there's a general tendency in intellectuals to be in one camp or the other: unifier or multiplier. I've seen this issue taken up by William James, talking about 'rationalists and empiricists' and by Freeman Dyson in an essay about 'Manchester and Athens'. At the highest heights of intellectual accomplishment, you find people who revel in particulars, and others who love only principles, each typically having an amount of disdain for the other approach. Both are necessary, though.

I think it's better to know about the existence of a problem, that it has solutions, and that the solutions can be applied to similar problems, than it is to know any given problem and solution in depth.

When you have a breadth of knowledge, you know where to look to start your implementation - or better yet, use someone else's implementation, until it's not good enough, which may never happen.

The risk with knowing a few things well is that you may be ignorant of of the existence of similar problems and know that they already have good solutions. Meta-knowledge, as you put it, is generally more likely to be useful than specific knowledge, unless you have specific knowledge about a lot of things - which is unlikely unless you've got a lot of experience.

I think you're overlooking that whys and whens are still knowledge leading to understanding of a concept, and they tend to come before the hows. You want to understand why you're using the algorithm before you begin working on how to implement the algorithm. As an extreme example, consider us teaching the Quicksort and the Bogosort as pure implementation but without explanation on usage. Granted, this is unlikely to ever happen, but it serves to illustrate that this kind of knowledge is part of the understanding of any algorithm, or even further, any concept.

Now the context you're referring to regarding your "implementing ols" example seems to be largely assorted trivia to be honest. They don't know the answer, they're filling in something, and have done so through highschool and then college because something might get mercy marks. I'd like to disagree on giving that the moniker meta-knowledge, it's losely tied factoids at best.

I completely agree that deep understanding of concepts is incredibly important, but i think it's unlikely we'll see large scale change in that attitude. The way I see incentive structures for students, making that shift unconsciously is disincentivised, since moving from a cramming schedule to a more thourough approach puts you even further behind temporarily which hurts your grades. The fact that any kind of disaster or holdup could push you into the former style of learning is what I think causes the entire thing to tilt.

Whilst there are a couple more points I'd love to adress, I kinda want to question your assertion that this is a current generation thing. That is an awfully easy assertion to make, can usually be said about the asserters generation as well and all generations before that. Unless there's more concrete data to back that part up, i think it does the present generation plenty of injustice.

Interestingly enough, we still don't know a lot about sorting :)

I have had strong opinions about the same, which have fluctuated drastically over time and now settled on less stronger opinion.

I was from an EE background and didn't have an algorithms course in our curricula (did my bachelors in India, so you don't usually have a lot of options of taking the courses you like). So I hated CLRS and like Skiena (though couldn't appreciate it well enough). Later on, went to grad school and had a amazing teacher and I was hung over theoretical text books (CLRS etc) and swore by them. Now I was teaching algorithms to couple of colleagues (again from non-CS) background, and initially tripped by suggesting CLRS. They are way more happier with Skiena. I guess, for some people applications inspire the theory and then theory take over. For others, they fall in love with theory naturally (or with good teachers). So my current take is, you probably want to go thru both the books if you want to code/think in algorithmic way. The order is going to depend on the person.

Reply on news.ycombinator.com