Comment by mturmon | Hacker News Reader

Comment by mturmon | Hacker News Reader

Parent Full thread

Your last statement is incorrect for one of the problems mentioned in the parent comment ("airline/package routing" which I take to be linear programming problems).

Simplex-type algorithms are typically used for these problems. These algorithms also have exponential time complexity in worst-case. In practice, well-designed simplex algorithms are of low-order-polynomial complexity. They terminate with an exact optimum.

This case, and others, are examples of situations where conventional complexity analysis fails to provide insight into real-world performance.

Last I knew, people were trying to analyze expected time complexity of certain (analysis-friendly) simplex algorithms under distributional constraints on the input structure matrices. But it's important to understand that this is mathematical back-filling to explain what is already known to hold in practice.

I would expect an airline routing problem to be an integer program (because you can't send 2/3 of an aeroplane along one route and 1/3 along another), and integer programming is NP-hard. On the bright side, at all times you have an estimate of the best possible solution so you can terminate early when you're within some percentage (say 1%) of optimal.

Interestingly they still use the simplex method repeatedly during the branch-and-bound solving process, because it only requires a little work to adjust at each step, whereas the methods with guaranteed polynomial worst case tend to take that polynomial time regardless of how "close" they start to the optimum. So, as you say, convential complexity analysis suggests that simplex would be an awful lot less useful than it is.

for linear programming there are several well known polynomial time algorithms (some interior point methods: ellipsoid, projective, etc)--the problem itself is by no means in EXP or NP.

for worst case complexities it's often just a small region of the problem space that makes the bound large--but enumerating or cutting out those cases is tricky. you just don't see them on average because they are few (c.f., quicksort: O(n^2) worst case, O(n log n) average).

Yes, here's a summary (somewhat dated) of this, given in a prize award for one of the key people who showed that worst-case is rare:

http://www.informs.org/Recognize-Excellence/Award-Recipients...

(expand out the "show more" button). Of course, there has been more progress since, but this summary is pretty good.

Reply on news.ycombinator.com