Comment by graycat | Hacker News Reader

Comment by graycat | Hacker News Reader

Parent Full thread

> the documentation is half done

The literature and documentation in operations research optimization is enormous, going back to the 1950s. The best of that literature is quite well written.

For combinatorial optimization, there is

George L. Nemhauser and Laurence A. Wolsey, 'Integer and Combinatorial Optimization', ISBN 0-471-35943-2, John Wiley & Sons, Inc., New York, 1999.

The details of optimization are not always trivial, and several fairly challenging graduate applied math courses, complete with non-trivial theorems and proofs, can be needed for much depth in the subject.

I've had four such courses and taught one.

> I guess it's extremely difficult to implement generic optimization solver,

A "solver"? Would be nice to have a "solver". Lots of people talk about giving a problem to a "solver". For linear programming, maybe usually can do that.

Otherwise my experience is that asking for a general purpose solver is, for now, hopeless.

Instead, for the real problems I've had success with, have to look carefully at the problem and exploit special structure particular to that problem.

Typically part of the work involves doing some derivations using the math of optimization.

> Otherwise my experience is that asking for a general purpose solver is, for now, hopeless.

> Instead, for the real problems I've had success with, have to look carefully at the problem and exploit special structure particular to that problem.

I am in complete agreement.

One way I have thought about this is that, rather informally, "the mapping from problems to solutions is highly discontinuous". That is, suppose you have a problem statement, and you figure out a method to attack the problem and solve it in practice. Now, vary the problem statement slightly. There is a reasonable chance that the new problem is dramatically harder to solve, and you need a completely different body of theory to even think about it, let alone solve it.

For example, suppose you have a polynomial equation in a bunch of complex variables. This is easily solved numerically in practice to any degree of precision you care for. Let's reword the problem a little and require that you are only willing to accept integer-valued solutions. Now you're talking about Diophantine equations! Have fun!

In practice, if you're participating in some project to solve business / industrial problems using optimisation, it is pretty common to have the definition of the problem change slightly every few weeks or months, as you learn more about the domain, or as people change their minds, or as the business actually changes what it does. Often you need to figure out clever ways of exploiting the mathematical structure in a particular problem statement to figure out a tractable way of solving the problem. Best to hope that the new version of your problem doesn't suddenly break that structure while you're "sprinting" towards the next release!

Yup!

IMHO it has been, since the 1950s, mostly this "sprinting" that was in practice just too difficult, that is, too costly in time and money, for far too large a fraction of organizations, budgets, managers, etc. that could have used optimization.

Now with current computing and the Internet, a lot in applied math, in principal powerful and valuable for important real problems, seems to be growing significantly in popularity. The "sprinting" is much easier but, sadly, too often still too challenging.

Of course part of the "sprinting" challenge was solved by, say, A Mathematical Programming Language (AMPL) where can just type in something like

        max z = f(x)
     subject to g(x) >= 0
                  x in C

where for large positive integers m and n and the real numbers R, x in R^m and g: R^m --> R^n.

So, then, with the problem thusly typed in, just need a solver! Right: Maybe from the back side of one of the moons of Pluto?

Gee, we know how to sort, know a lot about sorting, in O( n log(n) ), etc., so maybe we should have solvers that would do as well on optimization as we can do now on sorting? Then quickly we encounter the monster of the question P versus NP.

So, without an algorithm that shows that P = NP, we're back to column generation here, Lagrangian relaxation there, branch and bound too often, some group theory sometimes, etc. and "sprinting".

Yes, it was interesting philosophically how central and challenging P versus NP is, but I'd rather have just had a way to solve optimization problems routinely, gotten home in time for dinner, and spent Sunday at a BBQ!

Oh I absolutely agree, the algorithms and math are described very well. It's complex but clear, and the course I mentioned explains those concepts very well (and is fun, bonus points for that).

I of course meant the or-tools API docs. Overall I think it's nicely done and structured, but could be better.

I think we're talking about documenting the interface that or-tools provides. Even someone with an excellent understanding of the underlying concepts still needs to know exactly how or-tools exposes those concepts.

Yes. I was not clear: My impression of the OP is that most of the users or intended users will need good documentation on both the applied math and the APIs, details of input files, etc. My post was to say that for the applied math part, good documentation is and long has been available.

A bit tangential, but since you seem to be posting many _good_ math book advice. What would be a good route to read Neveu's probability book? Halmos' Finite Dimensional Vector Spaces and baby Rudin as pre-reqs? Those are within my confort area, but I will need to go through them again.

Or perhaps you recommend some probability books before too Neveu?

Reply on news.ycombinator.com