On the contrary, the value restriction is a part of the extended rules used for the typing of ML. It was introduced because ML isn't pure, it supports references. The value restriction is there to prevent them having polymorphic type. The rule is simple : syntactic values are the only things receiving a polymorphic type. What is a syntactic value ? To quote Geoffrey Smith notes on SML97 "Basically, a syntactic value is an expression that can be evaluated without doing any computation".
All this iss done in order to ensure type safety. Actually, if references could have a polymorphic type, the type system would accept incorrect programs such as :
let r : 'a list ref = ref [] let r1 : int list ref = r let r2 : string list ref = r r2 := "a" let v : int = 1 + !r1
Which does not work because r, r1 and r2 are references to the same value.
Value restriction is a bit overkill. It rejects program which could be safely typed. Other solution to this problem exists. Actually, Standard ML used a different solution at the beginning (carrying information about the typed hold in a cell along) but it has other drawbacks. MLTon has a really good page about all that : http://mlton.org/ValueRestriction
Some research was done about relaxing value restriction. OCaml type system, for example, has less constraints than SML and MLTon (you can see this paper about that http://caml.inria.fr/pub/papers/garrigue-value_restriction-f...).
I believe that if you don't do it and generalize everywhere, unification of types is harder or even impossible because everything has a type scheme.
This would look a lot like System F, for which type inference is undecidable, for example.
If you look at the [Abs] rule, you will see in the premise "x : tau". Here, by definition, tau is non qualified. It represents a unique type. It's necessary monomorphic. Concretely, it means x will be bound to a type variable in the type environment and this type variable will uniquely be unified.
On the other hand, the premise for [Let] is e0 : sigma, x : sigma. Contrary to tau, sigma is qualified (polymorphic). So, it can be unified to a different type variable in different expressions.
I completely understand why it fails you. The way qualifier affects the unification process is quite complicated. I finally understood it only after having to write my own rules when I was implementing a type checker for my master thesis.
If you don't mind the formalism, read carefully the Wikipedia article paying particular attention to how tau and sigma are defined and to the paragraph about free and bound type variables. If you want something more practical, look at the Peyton Jones book I gave a link to in this thread. Part 9.5 is the important part but it's difficult to get without having read the whole chapter (eventually chapter 8 too). It 's still complex but far less abrupt than the Wikipedia article.
If the SPJ book is the one about implementing functional languages, then I've got a copy on my laptop I can peruse when I get home from work.
This causes difficulties because the set of free type variables needs to be calculated in order for generalisation to produce safe type schemes, and that is tough to do for an unknown set of expressions.
This is Karl Crary's explanation; here's the only online recording I've found: http://cstheory.stackexchange.com/questions/8892/type-infere...
As for this explanation, I'm not entirely convinced. In the first example, seeing x as two different cells depending on the type is interesting, but I have trouble seeing how it is possible to express its semantics without keeping runtime type information. Is System F compatible with type erasure?
So in some sense the value restriction is there to prevent us from having to make the ugly choice between unsafety on the one hand, and inefficiency and unintuitive behavior on the other.
(Operationally speaking, the two forms (let and abs+app) are always equivalent, though).
Would it be correct to assume, then, that a pure ML would allow HM to be trivially modified to support the equivalence?