Nigel Richards, The Best Scrabble Player | Hacker News Reader

Nigel Richards, The Best Scrabble Player

fivethirtyeight.com

Richards has won an estimated $215,000 in his tournament career

Here you have a person that has dedicated their life to playing a game professionally and has attained literally the top ranking in he world. Yet in his entire career has made less than many software engineers make in a year.

That is why I stopped participating in Scrabble tournaments years ago. Casual play is fun but tournament play is about memorizing a dictionary. It was less fun than work and paid horribly.

The article certainly doesn't sound like he has dedicated his life to professional play, and the recompense is simply a factor of how much economic pull the game has - if people aren't going to shell out to watch other people play, no prize money for you.

The article certainly doesn't sound like he has dedicated his life to professional play

Read the book "Word Freak" and you'll read all about the people that win Scrabble tournaments. It's really sad.

Why is it sad?

It's also on Netflix. Scrabble champs have no job and now money and scrape together funds to travel to events. It seems like a lifestyle that no one withing a few standard deviations of neurotypical would desire.

Elite Scrabble feels an unrewarding life goal.

"there are more than 16 billion ways to draw seven tiles"

Wrong. Even assuming the order matter (which is false, ABCDEFG is the same draw as GFEDCBA) and even assuming there are 7 tiles of each letter and 7 blanks in a bag (which is false), that would be 27^7 = 10.5 billion ways. Because these assumptions are false, the real number of combinations is lower.

For calculating the number of combinations, it looks like they are simply using 100 choose 7, or C(100, 7) which yields 16,007,560,800.

http://what-are-the-chances.blogspot.com/2008/02/7-letter-wo...

That doesn't take into account repeated letters. You actually want the number of 7-combinations of the multiset { A * 5, B * 3, C * 4, ... } or whatever the letter frequencies are. That's for the first draw from the bag.

I was noodling around to find the correct answer to this question, when I found this. It has nothing to do with why Nigel is the best Scrabble player.

"It is very hard to draw a rack that does not contain a valid word" - http://stats.stackexchange.com/a/6075 This statsSE answer includes an R program.

I started working on a python script to monte-carlo the problem, but it's pretty complex.

Even if we assume that you always start with a full bag of 100 tiles, every tile drawn changes the odds of the next draw in a different way.

Right now I've got a series of "unique draws" that grow slightly slower than the number of tries, but I'm not yet sure what single number it would converge to at infinity tries.

If you want to estimate it that way, google "estimating wildlife populations", "population estimation methods", or something like it. It will give you, a.o, http://en.m.wikipedia.org/wiki/Mark_and_recapture.

I think it isn't that hard to exactly compute the number on modern hardware, though.

Given that he has a better Scrabble rating than a computer algorithm, there has to be an element of poker at this level - otherwise if it were pure probability and memorization of the words (which a computer would have done perfectly, with no room for error), then a human could not outrank a computer.

I'm wondering (for the professional scrabble players out there) - do you play words differently depending on your competitor? That is, do you leave a vulnerability with some players, that you wouldn't with others? Or do you play the same game every time.

I was surprised that element of Nigel Richard's Scrabble game wasn't discussed.

It could just be that the Scrabble AI is not sufficiently advanced to explore the entirety of the decision tree. For all board states X, there are many deep trees that increase in complexity at boards X+1, X+2, etc (though it's arguable that it decreases in complexity in the late game, or can a game can be played "closed" to reduce the chances of bingos).

It may not be purely metagaming that causes the difference between AI and humans; the algorithm probably plays a far-from-perfect game.

The other advantage a computer (probably?) has over a human, is the analysis of probabilities of words being available given whats known about the word play of the opponent given their current (unknown) tiles, and tiles already played.

That is, I suspect it's possible, but difficult for a human player to surmise the probability of tiles hidden in an opponents pieces based on the words they've recently played (or not played) - but being able to do so, based on a count of which tiles have already been played and are visible on the table, is probably more challenging - but straightforward for a computer.

Are the games that players have played publicly available? While I presume that players at this level have memorized all 270K or so words, they might have tendencies to favor some words versus others, in particular scenarios - that's another advantage that a computer would have - adjusting it's play based on a players history.

>That is, I suspect it's possible, but difficult for a human player to surmise the probability of tiles hidden in an opponents pieces based on the words they've recently played (or not played) - but being able to do so, based on a count of which tiles have already been played and are visible on the table, is probably more challenging - but straightforward for a computer.

I have played a very small amount of competitive Scrabble and a few of my friends are playing in the National tournament that this article references. What you describe is utterly standard by even mediocre tournament players.

Tournament Scrabble players do a lot more than memorize good lists of 2s and 3s. It is truly impressive how deep the game can be.

The article claims there are just shy of 200,000 scrabble words.

Why would it be so complex? The key is to use all 7 of your letters. The subset of valid words containing all 7 letters you hold shouldn't be that large, surely the board can be exhaustively searched to fit them?

You don't have to use all your letters for the same word, and there are strategic considerations regarding which spots you place your words, since that might enable the other player to use the score multiplier tiles for their next move.

200K words! I wonder how many does an average human know? 5k? 10k?

IIRC 15k-20k is usual for monolingual adult speakers of English. Since different languages have different total vocabulary size -- English's is particularly high -- it probably varies quite a bit across languages. I believe bilinguals tend to have a smaller vocabulary in each language, but larger if both are counted.

You can estimate your English vocabulary at http://testyourvocab.com/ (BTW, if anybody knows a site that does a similar test for other languages -- particularly German -- I'd be interested).

You don't just want good words, you want to position yourself so your next move is also a good word. So you don't just have to iterate over the current board, you have to iterate over all possible upcoming boards.

But it's even worse than that! You want to set up your opponent for a bad move, and vice versa, so you're trying to infer each others' tiles from past moves and bluffing starts to matter. So you also have to reason about counterfactual games where your opponent would have done the same thing, meaning you have to care about the entire game tree instead of just the sub-tree you're in...

In the general case, solving games involving partial information requires super-exponential time. It's no picnic. (Of course in practice we use approximations, and those can work quite well.)

Good point regarding setting up for double/triple word scores, however I doubt there is much soul reading or bluffing in scrabble - I think the cost of making a sub-optimal move is too great, you'd need to be very sure that it hurt your opponent even more than yourself.

Also although the remaining letters are known, your selection is random. I don't know how you could possibly choose between words to maximize your next move when you don't know what letters you will get?

> I don't know how you could possibly choose between words to maximize your next move when you don't know what letters you will get?

Your selection is random amongst a determined pool of tiles. You can easily maximize your move even if the move is not 100% deterministic. The fact that there is a random factor does not mean strategy is useless.

This is trivially true in a lot of cases, but sticking with games: The dealer's concealed (hole) card in Blackjack is not known precisely but by counting cards, you can determine the best play even if the effect is small.

yes, against a weaker player you can play a more "risky" game - that is, risky in theory because it leaves potential vulnerabilities, but not very much in practice because you can be fairly confident your opponent won't capitalize on them. Nigel actually did that to me once; in a game that he was already winning he deliberately left a difficult lane open to give himself a chance at one last bingo, knowing that I would not see what he was up to, and had the satisfaction of bingoing out in a spot with four of his seven tiles constrained, and picking up a nice bit of extra spread.

Identifying the best turn in any one play should be fairly trivial I think, but the strategy with keeping some letters in hand for a big play later on while still optimising for score is probably fairly complicated within the time limits.

Or alternately the AI could just be fairly mediocre, I don't have enough expertise to judge.

Maybe the ranking system is flawed? I don't think Nigel played directly against the computer.

A computer would only play perfectly if it were coded perfectly. Any nuances to the play style would have to be known by the programmer, and coded into the system. And that doesn't even count plain old bugs that would cause mistakes.

there has to be an element of poker at this level

Pushing the boundaries of acceptable words (uncapitalized Chlorodyne being a good example) is one strategy.

Because he can make a word out of O X Y G E N ?

Edit: see one of the earliest Simpsons episodes

Edit: it is interesting that the articles assumption has been he is better than current algorithms and so better than algorithms can be. This just indicates to me a lack of time and research into the correct algorithm.

That's not meant as a trivial throwaway comment but that in pretty much every area of human endeavour we have lots of headroom for new algorithms.

So, It seems like No Human Player could beat Nigel, time to take over the challenge of building a better Scrabble AI than Nigel is!

Well, you can still jump in on the very fun and competetive Al Zimmerman's challenge (which ends in about a week, though), the current task is exactly about solving the given Scrabble sets in the best possible way: http://www.azspcs.net/Contest/AlphabetCity

Competition scrabble games are typically more open, where players lay out longer words to allow more opportunities to play more words and larger scores to develop. After that it's vocabulary and experience.

Isn't Chlorodyne a brand name and therefore capitalized? I would have rejected that. By the way, massive scrabble fan here. If anyone finds themselves in south-west China hit me up for a game!

Can he actually beat the AI in a head-to-head match?

yes (as can a lot of the other top players). though due to the element of luck in the game, the right question is whether he would have an overall winning record after repeatedly playing quackle (also yes; as of now he's definitely better than the state of the art ai)

Well, so, what makes Nigel Richards the best Scrabble Player on Earth? The article does not even answer its own leading question, which is enormously disappointing (though it is otherwise interesting). There are a couple humorous quotations, and literally two or three sentences that could be interpreted as conjecture as to why Nigel Richards is the best player by such a phenomenally large margin: some loose statements about knowing relevant probabilities.

This article was a perfectly interesting read about how Richards utterly dominates Scrabble, but unfortunately plagued by a totally misleading headline: I was expecting an answer.

exactly. It does show he most likely is the best player, but I too was expecting some interestng theory about how his brain works. That he's somehow capable of some algorithms other humans cannot even grasp. Or something like that, you get the point.

For a low-status game like Scrabble, what makes him unique is that people with similar brainpower cluster in more lucrative or rewarding vocations.

One could be pedantic and say that the article correctly explains why he deserves to be "classified" as the best Scrabble player on Earth. The very last line says that he knows every legal Scrabble word. The article also claims that he has a "photographic" memory. That's certainly an important skill to have, but can't be the entire reason. Even the strongest Scrabble AI has a worse rating than Richards. The other aspect that the article mentions is a focus on the current game. Richards is not even interested in games that he has completed. So, the article does not definitively conclude, but it does offer two significant factors that are significant to the question it proposed.

For the purpose of stimulating the discussion:

The article also claims that he has a "photographic" memory.

But a true photographic memory in this sense has never been proved to exist.

http://www.scientificamerican.com/article/i-developed-what-a...

You are misquoting. "Photographic" is a misnomer because exact images are not recalled. But extensive detailed recollection does exist, as the SA article explains.

Well, the article does say "I try to score points. The goal is to score more points than your opponent." and "I’m not sure there is a secret. It’s just a matter of learning the words."

Perhaps that (incredibly unsatisfying) explanation is actually true?

There's a 'how to play chess' book I can't recall the name of, with similarly useless advice:

Rule #2: Make the right move.

That worked for Smyslov (7th World Chess Champion), who once quipped:

"I will make 40 good moves and if you are able to do the same, the game will end in a draw."

So he brute forces it? What's the time per move in official scrabble tournaments?

25 minutes for all of your moves.

I thought we'd delinkbaitified the title enough last night, but evidently not, so we've sent it in for another round.

Reply on news.ycombinator.com