name (...) as (...)
name (...) = (...)
name (...)
...where (...) represents an arbitrarily complex expression. The first one is a "name as" command, the second is an assignment to an array element, and the third is a procedure call, and you can't tell which until you've fully and accurately parsed the first arbitrarily huge expression.
Also, the following:
name:
In BASIC a colon separates two statements on the same line, and an empty statement is valid, so is this a label or a subroutine call followed by an empty statement?
Edit - just remembered this gem: You can use a "With" block to save having to type the name of an object when referring to its members...
With SomeObject
MsgBox .Name
End With...will pop up a dialog showing SomeObject.Name. Now go ahead and tokenize that second line. If you're like me you got three tokens: [MsgBox][.][Name] The problem is that's indistinguishable from...
MsgBox.Name
You could say that ".Name" should be one token. Hmm. So what if it's something like...
MsgBox .Name.Substring(1, 3).ToUpper()
Still one token? I think my hair's going to be white by the time I finish this project.