If each token leads to another, it eventually leads to what commonly comes out as an ‘end token’. So each token has a certain chance of leading to a next token which might, possibly, say to end.

  current_token = <prompt>
  while current_token == true:
    next_token = statistically[current_token][choose_random_from(common_order)]
    current_token = next_token

  s = inf
  for i in common_order:
      p = 1
      x = i
      while true:
          x = statistically[x][common_order[x]]
          if x == false:
              break

          p *= x

      if p < s:
          s = p
          shortest_to_end_start_token = i