It is hard to explain what is `beta8` and `curve.P` specifically, but they are arbitrary-precision integers so you can see what went wrong with an appropriate pseudocode:

    x3 = alpha * alpha
    beta8 = beta << 3
    // beta8 %= curve.P
    x3 -= beta8
    while x3 < 0 {
        x3 += curve.P
    }

Essentially we want to compute `(alpha * alpha - beta * 8) % curve.P`, so to say. The modulo is expensive though, so for typical cases we can just repeatedly add `curve.P` to compute the modulo a few times. This is indeed a valid optimization when we are sure of the range of `alpha` and `beta`, but `beta` can be controlled outside. So a very large `beta` from an attacker will cause the while loop run forever---a denial-of-service attack.

I cannot ELI5 this for you, but here’s more context:

  beta8 := new(big.Int).Lsh(beta, 3)
  beta8.Mod(beta8, curve.P)

https://golang.org/pkg/math/big/#Int.Mod