Turning 4 or turning 5? If turning 4, canceling is reasonable. The kid won't really know what a birthday is and won't realize anything is missing or changed. The parties are almost more for the adults.
If turning 5, I'd probably keep it. On the other hand, you can scale back how you do it. The kid won't really know the difference. Besides, at that age, bigger isn't better. More friends is just overwhelming.