I'm curious, if you owned 51% of the network what's to stop you from rebuilding the entire history of the chain and then adding new transactions (including any of your new fraudulent ones) on top?
As for deleting, unless you want to enter a philosophical argument on the nature of deleting, and whether anything in the universe can truly be deleted [1], for all practical purposes a 51% attack does allow one to delete transactions off the blockchain but the older the transaction is the more expensive and impractical it would be to do so.
A 51% attack does not allow someone to create new bitcoins or spend coins they don't have the keys to, but they could double spend coins or delete existing transactions.