Basically you have to crack hard crypto. With key lengths long enough and no mathematical weaknesses discovered in the algorithm it could easily take until the heat death of the universe.
Can't quantum computers theoretically break elliptic curve cryptography? I suspect we'll get there a bit before the heat death of the universe.
Bitcoin is actually somewhat hardened to quantum computers. Quantum computers can break ECC but are much less effective against hash functions. You can't attack the public key offline because it's not usually part of the blockchain, only the fingerprint is.