And then you have to match the surrounding paint, which is all but impossible since even if you have the same color, the original will have likely faded over the years, making your newly applied coat a mismatch, so now you have to paint the entire wall (no fun when it's a big wall). And if you had wallpaper instead of paint, good luck to you unless you saved some extra scraps.
All in all, an access panel would make the job much simpler.