What I like to do with rake tests and/or delayed jobs is write the actual business logic in some other file / module: a module in lib/, a DCI style class, whatever. Then the delayed job / rake task just calls that code.
So your rake / Delayed Job interface is one, maybe two lines. And the body of your code lives somewhere easily testable with standard unit tests without having to mock out Rake (and/or forcing your asnyc queue mechanism to be synchronous for testing).