But the fact that it's not supplied with the source opens you up to a potential legal challenge. Since the license was not actually provided (saying "License: MIT" is not legally the same thing as providing a copy of a complete contract), they could argue in court that the user was not licensed to use this protected work. Regardless of how a court would find it, it's an unnecessary risk.