No, it will still happen. The learned features will just be "overfit" to that small dataset.