batchsize 1 should always work. The trade off is always “bigger batch size/more efficient” versus “smaller batch size/better results”. By using larger batches you are able to reduce overhead, take better advantage of memory caches, etc.... which can give very large speed ups. But mathematically it can slow down your training convergence (check out the plots showcasing how smaller batches converge in fewer optimizer steps). This is because when you average over an entire batch, you throw away a little bit of information that could otherwise be used to better learn your model. For intuition on this, imagine that you are doing homework, and you are forced to finish 1000 practice problems before being allowed to look at the answers; if you had looked earlier you may have been able to correct your mistake earlier and not gotten all 1000 wrong. However it would have taken you more time to go through those 1000 practice problems because you would have been looking up the answers and mentally adjusting your model for each problem you did.