The fact that you didn’t change the learning rates for different batch sizes even following a linear scaling rule let alone tuning for each batch size is pretty important. IMO without tuning the learning rates comparisons across batch sizes are pretty meaningless because it changes the effective learning rate which can make a big difference in training.