For this particular problem, Mask R-CNN would have been the way to go -- it spits out instances as opposed to just deciding, for each pixel, to which class it belongs. Or an SSD (if we don't care about the mask at all).