If you look closely at the video, the quality of detection is not really great for anything besides `person`. I wonder if weighing objects by the rate they are being found over N frames, and conforming to a somewhat linear trajectory could improve results; feeding matches from previous frames could also help find the same object in subsequent ones, and maybe even work out temporary occlusion (thinking out loud here - probably way more complex than one would expect).