Directly telling someone that you have a problem with something they did and why you think it's a problem is not "passive aggressive" by any stretch of the definition.