Ask HN: How to evaluate COTS software?
Any advice on how to do this critically, effectively and efficiently across distributed test teams?
Before you or team members get "sold" on a particular vendor or product, write down (on a wiki, IM such as Slack, or other internally accessible spot) what your requirements are. For example, it must cost less than $x/user. or, we need features x, y and z, but feature a would be a nice-to-have. Also list any show-stoppers. (If it doesn't do z then the answer is "no", etc.)
Be aware that a cheaper/better product may not always get approval from the powers-that-be. Being able to show in terms that matter to the powers-that-be why your pick is the best certainly helpful.
Use the criteria to help you narrow down the products you wish to trial (maybe the top 2-4). It is important to try out the product in the environment you intend to use it. Especially, if the cost of the software is high.
Schedule time to try out products.
Actually try out the product.