Any advice on choosing a pentest provider for a company like this? What are the good/bad stories you had with pentest providers? Thanks a lot.
Edit : added more details
Check their approach both technically and project wise. See how do they scope out your requirements. See how do they respond to your requirements. Ask for references, see their previous work. See how do they define risks for customers in similar segments. Are you comfortable with their (sample) reporting, can it be customised to fit your requirements? Do they have people who are adept at what you require? A naive example - if you have APIs, but they are sending in an infra pentester (read: wintel/ network specialist) who has no experience in web APIs, then you would not get complete results. Ask if they are comfortable in having a discussion with potential pentesters at their firm before engagement, see if they are a good fit. We encourage our clients to pick profiles from available pool and interview/ have a discussion with them (if required) to see if they are a good fit.
But even before doing this, document your requirements. Why do you want a pentest (customer requirement/ regulatory compliance/ inventory risk classification/ inner peace et al). I have seen plenty of horror stories wherein a poorly scoped pentest (both sides of table - from customer and provider) provided no value to the customer. Ensure you have a dev/ UAT environment that is a replica of prod incase you are not comfortable in doing pentests in prod (yes this happens as well). Else, you won’t get accurate (or even representative) results for VA/ PT. If in doubt, start with VA, assess what is wrong, fix, do a pentest. Rinse repeat.
I worked with a customer who had huge gaps in their security ecosystem from identification, detection and recovery perspective. When they approached us, I asked them what is the need of this and why now. They had no concrete answer for that. Hence, they went back, worked on scope and we worked together. Fast forward 6 years, add periodic VAs, a well oiled pentest program and yearly red team assessments have matured them from a security standpoint a lot.
For instance, year one, they had poorly secured email infrastructure and got compromised through phishing. Year five, phishing is nearly mitigated with appropriate anti-spam, policies, hard MFA, employee trainings, external sender alerts, quick suspicious email reporting, appropriate SPF/ DKIM/ DMARC policies, vetted vendor/ client email contacts, signed macros, DDE disabled, deprecated legacy protocols, powershell/ script execution disabled and monitored et al. You can probably imagine how many scenarios were conducted and guidance given to their tech teams to ensure vectors like these were addressed over a period of time as part of their pentest/ phishing exercise. You can probably understand their will to actually implement these recommendations as part of pentest exercise and ensure these are fixed through repeated simulations.
You mentioned having a discussion with potential pentesters at their firm would help, any advice on how to evaluate pentesters given that we probably know much less on the topic compared with those pentesters?
For example, my infra+hardware guy reported mousejack vulnerability as an extra observation during 2.4 Ghz spectrum security assessment. The core scope was wifi spectrum, but since he had an understanding of devices operating in that spectrum, he was able to compromise vulnerable wireless mice (logitech ones) at a client which operated in the same spectrum. Most OSWP guys don’t even know about it, you can have my word on it.