Please allow me to disagree :) Most larger pure-algo HFT prop shops are self-clearing, they are exchange members (possibly with more favorable fees due to market making) and don't use pbs, instead relying on their own co-located execution infrastructure and funding.
Managing margin and leverage between prime brokers is an important job for the backoffice at the fund I work for. These numbers are watched very closely, and we have strong, multiyear relationships with the people who work at these PBs.
[0] https://www.nyse.com/markets/nyse-arca/membership#directory
Being a good prime broker is also a lot of work, and is an elaborate and involved process.
Now for the firm to take on two jobs, and then keep it secret... I wouldn't worry about it. It's like worrying your accountant is going to take your successful startup idea and run with it because they know how much money you make.
And in all markets, that's one reason why many firms invest in transaction cost and best execution analysis, to make sure they're not getting front-run when they see abnormal dips in broker quality.