This basically doesn't work because the mapping between phone numbers, users and operators isn't exactly 1:1:1.
Some businesses have a single number that they use as Caller ID on all their calls , despite having one corporate HQ in New York, one branch in New Orleans and one customer support callcenter in New Delhi. All of these use different carriers and are based in different countries, yet they're all legally authorized to use that number.
If you want to read more about why this is such a hard problem to solve, see https://computer.rip/2023-08-07-STIRred-AND-SHAKEN.html