You're talking as if there are only two parties here: owners (rich) and tech workers (not rich). But tech workers are well above the average US income. The unspoken group in your argument is the consumer, who on average is poorer than the tech workers. Lowering tech wages with H1B visas helps the consumers as well as the owners, because it makes the marginal tech product less likely to fail.
You might still think that the workers have the strongest claim. But as with most protectionism, there's more to consider.