There are optical accelerators on the market that - I believe - do that already, such as https://qant.com/photonic-computing/