There's some factors to consider here. For visible light yes, the smallest feature probably won't be smaller than several hundred nanometers, however, optical computing comes with several major advantages over traditional electrical circuits.

The first is that light beams can cross paths without interfering with each other, allowing for a level of parallelism and density of signal paths without the concern of crosstalk/interference or shorting. Additionally, the information density of an optical signal is vastly higher than an electrical signal, and multiple optical signals can share the same pathways simultaneously. Also, energy usage is greatly reduced, so the constraints due to heat waste are much less.

Having said all that, the idea of optical circuits in a VLSI is still a very foreign and exotic concept for us, so it's hard to say how far we can take it if we invest at the level we have for electrical ICs. It's naive though to say it's not feasible due to some oversimplification of feature size limitations.