It'd be balanced by the difficulty in getting the light
into the material in the first place. You're right, you can't change the laws of physics. Light will only enter an object with lower refractive index if its angle of incidence is close enough to straight on that the beam coming through into the material has somewhere to go that satisfies Snell's law. When the object's refractive index is near zero, the object will only accept incoming light across an extremely narrow angle. This solves your concerns about having a global decrease in entropy.
You can fudge this by making the incoming surface of the object match the shape of the incoming waves. Then all the light will be allowed in. However, that surface will only match the shape of the incoming waves for light originating from a tiny point in space. If your light source is larger than a tiny point in space, then the light coming from the larger area will bounce off the material and not enter it. In effect, the magic material will magically collimate a light source, but only if the light source is already fairly magic, and in that case a simple lens would suffice.