I have the same problem, namely a constant urge to look up things while reading paper books. My personal solution to that is to copy whatever sentence that triggers my urge to look up anything and then set aside time to do the look up later. I think the value it brings is that I won't be distracted into deep rabbit holes while finishing my book.

I'm not sure what's the end goal here though. It seems like you would like to look up things as soon as you encounter them, but on the same device (Kindle)? Then the benefit is not switching devices?