Cool idea! Does this work with electron apps? I tried automating some apps and the problem was that a lot of stuff was never reachable, only via screenshot + click
The tree is shallower than native Win32/WPF apps, but key interactive elements (buttons, inputs, lists) are usually exposed. You can check what's available with:
curl "localhost:7878/windows/{pid}/find?interactive=true"