For now unfortunately you have to turn on "Optional diagnostic data" - aka "full spy mode" - to access this feature.
There is currently no way to roll back from dev builds since it would mean to downgrade (which Windows can’t do). Even disabling reception of new builds is not working since they are either such far away from beta that you can’t switch before your build expired (and you don’t want your build to expire since it break Windows Update some weeks later) or those builds are « branches » that will never meet compatibility with any beta/stable build.
tl:dr; Being on Windows Insider dev channel is irreversible.