From thescribe's parallel comment it sounds like you're describing doctests which is a built in feature in Elixir, and I believe in Python also. Many languages have doctest libraries available, though not many support it being part of canonical documentation... I've been meaning to try the ruby-rspec doctest library
https://github.com/doctest/doctest-rspec though there's several others besides that one