Mitchell Hashimoto has published a blog post describing how he approaches complex codebases. That might give you an idea where to start.
https://mitchellh.com/writing/contributing-to-complex-projec...
https://mitchellh.com/writing/contributing-to-complex-projec...
> The first step to understanding the internals of any project is to become a user of the project.
It's normally easier to figure out complex behaviour from the spec/doc/interaction than from the code.