Yeah, that would be much better. But unfortunately the node_modules structure (which yarn is attempting to be fully compatible with) makes that impossible.
The reason is that each package only finds its dependencies relative to its own location. So your second level dependencies cannot vary from project to project unless you do copying.
(Example: AppA and AppB depend on LibX. LibX depends on LibY. Through their deterministic lockfiles, AppA and AppB disagree on which version of LibY to use. There is no way to symlink things together to satisfy that case without copying LibX or altering node's package resolution algorithm.)