Agreed.
very rarely is there some tricky data structure or algorithm that casts a looming shadow over everything else
Agreed, BUT...
In order to organize, sooner or later, you will have to get clever (with tricky data structures or algorithms).
How I have always built something big and/or complex:
1. Add lines of code.
2. Add lines of code.
3. Add lines of code.
4. Holy shit! What have I done?
5. Refactor.
6. Combine similar sections.
7. Genericize with parameter-driven modules.
8. Still too many lines of code! Optimize Step #7 with something clever.
9. Go to Step 1.
2 years later: What the hell was this clever parameter-driven multi-nested process for? Why didn't I just code it straight up?For me, organizing complex code has always been a delicate balance between readibility and cleverness.
Don't remove enough lines of code and it's too much to navigate. Remove too many and it's too much to comprehend.
Organization + Cleverness + Balance = Long Term Maintainability