Become a Better Developer by Reading Source Code | Hacker News Reader

Become a Better Developer by Reading Source Code

stevejgordon.co.uk

193 points0x54MUR4156 comments

A big part of me wants Comparative Coding classes to be taught to college students. Even though I have a pretty sad cautionary tale about Comp Lit classes from a very good friend who went for an English degree.

Some of the earliest conversations I recall were about the trap of taking Comp Lit too seriously and not spending enough time on the craft. Eventually your inner critic creates fear that your work isn't good enough, and it becomes a self fulfilling prophecy.

I spent the next decade watching helplessly as the inner critic ate my friend's hopes and dreams one by one. In her head, she was going to be a great fiction writer. But the only thing of hers I ever remembered was her persuasive writing. She was passionate about her causes, but somehow also had it in her head that this wasn't 'real writing', and I didn't have the gravitas to convince her she was wrong.

And so the sad fact is that if you look at what I've written and shown to people, and what she's written and shown to people, it would be charitable to say she's done three times as much as I have, to an audience perhaps ten times as large. Me, who only briefly wanted to be a writer in grade school, whose only college-level English class was a technical writing class.

I am not a Writer, but I write. She is a Writer, and doesn't.

But even so, I've got body parts stuck in this same trap. I write a lot less code that some other people. But I also work hard to make sure that I'm on the right side of the Pareto principle. It's not the lines that count but how you use them, or so I reason. But I've started noticing the consequence of that choice, and trying to find a new balance between quantity and quality. Old habits die hard.

Writers write. Coders code.

To quote a famous parable of the pottery class

The ceramics teacher announced on opening day that he was dividing the class into two groups. All those on the left side of the studio, he said, would be graded solely on the quantity of work they produced, all those on the right solely on its quality. His procedure was simple: on the final day of class he would bring in his bathroom scales and weigh the work of the “quantity” group: fifty pounds of pots rated an “A”, forty pounds a “B”, and so on. Those being graded on “quality”, however, needed to produce only one pot – albeit a perfect one – to get an “A”. Well, came grading time and a curious fact emerged: the works of highest quality were all produced by the group being graded for quantity. It seems that while the “quantity” group was busily churning out piles of work – and learning from their mistakes – the “quality” group had sat theorizing about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay.

The best way to get better at creative pursuits is to create. Then get feedback. As much feedback as you can find.

"The only way to learn a new programming language is by writing programs in it." - The C Programming Language, K&R

But "become a better developer" is not just "learn a new programming language"

It depends. Being able to stitch multiple languages together definitely has its “whole is greater than the sum of its parts” aspects to it. Better programming for me is creating or discovering a tool that is useful to me that makes some part of my path to programming better. Building your own version of a language can be a fun way to get better at coding too. Grab the base source of any language and find all the string values and start do a mass search and replace to new ones that don’t exist yet in the language. The problems you will run into will bring you to examine parts of the language you weren’t aware of.

Agreed!

Yes, but at the same time, the best advice that published authors give is to read great authors, because it impacts the feedback loop.

I've read the opposite, i.e., the best way to become better at writing is to write.

I've certainly found this to be true with programming. I can read great code and books about how to code, but I've really become a better programmer through writing lots of code.

I have heard this, but as cute as the story is, I can’t find evidence it is true.

It’s not a factual story, it’s a parable.

After I typed that I worried it would be misinterpreted.

Not true as in a something that happened, but true as in the underlying concept.

Gladwell talks about hours of practice, but it has to be conscious practice.

But this parable has always been strange to me because if your target is quantity, it seems errors, in that context, are things that slow you down in production not quality issues.

You will learn to be fast, not good.

Deliberate practice is definitely required to improve beyond basic proficiency, but mere practice is what gets you to proficiency. Mistakes will be reduced, but technique may be imperfect.

In Spain, around 1600 and the height of the Spanish empire, there were two particularly well known writers. One was Cervantes (Don Quixote); the other Lope de Vega ("the sheer volume of his literary output is unequalled, making him one of the most prolific authors in the history of literature" [0]).

It makes an interesting case study for this sort of thing. Both were prolific; but Don Quixote is clearly the greatest work in the Spanish corpus despite Lope having vastly more written. But to some degree both authors are remembered.

[0] https://en.wikipedia.org/wiki/Lope_de_Vega

I think the underlying mechanism is feedback (even your own). The more times you try, see the result, and adjust, the better you’ll get.

You need to either have good taste yourself or an arbiter who can tell you where you went wrong.

Of course at some point you reach a skill level where this no longer works. You still need as many cycles as you can get, but the cycles get longer because it just takes time to do the bigger and better things you can do now.

I want to say it was based on a true story about photographers, unsure why it was changed into a parable instead.

The story is from Art and Fear, if memory serves.

https://sivers.org/book/ArtAndFear

I was hoping that the "quality" group had produced more than 50 lbs of pots due to trial and error but I guess even parables can't be perfect

I like this. But I also like to take it as a grain of salt, because bad patterns emerge without large-quantity of work having some wise guidance. And breaking bad habits, specially when encouraging “experienced” folks to do so, is tough.

Everyone should try and read busybox. It’s a fairly complete implementation of a normal “unix” user space without the libc. Most commands are completely contained in files less than a few thousand lines long and can be read through and completely understood in an afternoon. I personally use it for most things because of that even if some utilities are a little incomplete, being able to double check the source easily makes me feel like I won’t get surprised.

Why not the GNU or BSD coreutils?

Presuming because they are significantly bigger compared to busybox due to the limited feature set of busybox

Had a quick look, found this little gem, a math evaluator with precedence.

https://github.com/mirror/busybox/blob/master/shell/math.c

Some funny comments like "Think about it. * It makes sense."

> The lower 5 bits are the precedence, and 3 high bits are an ID unique across operators of that precedence.

This is so cool. Nobody would write code like this today

People wouldn't optimize a shell like that anymore, but people still do encode small pieces of information into integers using bitwise operations when compactness is an advantage. If you're dealing with a large data structure with tens of millions (or billions) of entries, compact encoding still trumps convenience, because memory bandwidth will often be the bottleneck. A data structure that would benefit from a compact encoding could be a compiler's intermediate representation, for example.

I was surprised that it's not recursive-descent/precedence-climbing, and that ~20% of the linecount is copyright/comments.

Yeah most things have a lot of comments. It’s very nice to read.

Yes, it is great to read source code. One of the great things about most things being on GitHub these days. Also great when you can use decompilers (Java and .NET especially) to determine exactly how the library code you use works.

One of my favorite exercises though is to port code from one language to another. Reading the code gets you to one level of understanding. Implementing it yourself while following along gets you to another level. But translating it really gets you a level of mastery of what the code is doing.

“Also great when you can use decompilers (Java and .NET especially) to determine exactly how the library code you use works.”

Decompiling .NET and earlier reading leaked windows source code have given me the revelation that the people working on these systems are regular people like you and me. Somehow I thought MS or Apple code was something special but it really wasn’t.

But I agree: reading source code is good for learning and also empathy. It’s way too easy to look at a code base for five minutes and declare this needs to be totally rewritten because the previous devs wrote bad code

I run a newsletter[0] that has a section call "Code to read" for many language.

I tried to include small/interesting repos to do weird/cool things so people can learn about it. Eg, this code: https://github.com/evilsocket/opensnitch (UI in Python but the logic is Go)

---

[0]: https://betterdev.link

Decades ago I became a much much better web developer by reading the source code of web pages. Unfortunately that is rarely possible to do anymore. I really feel the web is the worse for it.

I find it surprising how often you can see a web page you like, and then go and find the repo on GitHub that built it. Definitely for open source organisations, but also often for consultancies and government agencies.

Some people here are mentioning code reviews for their teams and how they don’t enjoy that.

I would say this article is missing a fair point. I’ve learned a lot by just browsing Github.

For example if I’m interested in solving a Caching problem I’ll skip some of the open source libraries around caching and how it’s implemented.

Often times you run into a very elegant and readable solution that someone has created. And the most interesting takeaway from that is that you can get inside the persons head around API design and implementation.

When I stumble upon an awesome elegant library I always either take a mental note about it or try a small sample implementation of my own. This has helped me synthesize great APIs designs over my career. You develop your own style over time by picking and choosing your elegant solutions, and eventually you also come up with your own bright ideas through language features or loopholes that come across as exceptionally elegant.

As with any good habit, it's important to reduce resistance. If you want to eat healthy, have juicy apples at hand in the fridge.

I made a conscious decision to start installing all my dependencies into a folder within my project source-code so that I can painlessly jump to the dependency's source and read it. (E.g. in the Ruby world, this means installing to "./vendor" in your project).

The effect of doing this is that it's easier for me to read the source code of a library than it is for me to leave Vim and look up the docs online. Voilà, now I'm reading more source.

I write Python mostly, and whenever I want to quickly look at what a library is doing, I just right click and do Go To Definition (what it's called in VSCode, I believe it's something similar in IntelliJ/PyCharm) and it'll jump through the library folders in my virtualenv to get to wherever the function/class is.

Then I don't have to think about dependencies as much, albeit in Python there's always a struggle with that :P

I remember going through the original Doom source code after it was released as open source. Although it seemed way over my head, I learned a lot about how well C code can be made to work, even for a complex game (at least at that time). I also got to appreciate John Carmack's coding style/skill even more.

I synthesized a very small amount of programming ability and skill from reading randomly chosen and scattered source files of an open source Quake clone’s source code (Digital Paintball 2), even though its comprehension and understanding it was beyond my capabilities at the time, especially at such a young and immature age. That tiny amount was negligible, but it still functions today as a segment of my present and current programming knowledge infrastructure. So you can safely conclude that the activity of reading “seemingly” insignificant programming source code files is a future knowledge, capability, and understanding’s becoming-vital infrastructure. For what power does not have a base or foundation to plant its feet upon?

This is essentially what https://www.goodreads.com/book/show/529511.Code_Reading and https://www.goodreads.com/book/show/169414.Code_Quality are all about...

And about the "what if i don't understand it" bit, i would suggest one goes and reads https://news.ycombinator.com/item?id=15903771 ;-)

the reviews for those two books are really bad - do you recommend them?

With the Elixir language, I find myself looking at the source code of my dependencies more often, since I can actually read and understand what's going on most of the time, despite barely coding in Elixir. When I do, I can look for inspiration in other people's code.

I rarely feel this way with Kotlin or Java libs, and I'm only slowly warming up to TypeScript code.

I want to read some good C code. Can anyone recommend me a project that's not too big? I'm thinking of skimming some of these core linux libraries [0]

[0] http://linuxfromscratch.org/lfs/view/stable-systemd/chapter0...

lmdb is one that comes off the top of my head.

But even that the main file is 11000+ lines of code.

redis is very good. Also, any suckless project is very readable. I liked Lua code too.

I'm not a C programmer but these projects have this Quality in common that transcends the programming language (I think). Opinions?

If you’re going to try read along with Robert Love’s Linux Kernel Development. Even outdated its the best guide I’ve found. That said I’m kind of a C beginner so take with a grain of salt. I frequently get derailed by the Macros, architectural stuff etc in the kernel and can’t figure out what exactly the code is doing.

The FreeBSD core utilities are reasonably short:

https://github.com/freebsd/freebsd/tree/master/bin

You might want to read '500 lines or less'[1]. Here you not only read the code but also see it in the context of the 'bigger picture' of the design / architecture / problem being solved.

[1] https://www.aosabook.org/en/index.html

Late to this, & am not really a C programmer (longtime bash & java/scala etc, and studying Rust now), but for exemplary C I think of OpenBSD core utilities first (ie, not ports & packages), given their ongoing auditing for security and correctness. The code can be browsed online, or installed as part of a basic system. I found it interesting to browse starting with some smaller utilities like echo and ls.

Resources:

https://cvsweb.openbsd.org/src/bin/

https://cvsweb.openbsd.org/

https://github.com/openbsd

https://www.openbsd.org/anoncvs.html

https://www.openbsd.org/ (where I got the above links, on the left side under "Getting Source")

(just to improve my previous accuracy where I mentioned scala & rust, though it's too late to edit: ...and C++ and many other things .)

Seeing well architected code bases is how I went from total noob to "professional". You can't know what's good code if you've never seen good code before.

There really should be a list of "good" codebases for people to look at and study. If anyone has any good backend Node.js codebase recommendations, I'd love to take a look.

Read the git source code.

It's not exactly what you're asking for, but recently I started using Nest.js and it's been a godsend. It's a very opinionated framework, which means you can skip all the bootstrapping and go straight to building the actual application.

I already have to read source code by doing my team's reviews, and I don't enjoy it.

Maybe it's because of the responsibility (why didn't you caught this bug in the review?), but it's really difficult.

Then maybe you are not meant for this? Yes, I do find it really hard to read code I have no interest in, but it was proven that if I understand the underlying idea, I can both read and find issues with the code I'm reading.

The first rule is to care and understand what the code is trying to do.

I have found that the secret (for me) is to manufacture some reason to read the code. Code is difficult to read from top to bottom unless it's done in a literate style. Instead, it is much more pleasurable if you are jumping around reading code because you want to understand something.

I find that editing the code makes a big difference. Find something to chage. Refactor something. Add a tiny (even meaningless) feature. This forces you to understand what the code is doing. That, in turn, forces you to explore the code and to read it. It also gives you a gauge to judge the code against. Why did they do it that way? What if I changed it? Oh, I see why the couldn't do it like that. Yes, their solution is quite nice. That kind of thing.

I very frequently refactor PRs at work and show my refactoring to nobody. Usually it's 6 of 1 half a dozen of the other. However, it gives me an idea about how flexible the code is. Or I might start to implement the next feature that's likely to touch that code. Not TDD. Just sketching some code and seeing how easy it is.

If you've never tried it, I highly recommend it.

Stating the obvious. Reading source code, no matter what quality it is will make you a better engineer. Reading software architecture focused material (books, blogs...) will make you a better architect.

Some suggestions for the resources in the architecture domain?

Actually, I wouldn't mind a comprehensive guide about it. It needs not to go in the details in all of them but at least should name them.

Read the source of the libraries you use and the functions you want to modify or fork.

Reply on news.ycombinator.com