Book Review: Designing Data-Intensive Applications | Hacker News Reader

Book Review: Designing Data-Intensive Applications

henrikwarne.com

169 pointsrspivak21 comments

I’ve also read this book a couple times but sadly as a engineer in large and mature company I hardly find any opportunities to actually apply or strengthen the learning, then as time goes my learning just decays.

How are you tackling the issue? It’s not like that one can find distributed system projects easily.

Considered switching jobs to something more interesting?

ouch

I have switched reasonably frequently in the last 5 years (approx 1.5 years at each job). Jobs are never as good as promised and sometimes turn put badly enough that you want to switch again soon. And its always a hassle switching. It does seem to be the most reliable way of getting a pay increase.

Well, keep your head up. There are some companies worth working for out there… there must be :) Maybe you start your own thing…

This book is on my reading list, but I haven’t gotten to it yet. However, I would like to share my experience, since I was in a similar position as you. At my previous company, I was a project engineer (similar to a TPM) and acts as the technical sidekick to the Program Manager. I provide engineering assessment and recommendation to the PM, and oversaw interdisciplinary engineering teams to ensure they are on track for the project. Most of the time the work was not intellectually stimulating, and there was not much room for innovation since the contract, scope, and waterfall schedule pretty much dictated the entire project flow.

What I did, was to seek projects outside of work. In my case, I self-studied python, algorithms, and data structures. I started to implement my own projects, inspired by 12 in 12 challenge by Mackenzie Child. Somehow, I was able to get an interview with Google through the Google Foobar challenge, and I am now working at Google as a SWE.

Like what I shared, I haven't read the book yet, but if I was to apply learnings in distributed systems, I would build one using virtual machines or single thread processes with message passing via TCP, UDP, or HTTP, each representing a system for a small scale prototype. After I am comfortable with the concepts and implementation, I’ll probably scale up using one of the cloud service providers (AWS, GCloud, Azure, Digital Ocean, etc).

I'd be really interested in hearing more about how you were able to do that transition to SWE.

I work in Data Science so I have Python experience (at least for scripting) but the leap to SWE seems massive, let alone at some FAANG company.

Sure.

At my previous position I was growing project management skills but felt my engineering skills was stagnating. I decided learning Python on the side. Based on HN recommendation, I started reading the CLRS algorithms book and implemented all data structures and algorithm with Python up until red-black tree. After that, due to time constraints I only implemented select algorithms from the graphs section.

After I had a better foundation on data structures and algorithms, I used leetcode and hacker rank as a test bed for my implementations. It is great, since the submission tests cover correctness, edge cases, memory and time constraints.

During this journey, I somehow triggered Google Foobar challenge with my Python and algorithm searches. This was the turning point—-I had tried to apply to Google before, but even with referrals it went nowhere due to my background not matching what G was looking for. Foobar, if you didn’t know, is a timed online programming challenge similar to leetcode. It consists of 5 levels, and Google will give you a phone interview if you completed level 3. This allowed me to bypass the HR filter!

So I started prepping for the interview in anticipation. I did the usual study routine—-cracking the coding interview and lots of leetcode. I wrote a lot of code on paper, completely filling up 2 notebooks. My goal was to be able to pick a random leetcode problem, handwrite the solution in 20 minutes, and have it pass all submission tests. Eventually I was able to do that for most problems at easy/medium difficulty.

The rest is just going through the interview process. Phone, then on-site, the nervously await the result. And you know the rest.

The whole journey took about 1.5 years, starting from learning python and reading CLRS. I know there’s a lot of dislike for the whiteboard algorithm interview, but in my case it was a way to pivot to SWE despite not having the background and experience, and I am grateful for it.

You are a true inspiration for every developer. In the online programming forums such as HN and Reddit you see lots of entitled people who keep saying how poor the whiteboard interview process is. These people are mostly bitter they cannot compete with self-motivated, hardworking and talented people like you. You are the exact kind of developer Google and other big tech companies are looking for. I have no doubt you'll have a great career as a SWE.

I've previously written about my experience with this book. It's so good that I'm starting to recommend it to CS students and early Software Engineers as a great introduction to the theory around many cloud services.

Every book should be written like this one.

My experience with this book and my opinion of it -- both highly positive.

A detail I really liked were the maps. Somehow I feel they helped me retain more concepts.

Such a good book, recommend it to any software engineer.

Does anyone know of other books like this? That teach theory with good pedagogy?

From the ones I've read: The Head First series of books are really really nice... other such books... D Crockford's JavaScript: The Good Parts; B Eckel's Thinking in Java; Skiena's The Algorithm Design Manual; M Nielsen's Neural Networks and Deep Learning; J Kurose's Computer Networking: A Top-Down Approach; R Lafore's Data Structures and Algorithms; J Bloch's Effective Java.

I'd add Andrew Tanenbaum's "Operating Systems: Design and Implementation"

Can anyone recommend me a well written book on distributed systems? I've finished reading Designing Data-Intensive Applications last year and enjoyed it a lot.

I love this book! After reading it, I can finally say I know something about data. The book gave me a systematic view of the technics related to the data. I marked lots and gave the book to my intern.

I require this book for my Databases class since nowadays we cover more than just the relational model. I hope students will keep it.

Would anyone recommend a good book for devops or CI/CD?

Excellent book. I’ve read it a few times now.

Reply on news.ycombinator.com