Hacker News Reader

Comment by insaneirish | Hacker News Reader

Can I add another one? How about "The Network Doesn't Work The Way You Think It Does: A Distributed Systems Survival Guide".

As someone who has spent a good chunk of their career building high performance networks and another chunk of it working closely with developers who care a lot about those networks, I'm constantly amazed at how little top tier graduates from well respected schools know about networks.

Relatively basic stuff, like what does your application's traffic look like on the wire, how an OS determines what to do with a packet, how latency affects applications, and many other things are completely foreign to way too many developers. I could go on and on, but given the connected nature of things today, it seems like a very overlooked area.

View on HN

This is Networks and Distributed Systems at UChicago:

CMSC 23300. Networks and Distributed Systems. 100 Units. This course focuses on the principles and techniques used in the development of networked and distributed software. Topics include programming with sockets; concurrent programming; data link layer (Ethernet, packet switching, etc.); internet and routing protocols (UDP, TCP); and other commonly used network protocols and techniques. This is a project-oriented course in which students are required to develop software in C on a UNIX environment.

http://collegecatalog.uchicago.edu/thecollege/computerscienc...

The 3 major C programming assignments last year: an IRC server, a TCP stack, and a router.

Best course I've ever taken.

I dug around and found the course projects: https://github.com/uchicago-cs/cmsc23300/wiki

Thanks for this.

Should be considered a typical CS course.

I know a few good developers that don't even know how a computer works. Everything is so abstracted you never really have to know.

As a devops/infrastructure guy, this drives me nuts. You abstract everything away, but then are shocked when you don't understand what's happening when the underlying layers fail or break down in unexpected places ways.

It's rapidly becoming a two tier world with two types of programmers: those who are well trained in coding within known environments, and those who are like the folks in The Matrix films who ate the red pill-- they know what's really going on.

It's been like that for decades. Or do you really know what assembler your program compiles down to?

For some of them, absolutely. I change the assembly output and reassemble to verify that the atomics are necessary and fixing the cases I think they do.

As far as I'm concerned, if you're not removing the atomic instructions to make sure the breaks happen that you expect to happen, you're not really testing your code.

I'm an embedded systems engineer. It's always been that way for me: I own and compile the bootloader, kernel, filesystem, runtime libraries, and the application code. I have the schematics and gerbers for my boards. I have the reference manuals for every chip.

What's weird is watching people play with stuff like Raspberry Pi and they expect things to just apt-get and launch like it's always been done for them. And most of the time it does just work that way. Except when it doesn't.

many development enviroments would do good with some graphics representation of how much of the plattforms capabilitys is actually used up during a test run, how cache friendly the code is, and how much your latest changes affected this. And this coloured columns need to show in every svn/git commit message. Profiling needs to be not something that the embeded guy does after you messed it up.

Jerry Commited: +25% more time spend in function xy, due to chace unfriendlyness..

If you see it, and your colleagues see it, it becomes a issue. If it becomes a issue the perpetrator will look into it.

I understand your frustration, but the field has become too big for a single (average) person. When problems arise , you team up with someone who knows more about the issue or you learn on the spot. Nothing wrong with not knowing everything.

That's not my problem. My problem is the illusion presented that you don't need to understand the underlying infrastructure. Far too often I see infrastructure as an afterthought, secondary to the code.

What kind of organisations have you worked in? I've been lucky (?) to work in an organisation with a dedicated network/infrastructure team. My work life got a lot better when I gained more understanding of how the network was put together (vis-a-vis load balancers, DNS, data center locations etc).

I still don't have a "low level" understanding of the network I use.

I've always managed or been on infrastructure teams at other orgs; my latest gig is at a startup, and I'm the only infrastructure guy. Perhaps that is the problem.

Good chance to educate others? I know I'll be looking for an opportunity to work with an infrastructure guru at some stage in the future.

I'm going to embrace it as such! Thanks!

Fundamentals as small, compact and trivial. Everyone can get a good grasp of all the basics before going on to any "real world" coding with its diverse, needless complexity.

More likely when problems arise you "team up" with someone who knows [almost] everything.

And I agree with other posters that this isn't something new. That's the separation between senior and junior specialists (and no - if one doesn't know what network latency is, that one is not a senior frontend engineer).

You don't really have to know the internals but I personally think you ought to know the abstactions themselves at the very least. To be more specific, you need not know say the internals of your compiler but you ought to know how compilers in general work. Same with protocols, filesystems, OS {vm,scheduler} ...etc.

A good developer must know at least about the standard latencies and about the memory hierarchy. Otherwise it is a really bad developer who is going to keep annoying the innocent users.

I am a recent CS grad and I am constantly amazed by how little I know about networks. Do you have a favorite book or online resource on the subject?

One fun thing is to download Wireshark and look at the packets being sent in detail when you do common things like browse the internet or use FTP.

I think this is a really good idea, but I'd add one alternate possibility is well. Doing this on a typical user's computer today will present a mind blowing amount of data--so much that it is easy to give up trying to figure out what's going on if you've never done it before. Instead, perhaps spin up a Linux VM, maybe even no GUI to start. Then run Wireshark on its interface. Much less noise. Should be much easier to begin dissecting.

No need for a VM, just do some filtering in Wireshark. This will help you understand even better what is possible (filter by IP, port, and many other things).

It's even more fun to do it on a network, where you can see what things other people are doing.

Agreed. I took a networking course at University, and at community college, but didn't really understand networking until I started fooling around with Wireshark.

For a little softer experience, you can load up Fiddler and just look at the HTTP traffic. It's a little mind boggling to me to envision trying to debug web apps without Fiddler, Chome devtools/Firebug and server-side logging inspectable at the same time.

Also a little terrifying to realize just how much incoming and outgoing traffic all the things on your computer are generating all the time.

I'm a fan of lecture notes. http://www.cc.gatech.edu/~traynor/cs3251/f13/slides/

Computer Networks by Tanenbaum: http://www.amazon.com/Computer-Networks-Edition-Andrew-Tanen...

It's a classic.

TCP/IP Illustrated is what I cut my teeth on back in the day. You will need to be able to read C to follow the code, which is mostly from the BSD TCP/IP stack IIRC.

The Stevens series is a classic, and you should find a used copy to stick on a bookshelf for the nerd cred and because they are damn fine boos, but I think you will find the Kozierok's TCP/IP Guide from No Starch Press to be a more modern and up-to-date text.

The CISCO press books for the CCNA course are fairly good.

I actually did the proper 4 semester CISCO academy course at night school a while back which should give you real hands on experience - one of the tests is the instructor breaks a system and you have to fix it

I ended up doing the CCNA course just to figure out how networking works. I was designing a distributed trading system and I wanted to actually understand how things worked rather than just cargo culting a few commands and praying. There's just so many acronyms in the networking space it's hard to figure out without an overview.

Even with the CCNA course, you're still not going to know about stuff like security and firewalls on a deep level.

But get the textbooks and read through them.

This is probably complete overkill , but my favorite is still McDysan/Spohn "ATM Theory and Applications." The first few chapters before you dive into the details about ATM are a very good overview of all communications systems.

Closely related I think: breaking into software / hardware / networks. A course that has people doing basic and maybe not-so-basic exploits on real systems. Including attendance at a conference such as DEFCON.

Our future software engineers need to have security as a fundamental concern to software engineering and not as something that's an 'add-on'.

A thousand times yes. I've met tons of good programmers, even CS/CE Ph.Ds for whom networks and anything wire-releated is black magic beyond mortal understanding.

Not only does it prevent them from dealing with or writing such systems, but it's also a major contributor to clueless security FUD. It's not that there's nothing to be afraid of... it's that if you don't understand how networks work you are probably afraid of the wrong things.

https://www.nsnam.org looks like a good idea, but my head is spinning from the first tutorials - I'd love a "network simulator" that could just replay pcap files. Do you think that programmers watching and understanding some of what's going on in a Wireshark session would help?

Also found via the course wiki - http://mininet.org/

Part of my bachelor of Software Engineering included calculating to the nanosecond the latency of networks from the kernel interrupt to the other side (including across copper, fibre, and waiting in RAM buffers along nodes on the way)

Network speeds I don't know about, but the book "High Performance Python" comes pretty close to the description of "CSCI 4020: Writing Fast Code in Slow Languages". Check it out!

Thanks for the tip. I've been exposed to Cython a little, but my experience was very ugly... will check this out.

Awesome. I was just about to say I'd pay for CSCI 4020.

my school offered both networking and distributed systems courses, and it was a tiny school with an even smaller cs department.

The problem is getting practical experience in modern techniques.

Reply on news.ycombinator.com