Linux kernel exceeds 15 million lines of code | Hacker News Reader

Linux kernel exceeds 15 million lines of code

h-online.com

50 pointsgaoprea34 comments

Can someone explain how such a HUGE codebase is maintained and new people get acquainted with it? I understand the distributed nature of development, but still, this seems like LOTS of code especially for someone new and who wants to get started in terms of contributing.

Seven easy steps to get acquainted with the Linux kernel:

Step 1: Build gnu binutils, gcc and gdb from source for the correct target architecture

Step 2: Setup qemu, some kind of virtualization or get a gadget that can run linux so that you can run and debug the kernel without repeated rebooting. (in the early days, Linus actually used to reboot to test the kernel, booting from floppy disks. I wouldn't have the patience)

Step 3: Configure and build the linux kernel from git master branch

Step 4: Make an initrd root filesystem, e.g. uClibC + busybox.

Step 5: If applicable, install a bootloader. Boot to your new kernel.

Step 6: Hack with the source and/or compile time or runtime kernel configuration

Step 7: Start over from the beginning but do something differently.

Steps 1-5 takes about one saturday afternoon. The faster your CPU, the less there will be waiting.

You have to start somewhere, so pick an area of interest and work your way around from there. Don't start in the beginning of the boot code, that's arcane magic that's not very useful for learning.

There are several books written about the Linux kernel, so you might want to pick one of those. Of course, printed material about the kernel gets old fast but the basic foundations have been pretty solid.

> in the early days, Linus actually used to reboot to test the kernel, booting from floppy disks. I wouldn't have the patience.

Still required all the time if you develop drivers for physical hardware.

Still required at some point if you develop other kernel mode components because you can get bugs reproducible only on physical hardware because of different timings, cache coherency bugs (rare, but more often than you think) or broken ACPI routing tables (Every motherboard in existence has a broken ACPI table and the kernel has to deal with it).

> Still required all the time if you develop drivers for physical hardware.

Thankfully, this is not quite true. Most Linux drivers are implemented as kernel modules, which can be loaded and unloaded at runtime. No rebooting necessary.

The guys who develop drivers for hardware that cannot be implemented as a kernel module (like some motherboard chipsets) probably have another machine for debugging or have some kind of dev boards to make their workflow more efficient.

Thanks for telling me how to do my job, I'm a kernel developer :-).

A bug in kernel code means you panic the machine, so you have to do a reboot.

Whether you load the modules at runtime, on demand, or the driver is statically compiled into the kernel has nothing to do with it. The decision of when to do linking does not influence development a bit.

Also, having multiple machines is required, not optional, because you use the dev machine to connect with a kernel debugger through a serial/firewire link to the test machine.

You're not required to boot from floppy disks these days :-) You can even test on another machine without installing the new kernel by using network boot.

Of course, I was just saying the reboot - test cycle on a physical machine is still required, details are insignificant :-).

The development model has various people being responsible for integrating patches in various subsystems, with integration done in a tree-like fashion. A lot of people are contributing reviews. To get acquainted, there are some tutorials about writing minimal modules, and reference books like Linux Kernel Development, third edition, by Robert Love. If some area has changed since the book was out, LWN (Linux Weekly News) has an article about it, often quite detailed.

Most of the code is system-specific, either hardware or filesystem drivers. And believe me, that stuff is heavily compartmentalized or the kernel would have collapsed under its own weight long ago.

I don't know about the kernel specifically, but most large code bases are modular.

There's probably a small set of core kernel functionality that provide services for everybody, and then specialized interfaces for each of the driver types.

I would guess that few people are familiar with the entire structure. Most probably know the core interfaces and the design of the subsystem where they mostly work, like wireless drivers.

Non-code lines shouldn't be included as they don't get compiled, otherwise you are just penalising for documentation

Sorry, but I absolutely, completely disagree.

Why? Documentation has to be maintained.

The LOC metric isn't about the cost to the computer to execute the code. What is IS about is the cost to the developers to create and maintain it.

Would be interesting to see comparable numbers for other operating systems.

There are some numbers available on Wikipedia[0]. Comparing with Windows is especially difficult though, because there is no differentiation between the kernel and graphical systems or system tools.

[0] http://en.wikipedia.org/wiki/Source_lines_of_code#Example

I've worked on Windows. I never really did a thorough analysis, but I'd say kernel mode components total around 3MLOC (out of ~50MLOC total). Definitely smaller than Linux.

Presumably there aren't all that many device drivers in there though? I'd imagine that mostly lived off in a WHQL repo or something in a different department.

Drivers made by Microsoft are in the tree. WHQL'ed 3rd party drivers offered by Windows Update are somewhere else, but there's no source for those so it's hard to count lines of code.

On a typical computer more than half of the drivers loaded are made by Microsoft, you can check easily with sysinternals' autoruns.

And they are in the kernel? What are dozens of .sys files in drivers folder then?

There are at least three different definitions of "in the kernel":

- code that compiles to the single executable file that runs the OS (this can be limited to a mechanism for booting the system, a thread scheduler, basic memory allocator, but this may also contain some device drivers, file systems, etc)

- any code that compiles to anything that will run in kernel mode (many file systems and device drivers)

- the code that someone (say the people distributing what they call "the Linux kernel") call the kernel.

This count is about the third. I do not think it contains a kitchen sink, but it is close :-)

I've got a lot of .ko files in /lib/modules; that doesn't mean they aren't part of the kernel.

Linux supports a wider range of hardware than Windows.

Sure, I'm just offering some stats for comparison, I'm not trying to say anything in particular.

Yup, and I'm just adding something to consider when interpreting it.

On the other end of the scale, the Debian lenny archive (released in 2009, superseded by squeeze) totals about 324M lines of non-comment code: http://gsyc.es/~frivas/paper.pdf

Thanks, I was just going to ask what is considered a line it sounds silly but I couldn't understand if a "line" was the full length of a screen at a certain resolution no matter if it was filled with characters. I know a space (spacebar) is a character too but not the blank space before you enter space character.

I'm not a programmer but I know you can cram certain methods, functions onto one line instead of arranging it on separate lines for easier comprehension by other programmers. As well comments and blank lines would add a lot depending on the style of the programmers doing it and if a blank line is considered a line.

Logical SLOC and physical SLOC make sense.

Especially the driver breakdown, since many other kernels don't ship with the same number of drivers that Linux does.

So freaking what?

EDIT

It's not a valid metric. 15 million can be a lot or a little. It's all relative. This is 15 million LoC of plain C code. This is code that includes thousands of device-specific routines, but even if it didn't, what are you comparing it to?

Let's say it is too long even compared to other C kernels w/ device drivers. What price are you putting on well-written code? Do you know how many of those lines are hints to the compiler? Those don't even get compiled.

Pretty arbitrary benchmark. Reminds me of this.

http://xkcd.com/1000/

Has anyone out there done refactoring experiments? Perhaps as an academic exercise? I don't know the kernel well enough to say for sure but I do recall a fair bit of duplicated code in e.g. the drivers. It would be interesting to see what could be done there.

15 MLOC?

That seems like allot , this must include allot of things that can optionally be compiled into the kernel?

Last time I browsed the kernel source tree it looked like a few 100KLOC to me.

Like the article says, something like 75% of all of that is driver specific or related to filesystems. You can still compile a minute little kernel for embedding that will be a fraction of the size of the full kernel. And there's still quite a lot of old cruft in there that could be left out.

Even so, if true, it does seem to be getting quite bloaty. Still, it has to cover a lot of ground nowadays in terms of hardware.

"75%...drivers...filesystems...it does seem to be getting quite bloaty"

Does the code that supports those drivers and filesystems slow the overall system down in any way? If not, how can you call it "bloat"? Bloat in this context generally means additional code that is out of proportion to its functionality while also using a disproportionate amount of memory and diskspace. There is legitimate bloat in the linux kernel but suggesting that that has anything to do with the variety of hardware support is to misunderstand the meaning of the word. Oh yeah, YMMV

Look ... 95% could be pure Assembler. OK?

No, its largely irrelevant nowadays. But there's a pile of cruft in there that could go ...

I'm not critiquing ANYTHING about the kernel. BUT. There IS a HUGE pile of crap still in there and everyone knows it. Don't nitpick my innocent and uncritical comment and try to turn it into a major critique of the kernel. Its still WAAAAAAYYYYY superior to the shite that still exists in Windows (because it simply MUST).

NO. Of COURSE it doesn't slow down the Linux experience but it might one day become too large to be realistically distributed without a lot of trimming.

I've been a major user, developer and proponent of Linux for decades. I use nothing else in my day to day life. Don't dare to presume to pull me up because I dare to comment on a seriously discussed and current problem with the current kernel. NAMELY: IT'S BECOMING A FAT BITCH AND EVERYONE KNOWS IT.

It's still and always will be my OS of choice. Whether or not I knew how to exclude certain chunks of irrelevance from my build or not.

bloat: n. Any feature I'm not using at the moment.

Reply on news.ycombinator.com