I stumbled upon Google's whitepaper on Dremel. IIRC it explained how to store the data in columnar format, but I didn't quite get how that translated into quick queries. Happen to know where I can look to better understand how it works?
"Analytical queries" are faster in a column-based format, because you're doing things like computing the correlation between 2 variables. Instead of looking at many columns in a specific row, you're looking at most of the data in a few columns. So instead of reading a whole row at once, it would be nice to skip the columns you don't care about and just grab big chunks of two or three columns.
Does that make sense?
edit: For a long time I was confused about why HBase was described as a columnar data store when access was still pretty row-based. I think the reason is because you group columns into column families which can be stored and retrieved separately, so you still get some of the benefit of a true column-oriented store.
The term was overloaded by column-family stores, which were often referred to as just 'column stores', probably by people who were not aware of systems like Vertica and MonetDB.
http://dbmsmusings.blogspot.co.uk/2010/03/distinguishing-two...
Since the column values are stored together, more of them can be crammed into a data page. Querying the data of a column can process much more data per page load than row-based, whereas a row-based table's data page has all other columns.
The values of a column can be stored in a sorted order, which is highly compressible, enabling cramming even more data into a data page. You can get much more data with each data page loading. E.g. The column Name is stored as sorted along with the associated row id.
Name, (row id)
--------------
Joan, (101)
Joanna, (307)
John, (15)
John, (32)
John, (6)
Johnson, (31)
Johnson, (44)
Name, (row id)
--------------
Joan, (101)
Joanna, (307)
John, (15,32,6)
Johnson, (31,44)
Name, (row id)
--------------
Joan, (101)
4+na, (307)
2+hn, (15,32,6)
4+son, (31,44)
Select Name from Table where Name like "John%" would do a binary search down the sorted names, load two records 2+hn and 4+son, and expand them into 5 names.