HNHacker News
TopNewBestAskShowJobs

misterHN

-2 karma · joined September 12, 2017

submissionscomments
misterHN··on How should you build a high-performance column store for the 2020s?
put data in text files, ASCII printable characters, one data point per line

put data files in directory

name data files after columns

use ".data" filename extension for data files

write a tool to create index files (append ".index" to the name of the input text file) that map record number to byte offset in data file

If data files are all < 4GB, use a 32 bit unsigned integer to represent the byte offset in the index file

Each index file is a packed array of 32 bit integers

Write a tool to create length files ".length" that count the number of entries in a data file

Generate .length files for all data files

Use mmap to access index files

Use C for all of the above

This is for variable-length data values. Not every column will have these, making the .index files redundant in this case; the .index files should not be created in this case and program logic should support both uniform value length access and nonuniform value length access. The reason to prefer two access modes is to keep data from the .index files out of the cache when it is redundant.

When all of this is done, the next thing to do is write a tool to test the cache characteristics on your processor by implementing sorting algorithms and testing their performance. Unless you are using a GPU (why?) all data your algorithm touches will go through every level of the cache hierarchy, forcing other data out. If possible, use a tool that reports hardware diagnostics. These tools may be provided by the processor vendor.

Now, there is a trend to give the programmer control over cache behavior

https://stackoverflow.com/questions/9544094/how-to-mark-some...

I don't know if this is worth exploring or a wild goose chase. It may improve performance for some tasks, but it sounds a little strange for the programmer to tell the computer how to use the cache...shouldn't the operating system do this?

Anyway, that's a start.

misterHN··on Jewelbots – A Friendship Bracelet You Can Code
friendship bracelets are made, and the raw materials are strips of flexible plastic
misterHN··on Eric Weinstein – AI Can Now Self-Reproduce–Should Humans Be Worried?
It sounds like nonsense
misterHN··on Teens 'rebelling against social media', say headteachers
Facebook and Twitter are evil, period. Good kids.
misterHN··on The Land of Lisp
I think you want to read the cartoon guide to the computer
misterHN··on Longer jail terms for viewing terror content online
which law?
misterHN··on Ask HN: What is the best way to spend my time as a 17-year-old who can code?
lol launch a democracy bot

anythingbot, chaosbot

If you do, post it to HN, /r/javascript and /r/programming

If you want money, go to /r/personalfinance and ask for help

I would avoid making money with code. Google and Facebook are evil, period. If you try to make money with code, you deal with the devil eventually.

misterHN··on The design side of programming language design
you have a matrix, one axis is names of programming languages and the other axis is names of things a language can do

the entries are 0 or 1.

If language L can do thing T, then there is a 1 in in cell L,T. Otherwise there is a 0.

For example, a thing could be, "you can feed it a math text and it will give a list of defined terms and expand the definition of any term in the list in the signature {<-}.