KX Systems (KDB/k/q) overhauls its website | Hacker News Reader

q) overhauls its website

kx.com

26 pointskiyoto29 comments

Any HNer uses K/Q/KDB for personal/commercial purposes? What do you use it for?

Can't say I've ever really found a personal use, but in my StatArb days it was invaluable.

Want to calculate the 180 day rolling VWAP only between noon and 2pm for every NYSE stock? Yeah, there's a function for that.

Also, it's a hilariously illegible language, and most anyone who's proficient at it is as nutty (and brilliant) as Arthur ( http://en.wikipedia.org/wiki/Arthur_Whitney_(computer_scient... ), the founder of the company.

Holy crap. Looking at examples from the wikipedia page: http://en.wikipedia.org/wiki/K_(programming_language)

The following expression sorts a list of strings by their lengths:

x@>#:'x

A function to determine if a number is prime can be written as:

{&/x!/:2_!x}

This looks like brainfuck's cousin, but I'll bet Perl users feel right at home. Edit: I'm not trying to make fun of Perl or anything, it's just that I've seen some really hard to read one liners come out of it as well.

And K isn't nearly as bad as its predecessor APL [0], which makes extensive use of non-ASCII characters to bring even greater compactness.

For example, the Game of Life in APL is

    life←{↑1 ⍵∨.∧3 4=+/,¯1 0 1∘.⊖¯1 0 1∘.⌽⊂⍵}

[0] http://en.wikipedia.org/wiki/APL_(programming_language)

Like everything else in our business, it seems harder at first glance than it really is. It's a powerful language and like many powerful things it takes a while to learn. For instance here's my really rough I've-forgotten-most-of-K translation of that first snippet:

    x@>#:'x

Read from right to left:

:'x

means apply the function/operator to the left of the : to each item of x (the default first parameter to a function is called simply x - big time saver)

    #:'x

means the length of each item of x, and adding the sorting function:

means sort the items and return the indices the items would have if they were in ascending order. Finally:

is just the indexing operator (as you'd kind of expect when saying 'at'), so the whole line means: return the elements of x after sorting them by their length.

I interviewed at a New York financial services company that expected you to write Q on the whiteboard during the interview. That'd normally be fine in that market, I guess, but this was for a general operations and systems administration position.

Needless to say, I don't work there. However, that said, one of the graybeards that works there described his home MP3 collection which he wired up with a Q server, though.

For non-ultra-low-latency quantitative trading, almost all firms use KDB. At the ultra low level, KDB is too slow and cant take advantage of certain problem assumptions.

I assume, you're in the trading business. So, what do you then use for ultra low latency quantitative trading? Any insights would be nice :)

FPGAs are becoming increasingly popular

thanks for the tip. I thought gpu programming and cpu micro-coding were the popular things these days.

GPU programming is only useful in options trading because, in all other spaces, the limiting factor is bandwidth (afaik there are no gpu's with onboard 10g/40g/100g nics, so you have to pass data through PCI or some other bus)

true. Hm... good insight on the topic. So, what kind of programming languages and tools do you use for HFT? I've seen on the internet Erlang being used in the past, now it's said mostly C/C++.

If you happen to work on this sector, tell me smth: why don't you write your software in functional programming, and then use C++ bindings for the CPU intensive code?

Thanks

FPGAs have their own concerns, but they are useful as line-rate packet processors for data feeds (some try to build full-stack trading platforms, but in my experience those can't compete with a hybrid solution that uses CPU for logic and FPGAs on the ingress and egress)

Use it professionally for what it's intended for. After 5 years, I've come to realize that the language's limitations are too severe for building a robust analytics system. Also, the shortcomings of the database engine will eventually make you pull your hair out. Extending the language is a mean undertaking too.

Great for prototyping though.

...and your solution to production ready code is?..

so in trading what thing would you describe as robust for trading?..

Interesting comment - I occasionally kick myself for not diving into K/Q and J, specifically because I think of them being well-suited for "robust analytics system(s)."

I am on the edge of deep-diving into statistical modeling and have been considering using J (instead of R) to both learn J deeply and build my mathematical/statistical foundations back up.

As a fan of k/q/kdb purely from a hobbyist perspective, I'd love it if you'd expand on these limitations. I get that the language isn't everybody's (ok, is virtually nobody's) cup of tea. But it would be very interesting to hear about issues that go beyond taste, especially with respect to the database engine.

Can somebody tell which is the price range for this product?

If my memory serves me right, somewhere in the range of 100-200k per core for licensing.

starter pack is 25k for 2 cores

Thank you both for the reply, we dont do HFT but just some simple performance / indicators calculations so my dev colleagues use what they use for everything else (SQL Server)

Which one of the top ten investment banks don't use it? :)

I'm curious why they chose those particular syntax and operator naming conventions. Of course, syntax is superficial to some extent, but this seems a bit extreme.

If you're referring to the language, it's derived from J, which is derived from APL. There's a whole philosophy underlying this terseness, which is basically but probably inadequately represented as: mathematical notation isn't designed for amateur readability, it's designed as a tool for the mind of the working mathematician, so it doesn't need to be all that approachable to the newcomer.

> it's derived from J, which is derived from APL

Q is derived from K (it's a thin wrapper that uses words instead of sigils for some verbs), which is derived from APL; J is a completely separate lineage, also based on APL.

Yes! Also, interestingly Arthur Whitney created the initial one page fragment that started Roger Hui on J

http://www.jsoftware.com/jwiki/Essays/Incunabulum

http://keiapl.org/rhui/remember.htm

Brain fart. I was learning J but yes, it's K not J.

Spelling clearly wasn't a priority: "spohisticated"

Reply on news.ycombinator.com