240 karma · joined May 13, 2018
karl(ɑt)optimalxl.com
https://www.npr.org/sections/money/2017/03/22/521132960/epis...
That may be why there doesn't seem to be a large rentech diaspora, as opposed to Julian Robertson's tiger cubs, for example. One can't take the process with them and it's really hard to reinvent.
[...I joined a hedged fund, Renaissance Technologies, I'll make a comment about that. It's funny that I think the most important thing to do on data analysis is to do the simple things right. So, here's a kind of non-secret about what we did at renaissance: in my opinion, our most important statistical tool was simple regression with one target and one independent variable. It's the simplest statistical model you can imagine. Any reasonably smart high school student could do it. Now we have some of the smartest people around, working in our hedge fund, we have string theorists we recruited from Harvard, and they're doing simple regression. Is this stupid and pointless? Should we be hiring stupider people and paying them less? And the answer is no. And the reason is nobody tells you what the variables you should be regressing [are]. What's the target. Should you do a nonlinear transform before you regress? What's the source? Should you clean your data? Do you notice when your results are obviously rubbish? And so on. And the smarter you are the less likely you are to make a stupid mistake. And that's why I think you often need smart people who appear to be doing something technically very easy, but actually usually not so easy.]
[[at] my hedge fund, which was not a very big company, we had 7 Phd's just cleaning data and organizing the databases]
http://www.thetalkingmachines.com/episodes/ai-safety-and-leg...
quotes from ~30:06 & ~38:03
In addition, there was the question what not to implement. A notable omission from the design was the functorial module system of OCaml. Functors were a key part of Standard ML and a modified form of the feature was included with OCaml, a source of ongoing controversy amongst theoreticians. The author was positively disposed towards functors as a “gold standard” in what parameterization could be in a programming language but was wary of their theoretical complexities. Furthermore, at the time there were relatively few places where functors were used by practicing OCaml programmers. One part of the OCaml module system – nested module definitions – was eventually included in the design of F#. However, functors were perceived to be awkward to implement in a direct way on .NET and it was hard to justify their inclusion in a language design alongside .NET object programming.
The Early History of F# https://fsharp.org/history/hopl-draft-1.pdf
What I've found with interop is there's a little more debuggability/edit-and-continue friction that can accumulate into a large loss of time during development. Maybe that loss is offset, or more than offset, by doing the interop with a vastly superior language. I'm not sure. Any thoughts about this trade-off?
There are clever ways to answer these questions but if you condescend to Excel as a development platform, and believe you can't build anything great with it, you may never brainstorm enough to uncover them.
I've even found that apart from enforcing better design, it can also trigger it. As an illustration, I've had a function 'f' in module 'C' that I decided made more sense in module 'A'. When I moved the function however, module 'A' would no longer compile because 'f' was dependent on a module 'B'. No problem I thought, I'll just move module 'A' below module 'B'. Whoops, now module 'B' won't compile; I didn't realize it was dependent on module 'A'. If I resist the urge to just revert everything and return the function 'f' to module 'C', and I investigate "why does 'f' feel like it belongs in module 'A' and yet placing it there introduces circularity...", often I'll discover a beneficial refactoring that I probably would not have thought of otherwise.
Initially, we used F# to shovel data into kdb and to orchestrate queries and process results -- while keeping our q-sql simplistic -- and still saw astonishing speedups in our data processing. Over time, wherever we needed more speed, we'd do more and more in q itself (the 'Q Tips' book is very helpful).
One can get a lot of power with just a little kdb+/q.
Dim Hermes As String, Artemis As Long, Odin As Range
This is a good point. I've seen C# & Python devs who read the release notes and adopt new features of C#\Python with alacrity while at the same time their Excel development work, which they spend considerable time on, evolves not at all. E.g., still using IF(ISERROR(...)) rather than IFERROR; VLOOKUP rather than INDEX(MATCH()); slow & brittle array functions rather than SUMIFS; employing many intermediate columns to clean out errors rather than using AGGREGATE(); doing things in VBA that they would never do in their primary programming language (v = Worksheets("pnl").Range("N22:N77").Value).
https://www.institutionalinvestor.com/article/b1505pdwt02550...
I'd recommend starting by nibbling around the edges: find small pain points where F#'s union types, immutability, pattern-matching, option types, active patterns, etc, make some small part of the software easier to understand and maintain. And then if you like how that goes, don't stop.
let calcCorrelation (symbol1:string) (symbol2:string) = ...
Here, calling the function with misordered arguments won't change the result, so the wrapping\unwrapping friction doesn't have much ROI.but here, where 'beta' is the slope of a least-squares fitted line:
let calcBeta (stockSym:string) (indexSym:string) = ...
that's a mistake waiting to happen, because if you accidentally flip the arguments when you call the function you get a different number. Better to have: type StockSymbol = StockSymbol of string
type IndexSymbol = IndexSymbol of string
let calcBeta (IndexSymbol index) (StockSymbol stock) = ...
then call the function like this: let beta = calcBeta (IndexSymbol "SPX") (StockSymbol "AMZN")
Of course you could still make a mistake: let beta = calcBeta (IndexSymbol "AMZN") (StockSymbol "SPX")
but that would stick out like a sore thumb.