HNHacker News
TopNewBestAskShowJobs

clausok

240 karma · joined May 13, 2018

hovering in the air just like a brick doesn't

karl(ɑt)optimalxl.com

submissionscomments
clausok··on Ross Perot Has Died
Here's a navy seal telling a story about Perot helping him figure out what was wrong with his son -- turned out to be an incredibly rare genetic disorder.

https://youtu.be/snMzYfhXScQ?t=5065

clausok··on Show HN: Bvckup 2 – Fast File Replicator for Windows
I really like your landing page. FYI: on the feature comparison page, when you expand a row with the '+', the 'More on this...' links are not working yet.
clausok··on Filing Your Taxes Is an Expensive Time Sink. That’s Not an Accident
The Planet Money podcast just had an episode on this topic ("Joe Bankman, professor at Stanford, figured out a way to make filing your taxes easy and painless. Then the tax lobby found out about it.")

https://www.npr.org/sections/money/2017/03/22/521132960/epis...

clausok··on RenTech Created the Ultimate, Tax-Free IRA Account for Employees
Particularly when these analytical advantages may compound over time rather than dissipate. For instance, suppose a firm was exceptionally good at pairing world-class mathematicians with a team of mathematically-exceptional programmers who could reconceptualize the math wizard's insights as financial time series models and steer them towards areas of applicability. Now, when the next world-class mathematician comes to the office on a recruiting trip, he recognizes a higher expected value & better colleagues, joins, and the engine gets more repetitions with which to improve itself.
clausok··on RenTech Created the Ultimate, Tax-Free IRA Account for Employees
I've seen systematic portfolio managers who switched firms and then took a year or more to begin trading -- when they expected a maximum transition time of a couple months -- because they took for granted the tools and data infrastructure at the prior firm. Even tasks as mundane as handling ticker changes can have an unexpectedly large time cost when you're the one who has to handle them.

That may be why there doesn't seem to be a large rentech diaspora, as opposed to Julian Robertson's tiger cubs, for example. One can't take the process with them and it's really hard to reinvent.

clausok··on RenTech Created the Ultimate, Tax-Free IRA Account for Employees
Here are a few insights on the rentech process from Nick Patterson, one of their senior statisticians there for a decade:

[...I joined a hedged fund, Renaissance Technologies, I'll make a comment about that. It's funny that I think the most important thing to do on data analysis is to do the simple things right. So, here's a kind of non-secret about what we did at renaissance: in my opinion, our most important statistical tool was simple regression with one target and one independent variable. It's the simplest statistical model you can imagine. Any reasonably smart high school student could do it. Now we have some of the smartest people around, working in our hedge fund, we have string theorists we recruited from Harvard, and they're doing simple regression. Is this stupid and pointless? Should we be hiring stupider people and paying them less? And the answer is no. And the reason is nobody tells you what the variables you should be regressing [are]. What's the target. Should you do a nonlinear transform before you regress? What's the source? Should you clean your data? Do you notice when your results are obviously rubbish? And so on. And the smarter you are the less likely you are to make a stupid mistake. And that's why I think you often need smart people who appear to be doing something technically very easy, but actually usually not so easy.]

[[at] my hedge fund, which was not a very big company, we had 7 Phd's just cleaning data and organizing the databases]

http://www.thetalkingmachines.com/episodes/ai-safety-and-leg...

quotes from ~30:06 & ~38:03

clausok··on Memory usage of a toy C# server and client with 500K concurrent connections on
Don Syme, on omitting ocaml-style modules from F#:

In addition, there was the question what not to implement. A notable omission from the design was the functorial module system of OCaml. Functors were a key part of Standard ML and a modified form of the feature was included with OCaml, a source of ongoing controversy amongst theoreticians. The author was positively disposed towards functors as a “gold standard” in what parameterization could be in a programming language but was wary of their theoretical complexities. Furthermore, at the time there were relatively few places where functors were used by practicing OCaml programmers. One part of the OCaml module system – nested module definitions – was eventually included in the design of F#. However, functors were perceived to be awkward to implement in a direct way on .NET and it was hard to justify their inclusion in a language design alongside .NET object programming.

The Early History of F# https://fsharp.org/history/hopl-draft-1.pdf

clausok··on Applying programming language research ideas to transform spreadsheets
Where has NetOffice.Excel been hiding?! Thank you for that.

What I've found with interop is there's a little more debuggability/edit-and-continue friction that can accumulate into a large loss of time during development. Maybe that loss is offset, or more than offset, by doing the interop with a vastly superior language. I'm not sure. Any thoughts about this trade-off?

clausok··on Applying programming language research ideas to transform spreadsheets
Especially with ExcelDna, it's hard to make an argument for using VSTO. And even with ExcelDna (i.e., an xll written in F#\C#), I find there are still niche areas where vba is effective, particularly when working with the Excel object model. The split I like is to have vba orchestrate interactions with the object model, Sheets, Books, Ranges, Range.Find(), etc. So for example, have your F#\xll code parse "60d" into the last 60 trading days, and then use vba to select those days in Excel's autofilter.
clausok··on Applying programming language research ideas to transform spreadsheets
Crazy stuff indeed. I saw a type of "click-once", auto-update deployment for vba code at an IBank in 2000 that worked wonderfully. Excel models built by trading desks and investment banking groups themselves are often constructed under a lot of time pressure and one thing about Excel is you can do it wrong and it may still work reasonably well. I have seen some specialty Excel software teams in these environments that refactored those models and made them remarkably robust and coherent. Much of it is just having the time to study alternatives: what part of the model should be on the sheet in formulas, what part in vba, what part in another language via an xll? What questions are the users going to ask and what calculations do they need to be able to see and understand? Where does the flexibility need to be? If a user moves a sheet to another workbook, will the button on it still work? How do you deploy vba code so that if you fix a bug all the models that use that vba code, and all their 'save as' descendants, run the fixed code.

There are clever ways to answer these questions but if you condescend to Excel as a development platform, and believe you can't build anything great with it, you may never brainstorm enough to uncover them.

clausok··on The Early History of F# [pdf]
I share the same regret. In fact, by far my biggest regret in my career was missing an opportunity in 2008 to switch to F#. The portfolio management team I was with departed a large hedge fund to start our own. I knew I had 8-10 months before we started trading again. It was every programmer's dream: hit the pause button and get a chance to leisurely refactor everything you've never had time for. Even enough time to completely change tool sets. I remember discovering F# during this period and it was as though Don Syme (the F# BDFL) had listened to all my complaints about C# & Matlab and built a language specifically for me. Ultimately, I chickened out. I was afraid Microsoft would eventually Foxpro F#; or I'd hire people who'd tell the CEO of our firm that I was nuts and had built all our systems in an obscure language. By late 2012 the ecosystem had matured tremendously and I finally switched to F#. I really wish I had those four years back. F# would have saved me a ton of while-on-vacation troubleshooting from 2008 to 2012.
clausok··on Business Programming Languages and the Legibility of Programming
True. One time the portfolio manager I was working with wanted to examine an aspect of our systematized hedging rules. He was a non-programmer (unless you consider writing Excel formulas to be programming). I started to read through our F# code where we formalized our hedging rules and translate them into an email explanation. It suddenly came to my mind, "I'll just take a screen-shot of the code here and see if that does the trick". It struck me that the format of the F# active patterns and the layout of the code was a better depiction of the logic than I was achieving with my written explanation. So I sent the screen-shot. And he wrote back, "ok, got it." That became the first of many. I started writing and formatting my code with the expectation that I would be sending explanatory screen-shots to the PM and trading desk. It turned out that approach made the code much easier for me to understand as well. And I agree that the ML family of languages have uniquely advantageous characteristics when attempting this type of legibility.
clausok··on Why you should learn F#
I disliked the module\file order restriction when I started, but have come to appreciate it because every time it felt like a thorn in my side it turned out that the thorn was prodding me towards a better organizational structure.

I've even found that apart from enforcing better design, it can also trigger it. As an illustration, I've had a function 'f' in module 'C' that I decided made more sense in module 'A'. When I moved the function however, module 'A' would no longer compile because 'f' was dependent on a module 'B'. No problem I thought, I'll just move module 'A' below module 'B'. Whoops, now module 'B' won't compile; I didn't realize it was dependent on module 'A'. If I resist the urge to just revert everything and return the function 'f' to module 'C', and I investigate "why does 'f' feel like it belongs in module 'A' and yet placing it there introduces circularity...", often I'll discover a beneficial refactoring that I probably would not have thought of otherwise.

clausok··on In Praise of APL: A Language for Lyrical Programming (1977)
Re how to get started: what was effective for me was to pair kdb+/q with another language and constrain how much I tried to do with q itself. q-sql for joining, filtering, pivoting and grouping (particularly date grouping and grouping with custom aggregate functions) is incredibly fast and powerful and also easy to learn. (though certainly, q experts will be able to point out all sorts of sub-optimal things in your queries)

Initially, we used F# to shovel data into kdb and to orchestrate queries and process results -- while keeping our q-sql simplistic -- and still saw astonishing speedups in our data processing. Over time, wherever we needed more speed, we'd do more and more in q itself (the 'Q Tips' book is very helpful).

One can get a lot of power with just a little kdb+/q.

clausok··on Wayne Gretzky and the Mysteries of Athletic Greatness
Good point. Perhaps the only player the hockey gods were as kind to as Nicholls was Kevin Stevens when he broke the 100 point level twice in the early 90's while playing on a line with Mario Lemieux and Jaromír Jágr.
clausok··on Ask HN: What's the largest amount of bad code you have ever seen work?
I worked on an Excel\vba swat team at an investment bank. Whenever someone in the various business units got themselves into more trouble than they could handle with an Excel model, we'd try to set things right. We saw a lot of wonderfully creative vba code from industrious programming beginners (or, as we liked to call them, "Macro Recording Cowboys". The most amusing I saw was a model where the vba code used a Greek mythology variable naming convention:

Dim Hermes As String, Artemis As Long, Odin As Range

clausok··on Ask HN: 40+ Career Advice?
"A players hire A players, but B players hire C players..." -- Steve Jobs

https://news.ycombinator.com/item?id=3102639

clausok··on The First Rule of Microsoft Excel: Don’t Tell Anyone You’re Good at It
>excel is not usually treated as an engineering tool

This is a good point. I've seen C# & Python devs who read the release notes and adopt new features of C#\Python with alacrity while at the same time their Excel development work, which they spend considerable time on, evolves not at all. E.g., still using IF(ISERROR(...)) rather than IFERROR; VLOOKUP rather than INDEX(MATCH()); slow & brittle array functions rather than SUMIFS; employing many intermediate columns to clean out errors rather than using AGGREGATE(); doing things in VBA that they would never do in their primary programming language (v = Worksheets("pnl").Range("N22:N77").Value).

clausok··on A look back: The Bloomberg Keyboard (2017)
For example, last year I kicked the tires on Bloomberg's PORT suite of portfolio performance attribution and risk measurement functions to get a sense of what they offered and how programmable it was. In addition to the standard extensive help docs, they had 56 white papers totaling ~1400 pages (as of June 2017) explaining the calculations. And that's for a small star in the Bloomberg constellation.
clausok··on A look back: The Bloomberg Keyboard (2017)
That's true. There's a reasonably steady flow of articles on prospective "Bloomberg Killers"; this one in Institutional Investor (July 2017), Can Anyone Bury Bloomberg, is the best researched and informative that I've seen so far:

https://www.institutionalinvestor.com/article/b1505pdwt02550...

clausok··on Leveraging the type system to avoid mistakes
My own migration of a 7+ years C# codebase to F# (in 2013) began when I decided, belatedly, that I'm just not smart enough to pair a deep OO hierarchy with multicore parallelism without having my vacations spoiled with troubleshooting. The first thing I did was take a tiny part of the code, read the C# data into F# records and used F#'s Array.Parallel.map for the multicore crunching. Like magic, the intermittent threading bugs I had with that small part disappeared, so I kept going. I hadn't planned to rewrite everything in F#, but I had so much fun with it, and grew to love the language so much, that 18 months later, that's what had happened.

I'd recommend starting by nibbling around the edges: find small pain points where F#'s union types, immutability, pattern-matching, option types, active patterns, etc, make some small part of the software easier to understand and maintain. And then if you like how that goes, don't stop.

clausok··on Leveraging the type system to avoid mistakes
I've found there are situations where primitives are fine and wrapping doesn't add much value, e.g., (using F#)

  let calcCorrelation (symbol1:string) (symbol2:string) = ...
Here, calling the function with misordered arguments won't change the result, so the wrapping\unwrapping friction doesn't have much ROI.

but here, where 'beta' is the slope of a least-squares fitted line:

  let calcBeta (stockSym:string) (indexSym:string) = ... 
that's a mistake waiting to happen, because if you accidentally flip the arguments when you call the function you get a different number. Better to have:

  type StockSymbol = StockSymbol of string
  type IndexSymbol = IndexSymbol of string 
  let calcBeta (IndexSymbol index) (StockSymbol stock) = ...  
then call the function like this:

  let beta = calcBeta (IndexSymbol "SPX") (StockSymbol "AMZN")
Of course you could still make a mistake:

  let beta = calcBeta (IndexSymbol "AMZN") (StockSymbol "SPX") 
but that would stick out like a sore thumb.
← PreviousPage 2 of 2