HNHacker News
TopNewBestAskShowJobs

jnordwick

2,704 karma · joined June 5, 2015

https://github.com/minimaxir/hacker-news-undocumented jnordwick@gmail.com
submissionscomments
jnordwick··on Toxic Productivity
Tell this to Paul Erdos.
jnordwick··on The x86 architecture is the weirdo (2004)
> as the alignment padding had made for a bigger cache footprint

I argues with some of the Rust compiler members the other day about wanting to just ditch almost all alighnment restrictions because I of this exact thing. They laughed and basically told me i didnt know what i was talking about. I remember about 15 years ago when i worked at market making firm and we test this and it was a great gain - we started packing almost all our structs after that.

Now, at another MM shop, and we're trying to push the same thing but having to fight these areguments again (the only alignmets I want to keep are for AVX and hardware accessed buffers).

jnordwick··on Using SIMD to aggregate billions of values per second
can you please contact me to jnordwick@gmail.com?
jnordwick··on Using SIMD to aggregate billions of values per second
those things are so useless if your time points arent equidistant (ie, you don't care about the last 100 rows, you care about the last 5 minutes). they basically force you to use very slow correlated subqueries. please do something better.
jnordwick··on Using SIMD to aggregate billions of values per second
columnar, probably. row-based not possible.
jnordwick··on Using SIMD to aggregate billions of values per second
- nano are important for keeping ordering. while you may return results in insert order it is nice to have them so any other operations done in them outside the db you can retain (or recreate) that ordering. in financial systems nanos have become a sort of defacto standard for this. for examsple, all our messaging timestamps at places i've worked are always nano for anything written in the last 5-10 years.

Also when you are trying to do calculations on high-frequency data (tick, iot) it ruins your ability to take meaningful deltas (eg, arrival rates) since you get a lot of 0s and 1s for the time deltas. Its difficult to take weighted averages with weights of 0s.

the issues solving that (if you really need a wider range) are easier to solve that having to force everything down to micros and creating ways around that. (eg, kdb uses multiple date, time, and timestamp types and it doesn't use the unix epoch since it isn't very useful for tick, censor, or any high-frequency data i've seen).

better than a double that some systems still use.

-kdb's secret sauce that people don't seem to understand is its query language that more naturally fits the time series domain. It isn't really a database as more it is an array language with a database component. (eg, try to write an efficient sql query that calculates 5 minute bars on tick data).

I actually like Java too - I've written or worked on a couple trading systems written in core java. just get good devs who understand what it means to write zero gc code, abuse off-heap memory, and understand what hotspot can intrinsify. If you can stay in the simd code for all the heavy lifting loops (filters, aggregates, etc), I don't think java will be an impediment.

I think you have parts going in the correct direction, and you seem to have good experience from looking at the bios. Nothing really un-fixable (or un-addable) in what I saw glancing at your docs. I did bookmark you to see how the db goes. Will prob check out soon.

jnordwick··on Using SIMD to aggregate billions of values per second
because they seem to only support that, from their page:

"As of now, SIMD operations are available for non-keyed aggregation queries, such as select sum(value) from table."

not even sure if they support where clauses on that, sums of functions of a column, or even other things like stddev of the column.

their storage format though looks good and simple (similar to kdb actually), but they really should have an 8-byte char instead of the 16-byte one since that would be far more used.

their partitioning scheme is only on time so less advanced than other system.

single designated timestamp column (so no bitemporal), but do support asof joins which is nice.

they totally screwed up on dates and time. dates only to milli and timestamps only to micros. huge mistake.

long256 which is nice, but strangely no long128s (which wind up being nice when you have base-10 fixed-point numbers normalized to a large number of decimals).

i didn't see any fixed-width string/byte columns. Does have 32-bit symbols (i assume similar to kdb?) that might cover some of those use cases.

some good and some bad in there. never going to compete with kdb or onetick on performance (and nobody competes with kdb on query language/extensibity) , but could find a niche based on price and having simpler more easily adapted to querying and more human interface.

jnordwick··on Ask HN: How to prosper under negative interest rates?
This is actually more true than people realize. Gold is down to flat even after the the huge Fed announcements. The CRB index has been getting crushed in the last month. But this short-term liquidity demand is going to go away. The real inflation story is going to be when it does and the economy settles down. Then we will have all these programs that pumped huge amounts of money into the system, and unless they are sanitized watch out. I've seen people losing a lot of money on gold calls, but i think the proper time for that is going to be when this short-term credit demand has dried up and business starts returning to normal.
jnordwick··on Firefox's low-latency WebAssembly compiler
1. You can have both. Jumps (conditional or unconditional allow for more complex control flow and different control flow than was immediately thought of.

2 only having structured control flow makes it a difficult target for HLL, which is what was is supposed to be.

3. A few years ago I was working on an array language and was trying to use tree based optimization and code generation, but I put it on hold until i understood more - can you recommend and links or papers?

4. Wasm is an IR, both target and source. Why not add a goto and structured primitive. There are clearly some things that cannot be compiled anywhere remotely performant.

jnordwick··on Firefox's low-latency WebAssembly compiler
also no conditional jumps (equivalent to jnz, jeq, etc...) instead of uses this weird br_if that can only go to places on the control flow stack.
jnordwick··on How to write documentation for a Rust crate
Not even a useful softdoc though. Let's say I need to search then mutate an data structure. There are probably number of ways to do what I need. Right now, it is almost impossible to see what my options are.

And I'm not going to memorize the API either. Sorry.

jnordwick··on Firefox's low-latency WebAssembly compiler
Real question.

Most of the wasm instructions are very low level (eg, the test and branch are different, similar to assembly).

But then when you get to the control flow constructs they went in the this weird direction. There is no goto, some of the instructions (br) are context dependent, and if the wasm JIT tries to inline anything it has to muck with the br operand (not even the worst of it). The looping constructs are just odd too.

When I last checked you couldn't even do basic blocks with a control flow graph like most other compilers (so that also means no jump threading optimizations, and without goto you would have a hard time optimizing at the wasm generation level).

It just seems like this part of the instruction set was developed by somebody who wanted some weird higher-level asm, but others wanted an asm close to the instruction set. Just seems like a really bad decitions.

Is there a reason for this or has this been cleaned up at all?

jnordwick··on How to write documentation for a Rust crate
Yes, almost all my complaints about rust doc is in formatting, sectioning, and grouping. The content is good.

But those display issues make it unusable unless you know what you are already looking for.

jnordwick··on How to write documentation for a Rust crate
It is way too difficult to find a function you don't know exists, and I'm not going memorize the api.

It doesn't even come remotely close to the CPU docs. It isn't sectioned just a dumb list of methods where similar methods can be pages away from each other (eg all the methods to look up or index into a data structure should be grouped).

Maybe I just have a higher standard than most rust users.

jnordwick··on How to write documentation for a Rust crate
> This means that changing it means changing the compiler,

Wow. What a terrible decision. I'm quite surprised that people actually agreed to that.

I originally said it was shit but I toned it down a little. The information isn't bad, just too difficult to find if you don't already know exactly what you are looking for. You work on rust core so I don't think you are the best person to judge.

The good part is this can mostly be solved by better formatting and adding a top one-one summary section that groups similar functions and traits (creation, mutation, lookup, etc). With just that you could even convince me to ignore the terrible formatting in the lower section.

The general take away is that the docs are too difficult to scan and if you don't already know something exists and its name, the docs aren't very useful.

jnordwick··on How to write documentation for a Rust crate
I know, but not even close to the same thing. Those one line summaries that make scanning easy are very useful.
jnordwick··on Jane Street and the OCaml Compiler (2018) [video]
Ive worked a couple java based hft platforms at major firms. You write code that never gcs after startup. That means not using certain language constructs and rewriting almost every library along with using off heap memory.

I have personally worked on systems that would run an entire week between deployments and never gced.

jnordwick··on How to write documentation for a Rust crate
Why does the rust documentation still suck? It basically only works if you already know what you ate looking for, and still is difficult to read.

Compare the string pages.

Rust: https://doc.rust-lang.org/std/string/struct.String.html

Java: https://docs.oracle.com/en/java/javase/11/docs/api/java.base...

C++: https://en.cppreference.com/w/cpp/string/basic_string

Yes, the cop page probably has some bespoke work on the page, but it still demonstrates ideas that are partially auto generated (and most of that page is auto generated).

* the method and functions aren't sectioned in any useful way. Just a list of functions (solved by adding section and subsection to the doc comments). Methods that create, search, mutate, test, transform, etc should be sectioned off.

* no use of alignment, just blobs of text

* poor use of color. Both java and cpp use color to break up sections and alignment to distinguish different parts of the signature. Rust is just a page of text that is very difficult to scan.

* no summary. Cpp summarizes the group of method before clicking b through to individual listings, while java summarizes each method at the top. This is useful for searching in a page. Rust is once again just a block of text.

* rust also has a unique issue in how to organize traits and other rust specific constructs. It really should have figured this out by now.

I can keep going on the uselessness of the rust docs, and I say this as someone who was involved in the language in the past, and give it a new shot about once a year. But the docs have always sucked compared to other languages.

jnordwick··on What If Andrew Yang Was Right?
> Why are they getting tax breaks for building warehouses and data centers for themselves?

Are you fucking kidding me? They get capital construction break like every other company gets. If you don't see that as the literal definition of investment you don't deserve to have an opinion on this.

jnordwick··on What If Andrew Yang Was Right?
that's the same thing. we set up tax structures to encourage long term investment precisely because we hope companies will try to invest to avoid taxes.
jnordwick··on What If Andrew Yang Was Right?
It worked exactly as it is supposed to: it incentivized long tern investment where amazon lost a lot of money because they invested so much.

if you want to tax them on those losses that they rolled forward, you would basically have far less investment.

make up your mind - do you want them to invest and take losses or not invest and pay taxes - you can't do both.

jnordwick··on Apple fined $1.2B by French antitrust authorities
> It would allow you to be jailed when you make a decision that gets people killed or causes some other significant harm.

This is incredibly nebulous. All decisions involve tradeoffs. One or two people die because of a decision to go with one car brake over the more expensive? If "increasing greenhouse gases" significant harm? Good luck ever trying to make anything - there will be a non stop line of people trying to sue the executives of any company they don't like.

jnordwick··on What If Andrew Yang Was Right?
> which, like it or not, is the engine that drives our economy

no it doesnt. the US consumes a lot, but you are mistaking an accounting identity with something causal. if people decided to consume less, that means they are putting off current consumption for future consumption and saving those resources for investment to allow that to happen. don't worry - they'll consume at some point when they feel it is in their best interest (and if they don't, they basically just gave us a bunch of free labor).

all economies everywhere are driven by production. human desire is pretty much endless - you rarely have to incentive consumption. when people have the resource (ie, they produced), they will naturally consume.

the only way that long term standards of living improve is by increasing productivity (this is something virtually every economist agrees to). that is done by increasing capital stocks, machinery, technology. those are all production oriented activity.

jnordwick··on What If Andrew Yang Was Right?
> Amazon famous literally pays $0 in taxes), but not relative to how much they make.

because they took every penny and reinvested it and lost money for a long time because of such heavy investment.

just a few months ago everybody was complaining about companies not investing enough and returning money to shareholders - there was even legislation floated to force more investment.

companies cant win with people like you - invest and get assulted for tax reasons, don't invest and get yelled at for that.

you have this Utopian view of how you want everything to work, and don't even realize it isn't even possible most of the time.

jnordwick··on Apple fined $1.2B by French antitrust authorities
It would go way waaay higher. Especially for large corporations where there is probably always somebody at some level trying something shady. I could easily see it go 10x for the largest corporations. Trying to make C-level officers criminally responsible for others' behavior at the corporation would basically push the pay gap much wider.

Even for small companies, executive pay would at least double. And that isn't just going to come out of profits. The lower wage earners will take a hit too, especially if they have to be watched all the time.

And none of that "if you don't trust them then don't hire them" bs. Do you realize how difficult that would make finding a job? Your background would be gone through with a fine tooth comb.

I don't think supporters of this are thinking it through nearly enough. This would be the mother of all unintended consequences.

jnordwick··on Apple fined $1.2B by French antitrust authorities
C-level pay would go through the roof if they also had to deal with potential jail time.
jnordwick··on C++ 20 Is Done
Flagged for being obscene.
jnordwick··on How Big Technical Changes Happen at Slack
How do commercials keep getting posted here?
jnordwick··on Sealed Rust Update
I want much more runtime performance out of rust more than anything else. Faster builds are second on the list. This probably doesn't even make top 10. Who is this aimed at? And I would hate for those top goals to be subverted for this.
jnordwick··on Deutschland AG rethinks workers’ role in management
I've never heard of anybody using exports as development metric. It's usually just a sign of developing country with cheap labor.

> wage stagnation that set in in the 2000s.

And imagine that. Seems to fit the low wage story pretty well.

← PreviousPage 5 of 34Next →