2,704 karma · joined June 5, 2015
I argues with some of the Rust compiler members the other day about wanting to just ditch almost all alighnment restrictions because I of this exact thing. They laughed and basically told me i didnt know what i was talking about. I remember about 15 years ago when i worked at market making firm and we test this and it was a great gain - we started packing almost all our structs after that.
Now, at another MM shop, and we're trying to push the same thing but having to fight these areguments again (the only alignmets I want to keep are for AVX and hardware accessed buffers).
Also when you are trying to do calculations on high-frequency data (tick, iot) it ruins your ability to take meaningful deltas (eg, arrival rates) since you get a lot of 0s and 1s for the time deltas. Its difficult to take weighted averages with weights of 0s.
the issues solving that (if you really need a wider range) are easier to solve that having to force everything down to micros and creating ways around that. (eg, kdb uses multiple date, time, and timestamp types and it doesn't use the unix epoch since it isn't very useful for tick, censor, or any high-frequency data i've seen).
better than a double that some systems still use.
-kdb's secret sauce that people don't seem to understand is its query language that more naturally fits the time series domain. It isn't really a database as more it is an array language with a database component. (eg, try to write an efficient sql query that calculates 5 minute bars on tick data).
I actually like Java too - I've written or worked on a couple trading systems written in core java. just get good devs who understand what it means to write zero gc code, abuse off-heap memory, and understand what hotspot can intrinsify. If you can stay in the simd code for all the heavy lifting loops (filters, aggregates, etc), I don't think java will be an impediment.
I think you have parts going in the correct direction, and you seem to have good experience from looking at the bios. Nothing really un-fixable (or un-addable) in what I saw glancing at your docs. I did bookmark you to see how the db goes. Will prob check out soon.
"As of now, SIMD operations are available for non-keyed aggregation queries, such as select sum(value) from table."
not even sure if they support where clauses on that, sums of functions of a column, or even other things like stddev of the column.
their storage format though looks good and simple (similar to kdb actually), but they really should have an 8-byte char instead of the 16-byte one since that would be far more used.
their partitioning scheme is only on time so less advanced than other system.
single designated timestamp column (so no bitemporal), but do support asof joins which is nice.
they totally screwed up on dates and time. dates only to milli and timestamps only to micros. huge mistake.
long256 which is nice, but strangely no long128s (which wind up being nice when you have base-10 fixed-point numbers normalized to a large number of decimals).
i didn't see any fixed-width string/byte columns. Does have 32-bit symbols (i assume similar to kdb?) that might cover some of those use cases.
some good and some bad in there. never going to compete with kdb or onetick on performance (and nobody competes with kdb on query language/extensibity) , but could find a niche based on price and having simpler more easily adapted to querying and more human interface.
2 only having structured control flow makes it a difficult target for HLL, which is what was is supposed to be.
3. A few years ago I was working on an array language and was trying to use tree based optimization and code generation, but I put it on hold until i understood more - can you recommend and links or papers?
4. Wasm is an IR, both target and source. Why not add a goto and structured primitive. There are clearly some things that cannot be compiled anywhere remotely performant.
And I'm not going to memorize the API either. Sorry.
Most of the wasm instructions are very low level (eg, the test and branch are different, similar to assembly).
But then when you get to the control flow constructs they went in the this weird direction. There is no goto, some of the instructions (br) are context dependent, and if the wasm JIT tries to inline anything it has to muck with the br operand (not even the worst of it). The looping constructs are just odd too.
When I last checked you couldn't even do basic blocks with a control flow graph like most other compilers (so that also means no jump threading optimizations, and without goto you would have a hard time optimizing at the wasm generation level).
It just seems like this part of the instruction set was developed by somebody who wanted some weird higher-level asm, but others wanted an asm close to the instruction set. Just seems like a really bad decitions.
Is there a reason for this or has this been cleaned up at all?
But those display issues make it unusable unless you know what you are already looking for.
It doesn't even come remotely close to the CPU docs. It isn't sectioned just a dumb list of methods where similar methods can be pages away from each other (eg all the methods to look up or index into a data structure should be grouped).
Maybe I just have a higher standard than most rust users.
Wow. What a terrible decision. I'm quite surprised that people actually agreed to that.
I originally said it was shit but I toned it down a little. The information isn't bad, just too difficult to find if you don't already know exactly what you are looking for. You work on rust core so I don't think you are the best person to judge.
The good part is this can mostly be solved by better formatting and adding a top one-one summary section that groups similar functions and traits (creation, mutation, lookup, etc). With just that you could even convince me to ignore the terrible formatting in the lower section.
The general take away is that the docs are too difficult to scan and if you don't already know something exists and its name, the docs aren't very useful.
I have personally worked on systems that would run an entire week between deployments and never gced.
Compare the string pages.
Rust: https://doc.rust-lang.org/std/string/struct.String.html
Java: https://docs.oracle.com/en/java/javase/11/docs/api/java.base...
C++: https://en.cppreference.com/w/cpp/string/basic_string
Yes, the cop page probably has some bespoke work on the page, but it still demonstrates ideas that are partially auto generated (and most of that page is auto generated).
* the method and functions aren't sectioned in any useful way. Just a list of functions (solved by adding section and subsection to the doc comments). Methods that create, search, mutate, test, transform, etc should be sectioned off.
* no use of alignment, just blobs of text
* poor use of color. Both java and cpp use color to break up sections and alignment to distinguish different parts of the signature. Rust is just a page of text that is very difficult to scan.
* no summary. Cpp summarizes the group of method before clicking b through to individual listings, while java summarizes each method at the top. This is useful for searching in a page. Rust is once again just a block of text.
* rust also has a unique issue in how to organize traits and other rust specific constructs. It really should have figured this out by now.
I can keep going on the uselessness of the rust docs, and I say this as someone who was involved in the language in the past, and give it a new shot about once a year. But the docs have always sucked compared to other languages.
Are you fucking kidding me? They get capital construction break like every other company gets. If you don't see that as the literal definition of investment you don't deserve to have an opinion on this.
if you want to tax them on those losses that they rolled forward, you would basically have far less investment.
make up your mind - do you want them to invest and take losses or not invest and pay taxes - you can't do both.
This is incredibly nebulous. All decisions involve tradeoffs. One or two people die because of a decision to go with one car brake over the more expensive? If "increasing greenhouse gases" significant harm? Good luck ever trying to make anything - there will be a non stop line of people trying to sue the executives of any company they don't like.
no it doesnt. the US consumes a lot, but you are mistaking an accounting identity with something causal. if people decided to consume less, that means they are putting off current consumption for future consumption and saving those resources for investment to allow that to happen. don't worry - they'll consume at some point when they feel it is in their best interest (and if they don't, they basically just gave us a bunch of free labor).
all economies everywhere are driven by production. human desire is pretty much endless - you rarely have to incentive consumption. when people have the resource (ie, they produced), they will naturally consume.
the only way that long term standards of living improve is by increasing productivity (this is something virtually every economist agrees to). that is done by increasing capital stocks, machinery, technology. those are all production oriented activity.
because they took every penny and reinvested it and lost money for a long time because of such heavy investment.
just a few months ago everybody was complaining about companies not investing enough and returning money to shareholders - there was even legislation floated to force more investment.
companies cant win with people like you - invest and get assulted for tax reasons, don't invest and get yelled at for that.
you have this Utopian view of how you want everything to work, and don't even realize it isn't even possible most of the time.
Even for small companies, executive pay would at least double. And that isn't just going to come out of profits. The lower wage earners will take a hit too, especially if they have to be watched all the time.
And none of that "if you don't trust them then don't hire them" bs. Do you realize how difficult that would make finding a job? Your background would be gone through with a fine tooth comb.
I don't think supporters of this are thinking it through nearly enough. This would be the mother of all unintended consequences.
> wage stagnation that set in in the 2000s.
And imagine that. Seems to fit the low wage story pretty well.