These folks showing up in your trading arena means you've left money on the table and they're going to take that, and maybe some extra.
181 karma · joined September 14, 2010
These folks showing up in your trading arena means you've left money on the table and they're going to take that, and maybe some extra.
https://sniperinmahwah.wordpress.com/2014/09/25/hft-in-my-ba...
https://www.bloomberg.com/news/features/2019-03-08/the-gazil...
http://www.amsterdamtrader.com/2014/09/hft-in-my-backyard.ht...
https://arstechnica.com/information-technology/2016/11/priva...
They employ skills such as kernel driver optimization specialists to squeeze every last ounce of performance out of network card drivers.
What the fuck are they doing messing with this? Yikes.
edit - I answered my own question. These dudes specialize in arbitrage and being faster than everyone else. The crypto markets are insane with this right now. The professional frontrunners have entered the game.
I'll respond here so we don't have split threads :). I'm also quite familiar with at least the older versions of the node internals (I too maintained a popular db binding for a number of years) and I'm very confused by the way you're positioning the operation of the v8 vm in these 2 scenarios. Sure, the c++ is going to require you to do some sanitizing as you force your data into v8, but as we noted that's inevitable no matter how you slice it. After that though, even once in javascript land, you're still crossing these barriers constantly to allocate data and objects and memory, etc. You don't just end at 'parse in javascript', the virtual machine is going nuts calling into this same c++ codebase. Now maybe in some cases the v8 internals offer some advantages the generic c++ api can't access, but this argument isn't convincing of their existence so far.
My memories of the redis client is different than yours so I'd be quite interested to see those conversations / benchmarks. From what I recall those early advantages in the js redis client were similar to the ones we're seeing here, ie: better pipelining of commands.
As a simple thought experiment, in the scenario you're describing we should see a javascript implementation of a JSON parser to beat the pants off the v8 engine implementation, but this doesn't seem to the case.
As I noted in my other comment, in -any- of those implementations, you're still going to be required to traverse from libc somewhere to nodejs (even if it's just to read the network data out of the socket and send it to v8). The performance gains would have to either come from a faster (JIT'd) protocol parser, or from eliminating additional FFI calls (that it isn't obvious why they'd exist).
So, would you mind linking to one or some of those projects? I'd love to see what happened to the implementations.
As the author noted elsewhere in thread, the performance gains are mostly coming from prepared statements and pipelining which seems a bit of an apples to oranges benchmark as most of these native drivers will support similar features, just not enable them transparently like this library does.
Out of curiosity, do you have links to these other projects where they have similar benchmarking attempts/results?
Located in .nl, the oil and gas industry surely has some work that benefits from these kinds of optimizations. ASML probably does enough simulation as well. I'd guess there are probably a handful of financial firms operating on these time scales. Outside of those, the various universities will have groups in this space. I doubt that's an inclusive list, just some areas where I'd expect to find heavily optimized code for specific hardware.
That said, this pytorch work looks to live more in the realm of application than academia. Building and scaling large neural embeddings is pretty close to industry practice these days and this library at least claims to solve some of the challenges in doing so.
When you watch the uncompressed version, there are tons of visual cues to a human that there is an object in the way. This is no where near as subtle as the janky video makes it out. You can say you might have missed this at night but either you don't pay enough attention when you're driving, or you're giving tesla a pass here they don't deserve.
Could you elaborate on what your version of this reasonably suspicious notification compared to your competitors actually is?
The entire technical underpinnings of this solution rely on essentially neural image embeddings, which are very well known to be susceptible to all sorts of clever attacks. Notice how within a day or two of the announcement of this whole system and it's symbols, people were already finding lots of 'hash' collisions.
In the space of people and places that can train and implement/serve such embeddings, these issues are pretty widely known, which makes it very non-obvious how this happened IMO. Someone that understood all of these issues seems to have directly ignored them.
https://www.brendangregg.com/blog/2021-07-05/computing-perfo...
See slide 26 (and the rest ofc :)).
I don't recall e-commerce being hamstrung by it. Do you have any more information on some cases where the patent was actually leveraged?
By chance, are you aware of any research on this topic?
I'm not sure what kinds of applications you're building, but if any of your collections contain on the orders of thousands/10s/100s thousands of items, you really should be considering the impact of how you shape that data for consumption inside your application. You hint at the need to do so, arraylists being good enough until they're not and then hashmaps, etc.
I think there is a pretty reasonable line between expecting our engineers and peers to pick proper containers for the data they're consuming without stepping over into purely 'academic' territory. Shutting down the discussions with 'well I've never had to use this' feels a bit icky and not a great way to advance any of the conversation.
It's not about making a point, it's about not actively being part of the thing I see as a problem.
Does anyone know if OpenAI has retrained/updated gpt-3 yet?