462 karma · joined April 9, 2015
The hope isn’t that you find some unique signal to trade on. The hope is you find some signal that does not scale in a meaningful way, so it is less likely professional firms are going to devote resources to trading it.
If you stumble into a fresh, scalable signal it’s unlikely it will continue to be profitable after six months. Once you scale to any real profitable size the market will notice and either change behavior, or trade the same signal at a faster speed.
Cloud had a very similar vibe when it was really running advertising to CIO/CTOs hard. Everything had to be jammed into the cloud, even if it made absolutely no sense for it to be run there.
This seems to come pretty frequently from visionless tech execs. They need to justify their existence to their boss, and thus try to show how innovative and/or cost cutting they can be.
Still, the world I’m used to operating in is typically 5-10 kVA/rack.
Like, what in the good god damn are we using all this energy for?
I can see the utility in creating very simple web-based tools where there's a monstrous wealth of public resources to build a model off of, but even the most recent models provided by Anthro, OpenAI, or MSFT seem prone to not quite perfection. And every time I find an error I'm left wondering what other bugs I'm not catching.
When people talk about leaving their agents to run overnight, what are those agents actually doing? The limited utility I've had using agent-supported software development requires a significant amount of hand holding, maybe because I'm in an industry with limited externally available examples to build am model off of (though all of the specifications are public, I've yet to see an agent build an appropriate implementation).
So it's much more transactional...I ask, it does something (usually within seconds), I correct, it iterates again...
What sort of tasks are people putting these agents to? How are people running 'multiple' of these agents? What am I missing here?
Never did get into Babylon 5, but SeaQuest, for all its campiness, was my jam briefly in my childhood.
There’s a requisite curiosity necessary to cross the discomfort boundary into how the sausage is made.
The issue has less to do with intelligence silliness, and more to do with the fact that the overall geopolitical objectives of the US can not be trusted, and that rift has grown to a point where self-reliance on critical infrastructure may be in Europe’s best interest.
Of the many thing trading platforms are attempting to do, the two most relevant here are the overall latency and more importantly where serialization occurs on the system.
Latency itself is only relevant as it applies to the “uncertainty” period where capital is tied up before the result of the instruction is acknowledged. Firms can only have so much capital risk, and so these moments end up being little dead periods. So long as the latency is reasonably deterministic though it’s mostly inconsequential if a platform takes 25us or 25ms to return an order acknowledgement (this is slightly more relevant in environments where there are potentially multiple venues to trade a product on, but in terms of global financial systems these environments are exceptions and not the norm). Latency is really only important when factored alongside some metric indicating a failure of business logic (failures to execute on aggressive orders or failures to cancel in time are two typical metrics)
The most important to many participants is where serialization occurs on the trading venue (what the initial portion of this blog is about; determining who was “first”). Usually this is to the tune of 1-2ns (in some cases lower). There are diminishing returns however to making this absolute in physical terms. A small handful of venues have attempted to address serialization at the very edge of their systems, but the net result is just a change in how firms that are extremely sensitive to being first apply technical expertise to the problem.
Most “good” venues permit an amount of slop in their systems (usually to the tune of 5-10% of the overall latency) which reduces the benefits of playing the sorts of ridiculous games to be “first”. There ends up being a hard limit to the economic benefit of throwing man hours and infrastructure at the problem.
Excessive surveillance is necessary when you cannot convince people of the merits of your politics or morals on their own and need to use the power of the State to intimidate and control their access.
For the issue on minors, if you have a child (guilty here) you are obligated to actively raise and educate them on the nature of the world. For access to online interactions this doesn’t necessarily only mean active limits (as one might judge appropriate for the child), but also teaching them that people do not always have positive intent, and anonymity leads to lack of consequence, and consequently potentially antisocial behavior.
A person’s exposure to these issues are not limited to interactions online. We are taught to be suspicious of strangers offering candy from the back of panel vans. We are taught to look both ways when entering a roadway.
The people demanding the right to limit what people can say and who they can talk to do so under the guise of protecting children, but these tools are too prone to the potential for abuse. In the market of ideas it’s better (and arguably safer, if not significantly more challenging) to simply outcompete with your own.
5 host cluster; rebooted them all at completion and all of the containers came back up without issue (combination of VMs and LXC)
The first time I did a rebuild and now no longer needed the installation media for games, or the license keys in the manual/game jacket, and I was fully sold.
I don’t fully grasp the hatred, because almost every aspect of it is a vast improvement over what existed 20 years ago. But fortunately there are alternatives.
At the time we operated some industry specific, but national scale, critical systems and were discussing the balance of the crucial business importance of agility and rapid release cycles (in our industry) against system fragility and reliability.
Turns out (and I take no credit for the underlying architecture of this specific system, though I’ve been a strong advocate for this model of operating) if you design systems around humans who can rapidly identify and diagnose what has failed, and what the up stream and down stream impacts are, and you make these failures predictable in their scope and nature, and the recovery method simple, with a solid technical operations group you can limit the mean-time-to-resolution of incidents to <60s without having to invest significant development effort into software that provides automated system recovery.
The issue with both methods (human or technical recovery) is that both are dependent on maintaining an organizational culture that fosters a deep understanding of how the system fails, and what the various predictable upstream and downstream impacts are. The more you permit the culture to decay the more you increase the likelihood that an outage will go from benign and “normal” to absolutely catastrophic and potentially company ending.
In my experience companies who operate under this model eventually sacrifice the flexibility of rapid deployment for an environment where no failure is acceptable, largely because of an lack of appreciation for how much of the system’s design is dependent on an expectation of the fostering of the “appropriate” human element.
(Which leads to further discussion about absolutely critical systems like aviation or nuclear where you absolutely cannot accept catastrophic failure because it results in loss of life)
Extremely long story short, I completely agree. Aviation (more accurately aerospace) disasters, nuclear disasters, medical failures (typically emergency care or surgical), power generation, and the military (especially aircraft carrier flight decks) are all phenomenal areas to look for examples of how systems can be designed to account for where people may fail in the critical path.
This is the bit that really gets me fired up. People (read: system “architects”) were so desperate to “prove their worth” and leave a mark that many of these systems have been over complicated, unleashing a litany of new issues. The original design would still satisfy 99% of use cases and these days, given local compute capacity, you could run an entire market on a single device.
Counterparty risk (including custodianship) is monstrous in crypto. It’s sort of amusingly ridiculous in the same way most tech trends that are trying to break the status quo stumble into the reasons certain rules and regulations exist.
Every kebob is a good kebob when you’re a few drinks in.
I always approached this as a difference between incident management vs problem management, the later being the “what actually happened” phase, with lots of bureaucracy and post-mortems.
I always taught people in my group to manage out during incidents if they understood what was happening. In the vast majority of failure modes you don’t need the most technical people working the keyboard performing, for example, a failover. Most of those processes are well documented and well understood. Very technical/operationally minded people tend to want to solve the problem as quickly as possible, but I always found them far more valuable discussing the issue with stakeholders, and playing a blocking move for the more junior guys/gals on the keyboards. This also helps the juniors get the experience necessary to eventually be able to help develop future staff.
Nasdaq and NYSE have significant volumes because people think they have significant volumes (as circular as that reasoning is). There are entire entities on the fund/investment management side of the industry that are content to do their entire risk adjustment (meaning, trading) in the closing auction, and the dominant closing auction is on the primary listing exchange (just because...well, as I highlighted above).
There was a brief period in the mid aughts when Nasdaq (through the INET and BRUT acquisitions) and BATS were able to compete with the more dominant NYSE due to monstrous discrepancies in system performance, but as all of the markets evolved over the last 15-20 years they are all (at least as far as the vast majority of market participants are concerned) effectively identical in pure technical performance.