I'm not sure if this is a difference between other clouds, at least a few years ago this was a weekly or even daily problem in GCP; my experience is if you request hundreds of VMs rapidly during peak hours, all the clouds struggle.
3,662 karma · joined March 9, 2009
I previously built virtual power plants at Tibber, databases at Neo4j and fleet management Equipmentshare.
jake@davis-hansson.com https://tech.davis-hansson.com http://github.com/jakewins
I'm not sure if this is a difference between other clouds, at least a few years ago this was a weekly or even daily problem in GCP; my experience is if you request hundreds of VMs rapidly during peak hours, all the clouds struggle.
Are you saying you disagree that a new architectural leap is needed and just more compute for training is enough? Or are you saying a new architectural leap is needed and that or those new architectures will only be possible to train with insane amounts of compute?
If the latter I dont understand how you could know that about an innovation that’s not yet been made
Now it seems clear that what’s missing is another architectural leap like transformers, likely many different ones. That could come from almost anywhere? Or what makes this something where big tech is the only potential source of innovation?
Answer is pretty vague though, but sounds like it’s about not trying to “reverse” what the compiler did, but rather try and “analytically” work put what source code would likely have yielded the byte code it’s looking at?
But: I urge you to give constraint solvers a try. This is something very different from some hipster ORM library. The good ones will be a single dependency containing extremely carefully optimised matrix math code that let you solve optimisation problems faster and with much more maintainable code.
Try some toy problem with the HiGHS solver for instance, it has good python bindings and even ships as a wasm package you can run in browsers. “Find the optimal times to run this heat pump given electricity prices”, “find the ideal train departure times that optimise for when people need to travel and minimises transfers”
Postgres running on a single Raspberry Pi is something like 200 TPC-B read/write transactions per second.
Saying Ethereum “is not using very much CPU” is baffling to me. It is the state-of-the-art in this regard, and it uses something like six orders of magnitude more CPU than a normal database running a ledger workload?
They are arguing that stablecoins, specifically, require an off-chain entity that ultimately control them. And if you have an entity actually in control, why go through the trouble of blockchain? Then you can just have the controlling entity run a normal non-blockchain ledger.
I like the argument elsewhere in this thread that the actual reason is that it allows running a bank while pretending it’s not, bypassing regulation meant to protect depositors.
Again, I think pickup trucks are idiotically oversized and dangerous to pedestrians, but arguing against them by repeating things that anyone that uses a pickup knows is nonsense is not helping win over any detractors.
I generally agree with what you are saying, and frequently haul 2x4s without my truck - but the solution to that is a long flatbed trailer, not a Thule hitch attachment.
The same should he true here, right? The added energy needed to carry the weight of the motor would be easily overcome by the gains from regenerative braking?
If each game was bought by a different person, then both most games bought and most people never play the games.
However, it’s very likely a small minority of people take advantage of the insane sales Steam has sometimes - like if 50 people bought a game each and played it, and one person bought 50 separate $1 arcade games and never touched them.
You get the same statistics in both cases of number of games played, but it’s two very different scenarios in terms of how humans use steam.
For me I imagine it’s a third case, that people like me often take advantage of the $1 deals and never end up playing most of those old or arcade titles?
- Fast Frequency Response (FFR), sub-second power adjustment following frequency table
- Frequency Containment Reserve (FCR), ~second power adjustment following frequency table
- Automatic Frequency Restoration Reserve (aFRR), ~second energy production following TSO setpoint signal
- Manual Frequency Restoration Reserve, ~minute energy production following TSO activation signals
My understanding is the primary failure in Spain was that 9 separate synchronous plants that had sold aFRR(?) to the TSO then failed to deliver, so when the TSO algorithms tried to adjust the oscillations, nothing happened. Everything else was kinda "as designed".
No, that is not what the report says. It says, just like you say, that renewables reacted to market prices, causing a generation drop. It then says explicitly that synchronous generation caused oscillation, while PV plants showed a flat non-oscillating pattern.
From your comments I worry there are emotional factors clouding how you're reading the report - this was a systemic failure involving many separate technologies:
- Market signals - negative prices - caused a drop in PV generation (as frequently occurs)
- Synchronous plants caused oscillations as a side effect
- Plants procured to dampen exactly those oscillations did not deliver as requested
- TSO then took measures using interconnections to stabilize via other balance area
- This caused - presumed - overvoltages in distribution grids
- PV inverters then shut off, as mandatory by regulatory requirement in response to over voltage
You're absolutely right that PV played a large role here, but that point is diminished by making it out that PV is both the source of the initial generation drop and the source of the oscillations; it is neither.
The market design caused the generation drop, synchronous generators caused the oscillations, TSO action caused distribution overvoltages and regulatory requirements on PV firmware design in response to overvoltage caused the final blackout.
You can invent a new resource type that spawns raw processes if you like, and then use k8s without pods or nodes, but if you take away the reconciliation system then k8s is just an idle etcd instance
Much of the complexity then comes from the enormous amount of resource types - including all the custom ones. But the basic idea is really pretty small.
I find terraform much more confusing - there’s a spec, and the real world.. and then an opaque blob of something I don’t understand that terraform sticks in S3 or your file system and then.. presumably something similar to a one-shot reconciler that wires that all together each time you plan and apply?
You can do this in Python as well, but it does involve a bit of care; I like the pattern of a module named “internal” that has the bulk of the modules code in it, and a small public api.py or similar that explicitly exposes the public bits, like an informal version of the compiler-enforced pattern for this in Go
Do I want to work for Instacart? Not really. Do I prefer working for Instacart over Tesla? Well, I’d rather go back to hand digging trenches in the rain than work for Tesla, so yeah I prefer Instacart.
Or maybe I’m missing what you mean - can you explain with an example an API boundary you can’t define by interfaces in Go, Java, C# etc? Or by Protocols in Python?
The EU electricity spot price is set each day in a single giant solver run, look up Euphemia for some write ups of how that works.
Most any field where there is a clear goal to optimise and real money on the line will be riddled with solvers
There are negative spot prices in Europe all the time - but they are not usually negative enough to make up for the grid fees and taxes. Or they are in countries like Germany that hasn’t rolled out smart meters, so consumers have no way to access spot prices
I don’t know about SQLAlchemy, but for libraries like pandas I just don’t see how it can be done, and so people are actively replacing them with modern typed alternatives
For software that is going to be maintained, optimising for debuggability, comprehension and minimised ball-hiding is almost always the side to error on.
But: all other things equal, of I get to pick between “10-year-olds primary daily public forum is completely, cryptographically, devoid of any moderating adult presence whatsoever” and - what I had - 10-year olds have privacy but there are adults around that have a chance at picking up that things are going off the rails”