6,367 karma · joined May 14, 2017
On Azure if you have credits (even hundreds of thousands of USD of credits through programmes like MSFT for startups) you just can't get any GPU quota in any region, even if you beg and plead with support reps. Even if you have cash it's often difficult to get any of the cheaper cards. You can forget about spot instances etc because the regions are essentially all at 100% capacity.
I've heard on other hyperscalers that it's also challenging.
This is super chilling for anything other than GenAI inference/training because if you have some more traditional workload that requires GPUs (traditional ML, other kinds of modelling) you're SOL.
Large companies have often changed the depreciation schedule for server hardware to 6 years rather than 3. I think it remains to be seen if this is justified as it’s not as though the pace of change in hardware has slowed down by 50% arbitrarily.
I think it’s pretty obvious to me that a data centre full of ampere GPUs is worth less than a data centre full of Blackwell GPUs. I suspect this is also why AI companies prefer to talk in terms of gigawatts of compute, as if efficiency per watt does not exist.
Data centre costs are not one-off. Some are, for buying land and building buildings and infrastructure. The actual compute however is a constant investment. I’m not familiar enough with the market to know what proportion of expenses are ongoing but I bet it’s higher than people think.
I use Talos for my homelab and have been considering using it for work. I'm a little worried about the future direction. I'm not sure why they want to add VMs as a native resource rather than going in the direction of KubeVirt as a first class citizen (which already worked well).
This is a solution that hopefully will cease to be relevant as soon as the remaining infrastructure is finished.
Unfortunately I'm needing to run a lot of batch compute jobs (for which the hyperscalers are just insanely expensive - even to have a machine that outclasses a nice laptop becomes silly very rapidly)
I'm considering buying some machines and racking them in a colo but it feels like buying right now is also insane because of current pricing.
Does anyone else have any suggestions for competitive pricing for this kind of thing (e.g. batch jobs)? Was this applied retrospectively to existing customers?
I personally have lots of batch jobs like CFD simulations that could easily run on a fleet of phones with no real reliability issues, and I’d love to reuse old hardware and give it a second life. I’m already considering running old servers from e.g ETB but the cycles per watt on a phone are probably much better.
So much wealth is tied up in pensions and it’s folly to let it all go to supporting the U.S. and eschew local investment altogether.
You make three applications with those fees divided slightly differently each time.
That’s without any legal fees if you need someone to help you prepare your application which will be ~£2k per application.
If you have non-British children or stepchildren (which is generally quite rare) it’s approx those fees per kid as well.
For most other European countries including Ireland the fee is <€500.
Pretty much all forms of permission to stay in the UK other than asylum can only be granted from within the country if you hold an existing long term status. So if you're visiting as a tourist you can't then decide to apply for a spouse visa or even a working holiday or student visa without leaving the country first. If you're already on a student visa or a work visa or similar you can change categories without having to leave.
The graduate visa is essentially an extension to the student visa with slightly different permissions - it makes sense that you can only apply to extend if you're in country and you view it from that lens.
The historic reason behind all this is that there used to be a substantial difference between being granted "leave to enter" and "leave to remain" (out of country vs in country applications). Leave to enter used to be granted by embassies etc and the foreign office, but leave to remain was granted by the home office. Now the home office handles everything in the UK centrally so the distinction is not significant.
It's very true that retirement means losing many of the fun perks associated with a high-flying career, but you have to have those perks in the first place.
Even here on HN, many of us are living fairly ordinary lives, perhaps working on cool technology, but I think even in our fairly exclusive club, not all that many of us have PAs and frequent high-class work travel, even at the apex of our careers.
For those who have all those things and lose them it can be very profound. For a lot of people in that position they spend almost all of their time working in some sense or other, and retirement ends all that. I think that's why people in super successful careers often end up doing advisory work or ending up on boards, so they can keep some of the perks with less of the commitment.
What needs to happen is a sensible evaluation of the candidate's and employer's mutual suitability. We've had systems to do this in various ways for decades/centuries. What AI has enabled is a kind of shotgun approach to hiring that benefits nobody. The solution to this issue is more/better human interaction, not "agent-to-agent" communication.
I recently bought a sample for a NUC-style board that contains an N355 processor, ram, and disk in a little kit. In the couple of weeks between when I put the PO in for the sample and the time it was delivered it had gone up in price from ~£600 to ~£1000. That might be a huge difference in BOM costs for a given product based on that design.
Real-terms prices for embedded computers must now be 2-4x what they were in 2020 for "similar" (i3, i5, i7, though I know there have been substantial improvements since then) positions in the market due to inflation and price gouging.
It's terribly difficult to keep up. This impacts far more than just AI. IoT devices, scientific instruments, kiosk systems etc have all skyrocketed in BOM costs because of this. Almost everything contains RAM, and a ~2x increase in an already expensive component might scupper a whole design.