Lidl's Cloud Gambit: Europe's Shift to Sovereign Computing
horovits.medium.com
horovits.medium.com
In beautiful EU bureaucratic style It's a framework for how to talk about how a European Cloud could look like.
It's not about technical standards. It's about how we can talk about how we can think of maybe eventually deciding on how we can come up with standards that might one day lead to talk about implementations.
It represents to me everything that is wrong with the EU today. A bureaucratic monster that can't decide how to talk about things or come to any form of alignment.
Scaleway published an entire blog post on why they quit: https://www.scaleway.com/en/blog/full-steam-ahead-towards-a-...
I think that the EU can very well find a consensus when it wants to, going so far to push for legislature that will be clearly thrown out by the ECJ or HUDOC (see Chat Control for great example).
It's just that we also have a lot of "token projects" which serve for virtual signaling for topics where there is a lack of domestic competence. Gaia-X is one of these things, the idea of a "european cloud" as laughable to begin with, due to dependence on foreign technologies to facilitate it.
What's to stop an American cloud hyperscaler from creating a "properly patriated" subsidiary that it simply licenses the tech to? Wouldn't that side step the "sovereign" protectionism?
An American company would run circles around this mess.
https://hu.wikipedia.org/wiki/Felh%C5%91alap%C3%BA_sz%C3%A1m...
At this point Azure in China and AWS in China is a reskin around Tencent Cloud.
I'm sure early days will be painful but there is no reason for this not to work.
I think this was the press release: https://www.telekom.com/de/medien/medieninformationen/detail...
This was a Microsoft 365 cloud hosted and operated by Deutsche Telekom in Germany. It was more expensive than the global version and had less features. It often took some years till new features were introduced.
They stopped this offering some years ago, I think they did not get as many customers as they expected, most of the German customers used the global version.
https://www.open-telekom-cloud.com/de/blog/vorteile/die-sich...
Do you have any source that they switched to AWS?
Nothing. Amazon already does that in China, their subsidiary licenses the tech and support services from the US company.
https://the-handmaids-tale.fandom.com/wiki/Republic_of_Gilea...
Then I look at something like this Gaia-X "milestones" list and think "Meh, this is probably not the job for me..."
That is to say: it's not all just paperwork and paychecks, it can be greatly rewarding work.
[1] Strangely enough I was just talking about another aspect of air quality in another HN thread. Never noticed this was a theme in my life before.
It's not so bad. Looooong lunches.
The thing that makes it so chill is that we work on very long time scales, based on the length of whatever NIH (or similar) grant we're on. If you're used to building things in the private sector, the comparison I make is that what took us 3 months at my previous YC startup would take us 3 years at the non-profit where I work now. A lot of that is because there are many moving pieces to coordinate, and because you have to be careful when dealing with sensitive data and research ethics. Blah blah blah, at least part of it is also because the breakneck pace of VC-funded software hasn't got its fingers into this pie, at least not yet.
Downside: pay cut. I make $18k less than I did 4 years ago, despite having gotten promoted in this new spot. Also, it can be frustrating trying to actually produce software at a company with no culture for it. You find out that software delivery practices are something people have to learn, and at places that aren't software-oriented, they don't know about them.
They repurposed the word milestone to mean agenda. It's just a list of events they're organizing. Because they have no actual milestones or goals.
"25 years ago each major US company had a German and/or French equivalent. Today equivalents of US tech giants are in China and Europe is on its way to become an open-air museum. What happened?" https://x.com/MichaelAArouet/status/1827588190342979934
Some of the top replies:
"Bureaucracy, Regulation, Aversion to Innovation, Green myth of degrowth etc happened"
[...] Europe’s challenges are significant, but not insurmountable. To regain its edge, Europe will need to foster a more dynamic business environment, streamline regulations, and encourage risk-taking in its startup culture. Without these changes, Europe may continue to fall behind, watching as the U.S. and China shape the future of technology.
The EU has a lot of talent, but it lacks good leadership and good priorities.
Boeing makes the comparison easy for Airbus.
[0]- https://spacenews.com/airbus-takes-a-charge-of-nearly-1-bill...
> Boeing makes the comparison easy for Airbus.
What else to compare against this claim ?
Airbus is by no means perfect of course, but it is still miles ahead of the best the Americans or the Chinese have to offer in a very complex large scale industry. It is not just aerospace, even auto is still über competitive, European manufactures are on par or perhaps better than anything Ford and GM have to offer. I am sure Europeans can come with good examples for every bad one.
The point it is easy to paint a narrative however reality is lot more complex and doesn't match with sweeping generalization .
Plus a bunch of angry USians really irritated by the anti-trust stuff the EU has been doing (DMA etc).
It's not perfect, but it's competitive and successful. Lots of countries contributed to its success, leaving (most) political issues aside.
First of all this is an empty statement that reeks of rhetoric. You don’t have the full grasp of the picture to quantify if a company is miles ahead of the competition. Nobody does. Ahead in terms of what? Even if we want to compare companies in more specific aspects (for the sake of comparison) — be it revenue, vision, innovation, supply chain, or efficiency — Airbus is not ahead of the pack (which includes the likes of Lockheed Martin, GE Aerospace, Raytheon, Northrop Grumman, Rolls Royce).
Airbus is half of the commercial airline market duopoly with Boeing. Customers really have no other choices. Airbus benefits when Boeing drops the ball. Simple as that.
Passenger safety for starters, passing safety audits, units shipped or sold, not having a 60 year old airframe to iterate its product with [1].
> Airbus benefits when Boeing drops the ball.
It also has to build airplanes of quality, price and on time that can fly economically for its customers. Duopoly doesn’t mean it has to do nothing.
[1] MCAS was needed after all because Boeing decided to stick a big new engine on a low sitting 1960s 737 airframe, instead of developing a new airframe.
> Airbus is by no means perfect of course, but it is still miles ahead of the best the Americans or the Chinese have to offer in a very complex large scale industry
So now you don’t claim Airbus is “miles ahead” of the best that US and China can offer, moving the goal post from that very generic statement to just comparing A32X with B737MAX?
So why not comparing 787 vs. A350, or C130 vs. A400, or Sikorsky vs Airbus helicopters? They’re comparable products that Airbus offer vs the competition. Did you know that 787 has got 1,900 sales and 1.300 deliveries, whereas A350 has 1,300 sales and 650 deliveries in the same time frame?
> It also has to build airplanes of quality, price and on time that can fly economically for its customers.
Did I say it doesn’t? Still, customers don’t have a choice. Boeing has been doing badly, yet the backlog of 737MAX is still more than 4.700 units and growing. No meaningful cancellation despite its problems.
As for Gaia-X itself, governments are always on the hunt for programs to justify their spending of tax and debt money. Favorable outcome is the spending itself as a mean to subsidize this and that group.
Beliefs like these are common in Europe and I absolutely despise them. Inefficiencies in IT exist for boring reasons like requirements that are way too complex or that keep changing, internal politics, and inexperience. If you add more regulations that don't move the needle you just get more politics, more middle men that seek to profit from the regulatory capture (advisors, consultants, resellers), and you distract industry from focusing on those things that matter most.
Complexity is the enemy of progress. IT systems fail when they attempt to codify contradictory bureaucratic processes that make no sense. The solution is to simplify. Businesses that refuse to simplify get eaten by hungry startups, and deservedly so. What do you think will happen to a continent that refuses to simplify?
You are confounding the environment we are part of and the so called “free market”. Also interests of titans of the industry, the “free market”, and general interest of the population. Mixups like these are common in developing countries.
well, newer hardware is more efficient than older hardware, but the cost and e-waste resulting from replacing working but older hardware with new stuff is also non-zero.
desktop usage sure, it makes sense to keep it a good long time. in datacenter, for many situations the cost is not worthwhile because DDR5 is substantially more expensive for a given tier of memory, pcie5 is way more expensive to implement, etc. the newer platforms are really also higher-cost ones, due to the complete collapse of moore's law and hitting the limits of physics in link rates etc. On the other hand power does matter and datacenters are highly power-constrained etc.
it's completely application-specific, maybe if you do something that benefits from AVX-512 it's super worth it to upgrade, but for a lot of people it isn't, so it isn't something you can make a blanket regulation on when is the Right Time to upgrade.
MLID has good guests on sometimes and this is an interesting one. Just before this he's talking about the power issues ("they just can't get power into the datacenters quickly enough to keep up with needs"), and he balances this concern against the massive price factor confounding the newer DDR5 stuff.
https://youtu.be/evhkvGBljWI?t=588
This engineer is a good reality check on a number of sacred cows with the AMD fanbase too - for example he is excruciatingly negative on AMD's Platform Vendor Lock. He was asked if the AI market dumped if they could scoop up any cheap gear and the answer is no - they don't use GPUs currently, and they wouldn't even be able to benefit from (eg) epyc cpus being dumped because of the platform lock. They are basically e-waste (by design) once they hit the market unless the provenance is known, and even then it destroys the market efficiency (by design) since now you have separate market for Dell Epyc, Lenovo Epyc, HPE Epyc, etc. Once the value drops, surplus places won't even bother parting them out and basically the channel for that stuff dries up and they become actual e-waste.
And remember, this affects Ryzen processors now too, and platform lock is becoming much more common now as AMD makes the deals with OEM providers to get them into work desktops etc. In 5-10 years there probably won't be too much of a secondhand market left, largely because of AMD... and there's really not much that can be done since this is all hardware-locked/physically fused, short of just pushing a firmware which disables the whole thing.
https://youtu.be/evhkvGBljWI?t=5667
He also is not mincing any words about the Sinkclose/Ryzenfall exploits where an attacker can escalate from a VM guest to jailbreak/control of the PSP and BIOS persistence. Obviously that's a huge, huge issue for datacenter operators and it's bullshit that AMD just basically decided not to patch it for older chips. The amount of handwaving and corporate defense the AMD fan club runs is silly, of course those are major issues and need to be patched ASAP.
I remember the "root password lets you do root things, where's the exploit" and other insane cope/handwaving from HUB and GN and other tech media and social media. Shockingly, the people who actually own the servers aren't as keen on a VM guest being allowed to `sudo jailbreak psp`. And AMD just wanted to leave that unpatched on a huge number of chips, even though they had a working fix for that uarch they were already deploying!
It's unfortunately the same level of security focus that AMD has given to other exploits like the cache ways vulnerability or the PREFETCH+cache eviction vulnerability ("worse than meltdown", discovered by one of the researchers who discovered meltdown), which AMD simply left unpatched and insecure, and (very) quietly told people to enable KPTI if they cared. "Insecure by default" corporate mindset.
https://www.youtube.com/watch?v=evhkvGBljWI&t=3053s
https://www.youtube.com/watch?v=7HxkLlmh4EY
https://old.reddit.com/r/hardware/comments/849paz/assassinat...
https://old.reddit.com/r/Amd/comments/8goyuq/amd_ships_cts_l...
This is IMO the root cause of why most public services are going down the drain in most Western Europe: people are there to work for themselves, not for their country. And the higher people are, the worst it is. Keep the status quo, embezzle if you can and shut your eyes to not see we're in economic and cultural wars against the rest of the world.
And, bless they even have an architecture document which does nothing to bring this blurry vision into focus.
https://gaia-x.eu/wp-content/uploads/2022/06/Gaia-x-Architec...
I'd consider myself reasonably accustomed to and able to deal with bureaucracy and formal language, and still find every interaction with official EU sites massively off-putting. Now imagine someone who isn't a native speaker in any of the EU's languages, mentally impaired, or generally quickly feels overwhelmed by bureaucracy.
I am not defending this state of affairs. Simply pointing out that it's a category error to compare it to national governments. I think it would be good if we had more of an EU state. It seemed to be heading there 25ish years ago. But the nation states do have little appetite to cede authority to the central institutions, so that's probably not on the table. And it's also undeniable that as a coordination mechanism the EU has been spectacularly successful. The fact that people treat it as a national government is proof of that.
I get you like the EU, but "spectacularly sucessful" isn't something many people would use. See covid response, and Ukraine war response. I would describe EU's mechanisms as moderately successful, i.e. somewhat better if states did everything on their own and bilaterally.
> The fact that people treat it as a national government is proof of that.
People with triste knowledge of how EU works do that. I do not think having most people in dark about how EU works is "spectacularly successful".
I think a key difference between how the US works and how Europe works lies in the private sector and the people. When you build a new company in the US, you have a lot of private infrastructure and people to be active in all of the states. In Europe, this does not exist in that form. Here, all the investors are focused on their own country. Sure we have plenty of firms active in many EU countries, but the level of support especially for new firms is orders of magnitude lower than in the US. From languages to social issues to attitude and expectations of common people, the EU is much more compartmentalized and it is significantly harder to have EU scale.
So there are two issues, and the side of the government is only one. The private sector and the investors have to do their job too and provide their own side of the EU wide infrastructure.
We also don't have EU-wide media that needs to support the development of a shared EU identity, and many other things that unite the US population as one people. Much of that has to come from the private sector, from the rich, from investors. But apparently they don't think big enough here in the EU?
Also in the US, somebody in California can give telephone support to somebody in Rhode Island; not so easy to get somebody in Sweden to give telephone support to somebody in Italy though.
As for comparing democracies to China, the democracies of the EU are lucky China isn't next door to them. They're already completely unable to defend themselves against Russia, which is also expansionist and aggressive, but at least doesn't have China's industrial power. If the EU were next to China, they'd be completely defenseless against Chinese expansionism.
But you are ignoring a ton of stuff here, too. The EU comprises territories that are far more different than the territories of the US. The EU has 24 languages spoken, and its poorest member state has a GDP that is a factor of 9 lower than its richest (excluding Luxembourg). While it would be nice to have strong decision-making, how do you make sure that the decisions are also perceived as fair and democratically justified? Imagine a president who doesn't even speak the native language of the vast majority of people in the country. Would that person be seen as legitimately representing the people? How do you even begin to organize public political discussions in a situation where most people can't read the same newspaper/watch the same content? It's far from obvious that any of this is achievable. It's easy to fantasize about a competent, legitimate central government. But how do you construct it from the pieces given?
The political analogue of the EU might be India rather than the US.
Historically, the EU also comprises the territories that for more than half the time period since the inception of the USA provided all the globally dominant economic and military superpowers, expanding the areas they ruled to the peak of colonialism in 1914 [1]. So the squabbling mess of European powers, barely coordinating under a balance of power system at home, was dramatically successful militarily and economically (at an even more dramatic human cost). Contrast the Quing dynasty, that had a central government. At least some historians I've read argue that maintaining the central government consumed so many resources that it was a major reason for the widening gulf in economic and military might between European power and China during the 19th century.
[1] https://www.nationalgeographic.com/magazine/graphics/1914-co...
>Imagine a president who doesn't even speak the native language of the vast majority of people in the country.
This actually reminds me of Spain. Their leaders, while they speak the native language of a majority, it's not a vast majority, with very significant minority-language areas (Catalan, Basque, Galician). And that country sometimes seems to be barely holding itself together, with Catalonia attempting an independence referendum a few years ago that didn't end well.
However, this is a quibble, but as an American I'd argue that if the EU did federalize, they shouldn't have a president at all. The US has a presidential system and it kinda sucks (see: frequent government shutdowns). I think a parliamentary system works better in practice, or perhaps something resembling Germany's system (since Germany also has a federal government).
>So the squabbling mess of European powers, barely coordinating under a balance of power system at home, was dramatically successful militarily and economically (at an even more dramatic human cost)
I really don't think those days are coming back. Europe was so successful back then because it was far more developed technologically. That advantage is long gone now. Plus, Europe's population just isn't what it was. And I'm not a historian, but I thought the reason China didn't develop as fast was all self-inflicted: they intentionally turned inwards and refused outside contact.
Anyway, perhaps you're right and the current state of the EU is the best they can do with their circumstances. Still, I suspect it's not enough: I don't see this as a stable system. They can't even keep one or two of their own member states from torpedoing all progress, and if the US weren't there to defend them from Russia, they appear to be powerless there too.
It seems to be a pretty vast majority, actually. According to a 2019 Pew Research poll, we have the following breakdown of languages spoken at home:
Spanish (81%), Catalan (8%), Valencian (4%), Galician (3%), Basque (1%)
All we did for Gaia-X was the paperwork to get the money. It had zero impact on anything we actually did. Somebody I know who knew what other firms receiving Gaia-X funds did told me the others did even less than us. We certainly did not take it very seriously, apart from it being a great source of free money, and I say that as someone who reported quite a few developer hours for Gaia-X.
I think at most this project is about sending some money to some European firms, with little regard for actual outcomes, kind of as a concealed subsidy. I'm not sure if those who started the project actually wanted that outcome in the first place? A lot of these things are just ways to use the current system to achieve goals that the system does not directly allow. It could just be incompetence, but it could also be the case that somebody knows exactly what they created with Gaia-X and is perfectly okay with the outcome.
Governments want to grow areas of their economies that they think could be beneficial. It's not acceptable to just hand out cash, and that can be trivially abused, so they put a little work around it to make sure the right companies are applying, and they tie it to token artifacts to make it seem more specific. Some companies commit outright fraud, but they're likely the minority and it's just a cost of the program, some put in a ton of effort and do it properly, and that's fine, and the majority just carry on doing the work they were already doing that the government was trying to encourage, and they'll hopefully be just a bit more successful as a result of the extra cash.
It's all a bit silly, but I don't think it's malicious or actually that bad if you assume that the government isn't trying to achieve a specific outcome beyond growing a sector, despite what they say.
I mostly agree with you, except with this statement. It's not fine, the companies that fall into this trap are wasting their time and resources on something that is not a real opportunity. This is the main source of waste imho and unfortunately happens a lot in EU projects.
Other than that, spot on.
The wastefulness of EU is the real reason people want their countries out of it.
There seem to be zero journalists covering EU shenanigans so everyone just get the news from people in the trenches. While our local politicians gets fired for buying chocolate on the wrong account (true story).
Unless such wastefulness is directed towards your own country!
By the way, the only ones who actually voted to quit were those idiot Brexiteers who were not even aware of the EU funds invested in the UK.
I'm sure they enjoy it as much as you do.
Also only country with comparable language complexity to EU is India, so many languages adds enormous amount of paperwork and bureaucracy, US does not simply have to deal with it.
The danger would be if Gaia-X were kicking goals and laying down the law ... and had banned Lidl from setting up their own cloud until all the paperwork was signed off by 13 major committees, 666 undersecretaries of The Cloud and the High Bureaucrat was satisfied that there was a genuine market need for a new AWS competitor.
The EU has a lot of clever and motivated people. If the legal situation wasn't blocking success I expect they would succeed.
just give tax credits based on innovation / investment criteria, to both companies and employees, Europe needs digitalization so badly, yet they find more complex ways to enable it.
On what? The EU doesn't levy any taxes and has no authority to.
The reason they're paying with grants is because that's what they've got.
I've participated in a bunch of EU projects in the academic space. Most of them were catastrophic failures if you just looked at the direct outcomes (few of the lofty goals were actually achieved). But they did provide much needed funds to participating institutions, and I think they were an overall win.
They come with a lot of overheads (some had 10% or more of the funds go to a lawyer who managed the project). But they are a lot more effective than all the "unbureaucratic" Covid aid payments that mainly went to companies with close ties to political parties...
It also funnels money into open source office software and "European data spaces", which is (very loosely) tools to replace US clouds with open source. I would say it's doing exactly what it's supposed to do, making it easier for EU companies to ditch US cloud services.
And didn't exactly that happen here?
4 years ago, the "Schwarz IT KG" company was a day-1 member of Gaia-X. And now they have billions in revenue from AWS-like cloud services.
Given the fact they don't even allow public signups outside of DACH im highly skeptical of their claim they're doing billions of revenue in public cloud. It wouldn't surprise me if there is some interesting bookkeeping going on to boost the numbers.
The current status quo is that people in this thread want to try this but fail to figure out how to even sign up.
Also could you explain what part of being part of Gaia-X contributed to their success?
Teams up with Schwarz group to create StackIT cloud: https://www.handelsblatt.com/technik/it-internet/schwarz-gru...
As for the revenue, I would trust a multi-billion international company that if they say billions in revenue, that'll be halfway accurate. Or else, that would be massive securities fraud. Here's their (independently audited) tax filling showing 1.2 billion € in revenue 2 years ago: https://www.unternehmensregister.de/ureg/result.html;jsessio...
Unfortunately, those links expire after a while. To everyone else: Go to unternehmensregister.de and search for "Schwarz IT KG" (located in Neckarsulm, registered at Amtsgericht Stuttgart, HRA 730995) and open the document "Jahresabschluss zum Geschäftsjahr vom 01.03.2022 bis zum 28.02.2023".
The entitlement and refusal to listen to other people outside their direct org is mindboggling, its like if everyone at that company still thinks they are the hottest shit and they know everything better. Sadly, they are a gatekeeper. Super frustrating partner to work with
+1 on that. Same with the owner of the CSP mentioned in the article above.
It's weird because it was always only German companies that gave truly unreasonable feature requests AND were extremely pushy, despite not spending much compared to other customers.
T-Mobile is much easier to deal with, but they are also walking with their tail between their legs...
Another nice example, resulting in horrible deformed APIs: https://www.ibm.com/topics/what-is-a-digital-twin
What makes it very exciting is that there not too much innovation required to compete
and cloud is an essential service for many companies
and how things played out in recent years has created increasing insensitive to not use Amazone/Google/MS Cloud
but it's marked which isn't really that visible on HN and similar US focused sites
Well, this isn't even remotely accurate on the numbers. For virtually every big tech company the US is about half of the market, Europe about 30%, then ~15% Asian Pacific, and give or take a bit in the rest of the world.
Given that most large tech companies are locked out of China or quickly leaving the only large country with comparable purchasing power to Europe in Asia is Japan.
In Asia, South Korea is already comparable to Japan as a technology market (but also stagnating). The big Asia-Pacific growth opportunities are going to be in India, Malaysia, and Philippines.
Africa is the untapped market right now; Nigeria is expected to exceed 400mil (if not 500-700mil) people by 2050. I don't know as much about their economy, so I'm not sure if this growth has translated into the kind of infrastructure which would create more demand for computing services (i.e: I don't know what smartphone or 5G penetration is like there), but I would bet that if it's not ripe yet, it will be soon
Europe needs to solve its demographic pressures by either accepting more non-EU immigrants or doing something to encourage women to have more children, which would require solving for the opportunity cost (i.e: career stagnating or ending) that women bear 90+% of the time they have children, as well as the FOMO issues (which I don't think government can solve)
However, despite a trend towards much larger homes in the US over the last 50+ years, Americans (at least the ones who can afford nicer homes) have been having far fewer children. So apparently Americans preferred having lots of kids back when they lived in homes with less than ~900 square feet.
As for Europe, Europeans have been living in dense, walkable areas with small homes for millennia, and didn't have any trouble raising lots of kids in that environment all that time.
Imagine if you'd compare only California against all of China, it would look like the US was in seriously bad shape.
But I wouldn't say it's a niche if you look at the size of the EU even if it "hasn't being doing that well" it's still a lot of purchasing power
And especially in recent years there has been an increasing push away from US cloud providers and this somewhat evening out the playing field of "newcomers" compared to Amazone, MS, Google.
Also because HN is quite US/SV focused and differences in business culture especially compared to SV about e.g. businesses doing blog post and similar you don't really see much at all from this marked on HN. But that doesn't mean it's not a big marked.
Here is a press release from 2020: https://www.open-telekom-cloud.com/de/blog/vorteile/die-sich...
Title: Open Telekom Cloud – die sichere Cloud made in Europe
> Im Rahmen der Innovationspartnerschaft liefert Huawei mit dem Cloud-Betriebssystem Huawei OpenStack Distribution eine zentrale Softwarekomponente der Open Telekom Cloud.
English translation: Title: Open Telekom Cloud – the secure cloud made in Europe.
> As part of the innovation partnership, Huawei provides a central software component of the Open Telekom Cloud with its cloud operating system, Huawei OpenStack Distribution.
Software wise they where anyway OpenStack based, which is a trusteable open source project Huawei is a major contributor to but other major contributors include AT&T, Canonical, Hewlett-Packard Enterprise, Intel, Red Hat, IBM. This made moving away from Huawei quite viable.
Hardware wise they also moved away from Huawei, but I'm not sure if this apply to all data-centers of them. But AFIK at least some data centers are Huawei free.
Or at least that is what they told some of their business partners which wouldn't have used them if they still used Huawei hardware in the data center that specific bussiness partner uses AFIK.
Way cheaper than AWS and friends, and they just work.
AWS launched it before EC2 even it's that valuable.
Obviously the fully-managed nature of S3 is very valuable compared to MinIO but the licensing issues seem neither here or there. Or is there some extra part of the AGPLv3 that I'm not aware of?
What do you mean? Can you describe a scenario where this MinIO license matters?
Rolling your own storage comes with at least some operational expenses, if you’re counting your data/egress in gigabytes that’s likely not worth it compared to just using S3 or similar services.
The durability can have value at scale, because ensuring it at scale with MinIO can be a lot work. But I often recommend to clients that they deploy a Hetzner setup as a cache, often with storage all the way up covering 100% of their data depending on access partner, because the AWS egress fees are so insanely extortionate that you never really want to read data from AWS at scale other than as an emergency option.
K8s probably means more s/w than hardware, more bodies (for them) more helpdesk and more documentation, more process. More up-front cost. And, they seem to be making money without it. Maybe thats their position?
You can complain about AWS and GCP support as much as you want, but given their sheer volume it is an accomplishment how rarely you actually need it. The situation when you need it is of course a different thing and I understand that satisfaction there greatly varies. Their support might be hit or miss, but at least their processes are streamlined and tried and tested enough to have you rarely need it.
With Hetzner on the other hand, if you use them seriously you will run into a situation where you need support sooner than later and my experience with that always has been abysmal.
Hetzner did nothing to help rectify the situation. Thanks for nothing Hetzner.
What is the benefit that your customers see in using an European cloud?
Unrelated, I think this also happens the other way around.
Sounds about right ...
Every time they make use of a zero day or a backdoor they run the risk of it being discovered. The harder it is to get a new one, the more they will think twice about using it for mass and low stakes surveillance. A non-US company will be less inclined/forced to cooperate with them, making it harder for them to siphon data out, hence lowering probability.
No one is 100% safe, agreed though. Probabilities and threat models is all we got.
Customers do ask about it, and it's always iffy to have to justify your GDPR compliance when using US cloud companies.
AWS actually joined Gaia-x years ago. I think MS and Google did as well. They want to be compliant. Keeping their European customers happy is important to them. They use European legal entities as well to do business here. Because that is indeed required. No need to switch cloud provider. Just make sure you use them correctly and do all the right things. Which in any case is 100% your responsibility as you will be on the spot if you get that wrong. This job doesn't change if you switch cloud provider.
Deutsche Telekom used Huaweis OpenStack implementation.
I haven't found any information that this changed, so I assume it is still running completely on Huawei. At least they made sure that they still get chips from the US despite sanctions. ;-)
Haven't talked to the guys for two years, so it might have improved in the meantime, but it's Telekom, so I heavily doubt it.
Basically it's just Openstack with some customizations. The rest of our customers run on gcloud. I'm currently eyeing Hetzner as I want to lower our hosting cost.
But the point of Openstack is that it's generally fine and that the feature set it offers is a commodity. AWS definitely overcharges for this stuff. If you do the math, most of their vms will cost you the hardware it runs on within months typically. Amazon runs this hardware for many years. In some cases they host multiple vms on them. Their margin on this stuff is huge. That's why they are so rich. You pay for the convenience and the uptime of course. But undercutting their pricing profitably isn't that hard.
Yeah, sure.
This is the kind of mentality which lead us to "Cloud" being developed by an online book seller and not hosting companies. Then those hosting companies needed 10 years for open source solutions to be available to close some of the gap. Next time a random company disrupts the hosting market, it won't come from hosting companies because "there not too much innovation required to compete" and those companies will wonder what is going on.
Their offering is here: https://www.stackit.de/en/ You have to be a company to make business with them. You can not just sign up, you have to contact them first.
There was a discussion about STACKIT some years ago: https://news.ycombinator.com/item?id=30853778
I do not see anything about Gaia-X on their directly website, only when I search for it there are some older press releases.
Menu -> Compute Engine -> Pricing
This is not necessarily a bad strategy if they want to do something different than OVH/Scaleway/Hetzner.
For instance, I understand you cannot possibly run a decent mail infra on the above hosting vendors IP ranges because they are so popular cheap self service hosting service they have been used a lot by spammers.
So definitely not targetted at the lone developper starting a side gig but more as a vendor for the many larger EU institutions/companys that mostly kept their stuff on prem until now because of patriot act.
Given the CLOUD Act and FISA, no it should not be enough to regain the trust of those European corporations that look for data sovereignty. As long as those exist, all proposed "sovereignty" guarantees by vendors that have their (or their parent company's) HQ in the US are entirely worthless and should be ignored.
Now potentially they could try to trick the operator, but I'm not sure a company could be compelled to do so under US law. While there doesn't appear to be any relevant cases, this would fall under compelled speech (https://en.m.wikipedia.org/wiki/Compelled_speech) and it seems like it would fall on the impermissible side to me.
Note that the money is simply a matter of a contract (e.g., we will hire your company, which is located in China to operate our cloud region. We'll give you X dollars, and you'll give us Y revenue).
For the Germany region, they're using a mixture of technical controls (e.g., the AWS user has to sign off on accesses in a way that's technically not circumventable (think like a phones unlock screen or something protecting the data on the device) and only allowing AWS employees located in the EU to operate it (presumably the goal being that employees physically located in the EU can't be compelled in the same way as those located in the US).
You can read more here https://aws.amazon.com/blogs/aws/in-the-works-aws-european-s...
For comparison, the structure in China is more like what I was describing above: https://www.amazonaws.cn/en/about-aws/china/
I'm not as familiar with it, but it looks like GCP is going with an operating company approach, see eg., https://cloud.google.com/t-systems-sovereign-cloud?hl=en for Germany.
This is not a new idea and is how Azure(or AWS) always operated in China. The Azure Fabric software is licensed to DCs owned and operated by 21vianet a Chinese company. Microsoft has no control over what happens there.
No amount of legal[1] US pressure can make Microsoft give access to those DCs as they don’t have it in the first place
This is why you cannot just provision hardware in China in AWS/Azure, you have to enter into separate contract with the Chinese operator first and comply with any government restrictions that the Chinese state may require
[1] illegal/unauthorized tapping is a different matter and preventing that is not the intent of sovereign clouds .
1. https://aws.amazon.com/blogs/aws/in-the-works-aws-european-s...
If you understand German and want to take a look at OEDIV's remarkable datacenter, der8auer posted a video [1] around two years ago giving a tour through their datacenter. Small but high-quality. This is what Schwarz Gruppe is after, though not as closed as OEDIV.
[1] https://www.youtube.com/watch?v=fMFo74rArBw
* Yes, Oetker, the pizza-maker.
Does anyone remember 'derBauer', the Flash god?
There are a few videos of their previous homepage designs available on YouTube, it's an amusing window into the past. Right below the first one was another video about 2Advanced, which I also hadn't heard of since 20? years.
Presumably pronounced derachter?
But the relevant point here is, that Youtuber has a channel about building and testing Hardware. I don't know whether he has any relation to farming, but considering the content I would think his intention was a wordplay on building, not farming.
If what you need is a DigitalOcean, then yes, you could shift to this. But when you need a DigitalOcean, you're probably in the wrong place if you were using an AWS/GCP/AZ instead, which is also where this article seems to create a failed comparison.
The play itself does make enough sense, there is a significant duplication in effort across companies, even if you're not doing hyperscaler things and using 'enterprise hardware', the people, processes and technology involved are pretty much the same in all places (which means you wonder what value is added by doing it internally at all -- spoiler it's usually legacy reasons, legacy governance and aversion to change).
When there are enough regions and scalability (capacity, higher resolution consumption pricing, shorter cycle times) you could probably use this as a datacenter-in-the-cloud type of deal, which while 15 years too late is definitely still an improvement in so many businesses. We have some larger companies like Hetzner, OVH and Leaseweb which also try to pivot to more of an XaaS but that in itself is just adding to duplication and a fractured ecosystem. Will this actually work out? Only time will tell...
Like I get that this is part of the platform's marketing but I don't see the sovereignty (which is a rather cringy term imo, as it implies that something as big as the EU isn't sovereign) angles to this.
How will the sovereign EU stop the US from exercising the CLOUD act over the data stored in AWS in Europe?
China's tech is already self-contained, to a lesser extent so is Russia's, the US is investing a lot in semiconductors just in case something happens to Taiwan, and India has already banned pretty much every Chinese app. The EU, on the other hand, is completely reliant on the US for core tech infrastructure and trying to address that.
None of it makes sense economically, but essentially every superpower is doing something towards at least making it a possibility to "self-contain" their own tech sector. I don't even think this has anything to do with the current political climate (Russia-Ukraine war included), I think it's more in preparation for when climate change consequences start ramping up.
The economies are too integrated for this scenario. There are big R&D centers of many American corporations in Europe, doing big part of their tech. They won’t move, instead in a very hypothetical “Russian” scenario they will change owner and continue to operate. I’m sure EU regions of AWS or GCP won’t cease to exist, for example. There are some services that are operated from USA, e.g. CloudFront, but that won’t be too hard to replace.
The EU should therefore not rely on the USA staying a reliable partner just for the sake of the market relations, especially with the possibility of a second Trump term. America is self-centered, unreliable, and politically ruined—you can’t work with populists.
All this is to say: Technological sovereignty is the right direction.
Those people were completely clueless. I don't have a better way to phrase it. Nordstream 2 was an insane idea the second it was announced.
In a way, we’ll probably see more cloud fragmentation in the future, especially as other countries develop their IT sectors more and feel like they want more control over their own infrastructure, and whatever tertiary benefits can be extracted from that.
Relationships don’t even have to turn sour, there just has to be enough protectionism and popular appeal to support it. Just like saying “build it here”.
This exact reason was put forward by political thinkers in the 1910s to "prove" that a major war in Europe was completely impossible. It was then put forward again in the 1930s to "prove" the same thing; for real that time.
And it wouldn't be the first time Trump specifically caused grudges. The US just giving up on the Iran deal has certainly already done the same. Not only was the EU also a signatory, but so were the UK, Germany and France individually. And then he moved the embassy to Jerusalem, which was also openly opposed by the three I've already mentioned. So now the EU has to second-guess every treaty the US signed, because apparently that signature means less than it used to mean.
But those two past actions would be nothing but a sentence in history books compared to at least a paragraph about the US cutting funding to Ukraine.
Not going to happen. This will result in partitioning of Ukraine and non-aligned status of it. It is a satisfactory outcome for everyone in the West in such circumstances. That war is already lost by everyone except EU, so whatever is American strategy there, it won’t do much harm.
It is absolutely not a "satisfactory outcome" for anyone in the West if Russia's neocolonial aggression is allowed to succeed.
Except for certain segments of the populist right and far left, who are bizarrely unified on this issue.
So are the US and Chinese economies. Yet, here we are.
> I’m sure EU regions of AWS or GCP won’t cease to exist, for example.
The servers, maybe. I highly doubt AWS will continue to provide access to its platform. Actually there is zero chance they do. The servers are not useful much without the AWS platform.
It doesn't need to be suddenly hostile, it already is for decades. This is about economical competition and not trusting foreign agencies (like NSA, CIA, etc.) which are known to spy on European data and abusing it for the benefit of US-companies. Since Snowden there are several long-running discussions about independence of data, and USA being unreliable in their laws and actions.
After the second world war, Europe was developing space launch capability (multiple governments and companies in Europe joined together to do this). USA said, don't waste your money on that, you can use American launchers. So it didn't continue. Some time later, European commercial sats that competed with American ones couldn't get launch opportunities on American rockets. So then Europe developed its own launcher.
EDITed to remove caveats, after checking that it really is accurate: https://www.inventingeurope.eu/knowledge/the-unfinished-symp...
It's already working.
but DACH mentality? Perhaps hard-working like people in Germany, Austria and Switzerland? Or overengineering or being stubborn and old-fashioned?
Exactly this
Sure if you are one of the best of the best and are willing to take high risk for high reward and in general give up QoL/Work live Balance then especially in SV you have better chances to make a lot of money.
But for most skilled engineers they can get their money worth in the EU, through depending on their priorities and goals in live.
Like to put it in context to have a similar quality of live in US I think I would need to earn around 50% more before tax and that is even through US has much less tax. Through that 50% more also would allow me more flexibility for reducing my QoL at the current time, invest it and long time have more money (or much less if you mess up). So again a question of priorities.
Are there any other countries where tech engineers are among the best paid workers?
Unlike in Armenia or Georgia, the Belarusian IT industry was never a major tax contributor, but its internal spending power was enormous and it transformed the country into a much more livable place.
Since August 9, 2020 this all collapsed. Out of slighly less than 9mln population, 300,000 left the country out of fear of prosecution. Many of them -- from that same IT industry that once defined the bright future of the country.
I'm not sure this model ever applied to A & CH, and might be starting to collapse in D as well.
For example even when a larger company gets acquired (or a merger happens) it could take a decade to consolidate overlapping services.
Historically, too: Poland, Czechia, Hungary and Romania, but German speaking communities have disappeared or are in massive decline.
Disclaimer: this list of German speaking countries might be incomplete.
https://en.wikipedia.org/wiki/Vehicle_registration_plates_of...
In a nutshell: German speakers mentality
If you happen to want your supplier to be slow, extremely reliable, and you don't mind paying for the high profit margin they expect to be able to extract, you'll be a perfect customer of a DACH-mentality company. There are hundreds of niche categories where they dominate the market, including machine tools, forging, factory automation, etc.
But don't write off DACH: there are plenty of companies in DACH that run circles around their competitors by blending typically DACH traits with agility.
At least DACH made the progress of opening up. It would be ideal to combine DACH liberalism for language and french attitude towards tech and innovation.
But.. ..why would you want to speak anything else? Who settles for the mediocre?
For future career possibilities worth investigating, would you care to name a few good examples/companies you are aware of?
In a stable manufacturing business, it’s a superpower. In tech, not so much.
Most EU tech and non-tech companies, with some notable exceptions like Spotify, have this mentality.
And then all of a sudden due to lack of workers and other factors even DACH began to change and basically accepted English as the defacto working language in tech.
Unfortunately it's a small change, too little too late as they say. Without proper transeuropean companies and unified market we will never be able to challenge competitors from Asia let alone the USA.
Though companies like the Schwarz Gruppe or OEDIV tend to understand their value so I don't think parent's comment is valid.
Is that, like, actual information, or just an educated guess based on some industry average.
at least for jobs which often are referred to as engineers (through legally speaking are not as enginer is a protected title having little to do with software development)
maybe for jobs of people which mainly idk. replace hardware in servers all day but don't really administrate the servers at all
German corporations (and politics) are full of bean counters, bureaucrats and underachievers. It's filled with people who love to talk, excessively plan, draw flowcharts and build frameworks - essentially everything except getting shit done.
While I agree, that DACH mentality in big German corporations "has been devastating on serious tech development" at these companies, I don't see how this would affect the whole of Europe. It's an opportunity for other countries/companies to do better.
But now lol and behold unbeknownst to me that there's actually a Lidl cloud and it's not an April fool news.
[1] Hetzner Pricing:
https://news.ycombinator.com/item?id=41179371
[2] How does Aldi keep their prices so low | Aldi Vs Lidl: Supermarket Wars | Channel 5 [video]:
How do I try this? Do they have a free tier?
There's also IONOS.
While there are no free credits the services are priced pay per use to the minute with a much simpler pricing model than the large hyperscalers like AWS. See prices for EC2 here: https://www.stackit.de/en/pricing/cloud-services/iaas/stacki...
You can find the docs here: https://docs.stackit.cloud/stackit/en/knowledge-base-8530170...
"The European cloud" that doesn't allow sign ups from Europe is extremely ironic.
I don't know how they keep getting all this press without actually delivering anything
Ah well, next!
It's the same thing again.
I've never heard of this. Does it mean best cash cow?
My physics teacher would get spitting mad at them for not specifying the unit.
Of course their billing is also 'hours'. Instead of 'hourly'.
(I work there, and my job tomorrow is to get my 2 apprentices new accounts so they can start following the self-paced training in the Exoscale Academy.)
They have three zones, Paris, Amsterdam and Warsaw.
Not sure if they have a free tier, but I still pay about 1€/month for two (really) small instances that I used for testing their service (and kept around for personal stuff).
Hm, maybe I should pitch this to them instead.
Add an option to enable encrypted backups with Shamir's Secret Sharing [1] to some of your closest friends/relatives, so that a few of them together can decide to decrypt your stuff in the event of your untimely demise.
Gimme a shout if you're hiring ;-)
That's ALMOST the case for most EU states so far, but less and less the case, and more and more with private partnership engendering public IT, which is public information, nervous system, witch is the OPPOSITE of sovereign computing and Gaia-X (a failed project anyway) it's the apex of such disgraced model. Oh BTW to be sovereign ALSO DESKTOP must be FLOSS, witch is almost not the case in any public administration. The hw since it's full of fw to the point of being de facto connected black box, network hw included, mush be open or state-made. Witch is not the case in the 99.9% periodic of the cases.
A homeserver is much cheaper, not less monitored, much powerful and much more flexible than living on someone else computer. Since IPv6 it's not an option but a need, it's about time to IMPOSE a public global per host to any ISP, without clauses to avoid legally hosting a server because hey, that's how internet work, it's not a damn mainframe.
And even if you can self-host, fighting against (D)DOS attacks is not something I have seen done at any consumer ISP.
My homeserver is a simple NixOS, so I have to maintain just a config, easy to replicate anytime without manual setups or complex orchestration, it's a small celeron machine with 32Gb ram and two sata classic disks + 2 nvme on a PCIe adapter card, total cost around 300€ few years ago, cooling is juts the home cooling, power it's free on sunny days (domestic p.v.) and otherwise it's still cheaper than the cheapest VPS, plus it can do much more. It's run my HA, Asterisk (for having some VoIP numbers on my deskphone and diverting call to my mobile when I'm not at home, nothing more), a small video-surveillance setup, fetchmail+maildrop+notmuch to serve mails via muchsync, etc etc etc a minimal equivalent VPS setup would costing me around 100+€/month, performing much less. If my server die I have a spare motherboard+cpu a little bit outdated but powerful enough drives are both mirrors from different brands, I have some cold spare anyway shared if needed with my main desktop etc in case of a complete crash I have my config, few kb of text, and I can replicate it anywhere. For personal usage is MUCH more than any classic hosted setup.
The only real issues is for most:
- knowing the software stack they need, witch is rare, because yes maintain a classic Arch or FreeBSD server it's much less comfy than NixOS/Guix System especially if you never heard of them but heard a gazillion of recommendations to use k*s or docker, proxmox and co AT HOME some even trying on raspi sbc...
- some minor legal and hw things, depending on your home and how much the temp mount in summer inside.
Essentially for most it's just about knowing the sw stack witch is a big issue since no university seems to be interesting in really teaching FLOSS nowadays and most professors themselves have very little practical knowledge.
But thinking this is for more then a very very small group of people is not confirm reality.
And I really applaud you for having a setup that generates enough power so for you it turns out to be free. If only I could get a setup like this. For the biggest part of the population this is, unfortunately, not an option.
Also, there is a very big difference between running k*s, docker, whatever at home in a homelab, and hosting your stuff at home. A homelab is to learn, learning means breaking stuff all the time. hosting things dont really combine with that. Most people that actually host things at home have 2 setups, one that hosts the online tools and one homelab where they can break stuff at their will.
At the end of the day, for the majority of people throwing a couple of dollars per month to a company to handle all this crap is the better option :)
How different is from choosing let's say FileHoster inc who works well enough, albeit much less than your infra, and have no capex, than it experience a big issue (cfr. Gandi/FR two times few years ago) all your data are lost, you still have some here and there you get up again on another one, than it became too expensive, you switch from another, ...
In the two scenario:
- on one side you spent in capex more than in opex, so you spent at a specific point in time, in an inflationary economy, instead of being vampirized every month;
- you are tied to your infra reliability and consumer grade assembled iron it's pretty reliable for such usage, as most well know giants are for the same usage/point of view BUT on one side the reliability is in your infra, something you can tune, learn and check, on the other it's about third party decisions who can happen at every point in time without anything you can do.
You've certainly read countless of time about $BigName impromptu ban for instance. You've certainly experienced terms of use unilateral changes, sometimes ok for you sometimes not. Where is the balance?
If you live in a Korean goshiwon -alike you can't host while you might still have docs/photos etc you have no choice, but when you have it and honestly MOST people who need IT at a certain level do have the choice, at least formally, if he/she knows it exists, then where is the convenience? Trusting the market on one side, with little to no capex but much bigger opex and uncertainty or more capex, less opex and uncertainty? Think about schooling: what we do in most part of the world? A big initial capex (long school time) than profiting for life on the acquired knowledge or a jump in the wild than we will learn on the go?
In the mean people are honest, so trusting someone else IF you can verify or the exposure it's low it will generally end up well anyway, otherwise... I trust myself more than someone else shielded under a corp name, eventually in another country (so legal protection issues, geopolitical risks and so on)...
How can you verify what any company state in a GDPR nightmare letter response? I've sent one time ago where a bank asking me to drop an RSA physical OTP for an Android app, that alone violate PSD2 (since the app it's not only a soft-token but also allow to operate on the same device, the reason why banking piracy was a thing again and more then ever), they respond accordingly to the law, but I can only choose to trust or not their response, I can't prove anything and I have nothing tangible to push some public inquiry on them.
Oh, you might feel protected if you upload ONLY encrypted contents, at least feel protected for an unknown amount of time, potentially very long but potentially not enough long.
So no, you can choose to trust someone else, but it's a choice that demand trust, you can't verify, so it's a vulnerability.
Which reminds me that I need to pick up some cheap pliers...
> As I pointed out in my previous blog post about the shifts in AWS, the one-stop-cloud-shop approach has shown cracks. Amazon, Google, Microsoft, Alibaba et al. won’t be able to cover all grounds, neither in tech domains, nor in geo’s.
This doesn’t make much sense to me. What cracks? Aren’t aws regions the solution to geographic control to where your data and infra reside? Also, what tech domain does aws not have a solution for?
I’m not saying the underdog can’t catch up. I’m saying that when aws made this major switch to offer their cloud, it was certainly a first-of-its-kind offering. Lidl offering competitive services to aws doesn’t sound that scary, especially considering that I don’t believe for a second that lidl’s cloud offering comes even close to what aws provides. At that point, if this competitor’s only real value proposition is that they’re “in Europe,” then I’m not sure how compelling of a selling point that is for me to give up everything else I get with my aws offering.
Note: I am using aws as my example solely because it’s what I know best, not affiliated in any way with them as of this writing.
Also, if lidl plays this right, there are a bunch of engineers in Europe, currently working for faangs in places like Dublin and London, highly skilled and and quite desperate to go and live somewhere with a lower cost of living.
As for feature parity is concerned, AWS was pretty small at the beginning. Just like some 12 years ago. Even VPCs didn't exist at one point and you could scan whole cloud from your VM.
I have yet to see a company fail because the engineering team failed to build, whatever monolith, MEAN, micro services.
It is almost always the product tier that fails to articulate and envision the product and place it on a pedestal where people can immediately and clearly see the value proposition.
I hope that doesn't happen here.
Some good products in this realm would be Hetzner and Scalway for example which is certainly good engineering no doubt but great product management apparently.
.
"Case Study 12: Lidl’s €500 Million SAP Debacle" (2020): https://www.henricodolfing.com/2020/05/case-study-lidl-sap-d...
They're already generating revenue so congratulations to them, but would I put my infrastructure in their hands for the long run?...
And their in-store discounts require you to have an Android or Apple device and install their proprietary app on it from Google Play or iTunes, and sign up for an account using your e-mail address and personal cellphone number (landlines and non-geographical numbers are disallowed). It also collects your data and sends it to Google and Facebook.
This is the worst IT of any store I've seen.
Going to be hard to beat the scale effects of US big tech though
How does spark work on this ...
does not seem very fit for large scale.
I would worry that it becomes a mandated / feature poor service whose customers are guaranteed not by competitiveness, but by government requiring it.
It’s about protectionism and tweaking the law to favor local companies.
please revisit the dictionary
Preventing a good part of western big tech from entering the market was protectionism too.
In general, all parties do better in freer markets and all parties do poorly in restricted markets. However, when one party in a trading system implements restrictions and the others don’t that party can gain outsized benefits versus others.
The world spent almost 50 years liberalizing trade systems, mostly with benefits at the national scale. It took 10-15 years for most of leaders to realize that China was successfully subverting the liberal system.
“it works when implemented correctly” is the wrong lessons and will lead us to widespread protectionism and make us all poorer.
The right lesson is that “bad actors need to be dealt with and excluded from the system.”
Once you start dealing with bad actors, you'd have to kick out pretty much anybody. Did western europe played fairly with post-cold-war eastern europe? No. I hear South america have issues with US too.
In reality this works great for established powers to keep status quo. For the rest... It depends on how many rules you're willing to bend. Or how others go above-and-beyond to help you out for some reason that is not part of the system.