AWS services explained in one line each
adayinthelifeof.nl
adayinthelifeof.nl
Route 53 - Holy shit! It's NSD
WAF - Holy shit! It's modsecurity
SES - Holy shit! It's Postfix
Inspector - Holy shit! It's OSSEC
GuardDuty - Holy shit! It's Snort
Data Pipeline - Holy shit! It's Cron and Bash
Athena - Holy shit! It's Prestodb
Glue - Holy shit! It's Hive Metastore and Spark
OpsWorks - Holy shit! It's Chef
VPC - Holy shit! It's a VLAN
Snowball - Holy shit! It's a truck full of hard drives
CloudWatch - Holy shit! It's syslogd
Neptune - Holy shit! It's Neo4j
ElastiCache - Holy shit! It's Redis
DynamoDB - Holy shit! It's MongoDB
S3 Glacier - Holy shit! It's DVD backup
EFS - Holy shit! It's NFS
Elastic Block Store - Holy shit! It's a SAN
Elastic Beanstalk - Holy shit! It's Apache Tomcat
EMR - Holy shit! It's Apache Hadoop
Elastic Cloud Compute - Holy shit! It's a virtual machine
Kinesis - Holy shit! It's Apache Kafka
QuickSight - Holy shit! It's Tableau
Neptune - Holy shit! It's Blazegraph
Route 53 - Holy shit! It's NSD
WAF - Holy shit! It's modsecurity
SES - Holy shit! It's Postfix
Inspector - Holy shit! It's OSSEC
GuardDuty - Holy shit! It's Snort
Data Pipeline - Holy shit! It's Cron and Bash
Athena - Holy shit! It's Prestodb
Glue - Holy shit! It's Hive Metastore and Spark
OpsWorks - Holy shit! It's Chef
VPC - Holy shit! It's a VLAN
Snowball - Holy shit! It's a truck full of hard drives
CloudWatch - Holy shit! It's syslogd
Neptune - Holy shit! It's Neo4j
ElastiCache - Holy shit! It's Redis
DynamoDB - Holy shit! It's MongoDB
S3 Glacier - Holy shit! It's DVD backup
EFS - Holy shit! It's NFS
Elastic Block Store - Holy shit! It's a SAN
Elastic Beanstalk - Holy shit! It's Apache Tomcat
EMR - Holy shit! It's Apache Hadoop
Elastic Cloud Compute - Holy shit! It's a virtual machine
Kinesis - Holy shit! It's Apache Kafka
QuickSight - Holy shit! It's Tableau
A list of actual competing software/products/services would be pretty useful.
Also if you start trying to use DynamoDB like you would Mongo, you’re going to be really upset. The use cases, design considerations and performance considerations of DynamoDB compared to Mongo is night and day.
Hype and usage are very different. It's not like a lot of people really used it during the hype either. There are a few places which actually need something like that and can't use k8s.
That being said, if I needed the flexibility and wanted to stay in the AWS ecosystem, I would use DocumentDB with Mongo compatibility. It uses the same storage engine as Aurora, it’s much more flexible and it doesn’t have any of the well known downsides of Mongo.
I'd go with grafana/prom/loki. It's both for logs and for metrics.
DynamoDB - Holy shit! It's broken MongoDB
It's quite a tour de force how Amazon have taken "separation of concerns", applied it to web services and used it to create complex and difficult to understand or predict pricing to print money. Bravo.
One can mention that there is a LAMP stack behind WordPress without knocking too many unconscious.
Amazon's genius has been to sell the various permutations of the acronym "LAMP" as services.
Although I guess Lightsail would be a lot more affordable.
> It's quite a tour de force how Amazon have taken "separation of concerns", applied it to web services and used it to create complex and difficult to understand or predict pricing to print money.
Is applying separation of concerns to web services really that bad? Look, if you're a small company with a simple product, you can just put your stuff on a few EC2 boxes and pay the monthly bill for that. At that size, your infra costs are going to be dwarfed by your other costs of doing business anyway. If you're a big business, you can literally pay people to keep track of this stuff. You've got the extra money, because now you don't employ data center architects, server engineers, etc. AWS is able to "print money" because it brings a lot of value to many businesses.
They don't work. To elaborate, they fail to provide the user with information with enough clarity to allow him to form accurate mental models of what is being used,what it costs, and what is the implication in cost of applying some change.
Heck, at any given moment it's unexplainably hard to tell exactly which AWS services are you actually using, and being billed for.
The only reliable info on cost it the invoice you receive at the end of the month. Even then, the AWS guys felt the need to develop a full blown machine learning service to help customers predict what their next invoice will be.
Did you actually look at the billing page? Here's a screenshot of my billing page:
It meticulously breaks out per-service costs across whatever dimension you want. "EC2-Other" is rolled up in the chart, but it's broken out by each individual item further down in the list.
That doesn’t show what you are being charged for right now, nor does it help you predict what your total costs will be at the end of the month.
Since they don't, if you are literally so concerned with your current costs for this month, you can set a budget and alerts for whatever you want, so that you don't exceed some cost. Also, they show you your current cost for this month, up until the moment, as well as a daily breakdown of your costs.
I didn’t say anything about this at all. It would certainly be a nice feature, and one people have asked for since the very beginning of AWS.
> Since they don't, if you are literally so concerned with your current costs for this month, you can set a budget and alerts for whatever you want, so that you don't exceed some cost.
Again, this is a strawman. This isn’t a response to anything I said.
If you’re trying to understand how much a new deployment on AWS is costing you, and find ways to optimize that, you don’t care about the total budget of the account, or having some auto shutoff. You need detailed, real-time information from Amazon, which I’ve never seen in the console.
It’s been several years since I’ve had to be digging around in the billing console, so I have no stake in this discussion and things might have changed. Based on other comments being made around here, I really doubt anything significant has changed.
I was just trying to clarify a comment you completely misunderstood.
No one who is working on this cares about real-time by the second data - the daily cost is enough.
> Again, this is a strawman. This isn’t a response to anything I said.
OK, what are you trying to say? If my response above did not answer the question, can you give me a one sentence summary of the point?
>> I was just trying to clarify a comment you completely misunderstood.
Alternatively,
“The billing UI and billing infrastructure is a major weakness of AWS when it comes to helping people understand what their usage will cost.” You can clearly see this is true by the large number of people who complain about it.
- There's almost no demand for it from the majority of their customers.
- Billing code is a big, scary, tangled mess and also happens to be the one area where you really can't afford to introduce bugs and make mistakes, so meaningful change is already exceptionally difficult.
Amazon, in my experience, has issued organizations a one-time credit in cases of surprise bills.
The bigger crux of the issue, is Amazon has democratized CapEx into OpEx, and it's too easy to be ignorant of the details of infrastructure planning, which is rolled into AWS pricing.
Before AWS, IT teams had to plan for and estimate costs prior to procuring hardware. This is no longer needed with ondemand, programmatic infrastructure.
Engineers, who (reductively) focus on solving business problems with code, aren't necessarily thinking about the costs of deploying their solutions, the way Ops and IT would be.
I already got a surprise, because in one place it says t2.micro and t3.micro instances are free for twelve months.
But, if the instance runs Windows, or the zone is this or that, or something else I don't know, the only free tier instance is actually the t2.micro.
And then there's the Unlimited option, and CPU credits, where if your CPU use goes over 25% for some time, it will either: Be throttled so slow as to be useless, or, not being free and you still have to pay for each hour of use.
And that's just the beginning!
What I need is clear and upfront documentation about pricing, before the bill comes.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...
A lot of people use Terraform (my shop does) and I've found a good terraform setup way better than CF templates. You can do tags in Terraform too. You should be forced to use their already bad tooling just to get correct cost estimates.
Blame Hashicorp if they don’t have a similar feature.
Also if you are at any type of scale, your IAAC choice is the least of your problems.
Have you actually done a realistic project plan and estimated how much it would cost to migrate infrastructure from one provider to another once you reach any scale - including regression testing, auditing, training, etc? Do you really just use your cloud provider to host a bunch of VMs? Even then it’s not that easy at scale.
Yes I've dealt with this issue (to be more precise multi-cloud rather than migration, also used other TF providers in tandem like the Kubernetes one), and yes your TF resources are provider-specific, but being able to handle it all with the same tool instead of having to deal with an awful vendor-specific tool (to be charitable, because the reality is having to deal with mutiple ones and having them inter-operate through bash glue) vastly helps in reducing and controlling lock-in, and I would argue it's the baseline step you have to take if you don't want to risk your whole business on a single cloud provider not going to the gutter either because they've grown too big for their own good or because they're being squashed.
CloudFormation means jumping into the pit of lock-in and you can only climb back out by digging your fingers into the dirt wall, Terraform means you have climbing gear to rappel down and if you have to, get back up without as much hardship. Sure you have to put the gear on but your descent is controlled and then you can climb back up at your own leisure.
If I get to make the call and I care one bit about the business beyond next quarter, I would always have a clear way out to heavily reduce risks to a situation that we have seen play out many times before with the Oracles and IBMs of the world.
You can’t imagine the red tape moving from Workday and their homespun EMR (Healthcare) written 20 years ago using Powerbuilder.
I get that there are times to dive all the way in, but there is still a part of me that says "Is this what the Internet is now?"
Terraform in no way provides you an abstraction layer over the different cloud providers.
A unified abstraction layer would certainly work for a subset of cloud offerings. Think the more basic stuff like EC2, VPCs, etc. Platform specific extensions could be handled as optional arguments to the unified objects.
That said, even though you need custom Terraform definitions for each cloud vendor, using one IaC tool still beats having to use a cloud-specific one for each of your deployments. The parent's point stands.
Learning the different IaC tools for each provider = Learning Python, Javascript, and PHP.
Also, Terraform does state management and can be used to package deployments of apps to different platforms. Our IaC pipeline uses Terraform to deploy resources to AWS and several other cloud toolsets, all in one language.
Using Terraform vs CF is easy compared to using AWS vs Azure.
Given that AWS supports terraform, it's really on them to provide a calculator for it as well. We're large enough that we just spent an obscene amount of money on employee time and effort to track our billing (lots of automated tagging), but until we did that it wasn't uncommon for a couple hundred thousand dollars of unnecessary charges to hit our bill every month.
Second, the "in-between" costslike bandwidth from service to service, iops on your disk, ... They mostly don't exist on prem, you either have enough of it and it works or you don't and it doesn't work and you need to go buy a bigger pipe. They're one of the biggest source of surprise cost on AWS.
Eg take a basic data storing application, the cpu and storage space are easy to estimate and are the same as on prem, but suddenly you also need to estimate your iops, and you realize you have no idea about it. Do you have an on-prem database running somewhere ? How many iops does it need ? Unless you've had to figure it out, you have no idea and will never know on prem.
I’m often fielding questions over on r/aws about how to do X on AWS where X is something simple. I then ask are they trying to solve problem X or are they trying to learn AWS. If it is the former, just use LightSail. Outside of serverless applications, I wouldn’t want the complications of full fledge AWS for a small project and I get paid to know the ins and outs of it.
It could be there are such tools. But they are impossible to find, setup or understand for someone who is not already an AWS expert. I recently got a $500 bill from AWS. It was impossible for me to find out which S3 buckets (or even specific files) caused these costs. I looked at this for half an hour and have used AWS before. They just don't tell you how to obtain this information. Or maybe they do, and it is so well hidden that it might as well not exist. This was a horrible user experience. As a customer, I want to know what I'm being charged for. The fact that AWS make it so difficult make me never want to use them again.
EDIT: sorry, I forgot to include the screenshot link - https://imgur.com/a/tVFoocr
https://aws.amazon.com/premiumsupport/knowledge-center/s3-fi...
Why do people think that they can give Amazon a credit card number and expect everything to be easy enough for them and suitable for a large enterprise? AWS offers something for beginners - Lightsail. When that outgrows their needs, they can peer their behind the scenes LightSail VPC to a full fledge VPC.
The biggest mistake I see is that people jump into AWS whole hog without doing the research or if they are large enough, hiring someone who knows what they are doing.
https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2...
Just like AWS is not understandable to someone new, I feel the modern front end ecosystem is royal cluster but yet and still thousands of people understand it after a 12 week boot camp.
It's not my intention to victim-blame here, yet do bear in mind that there are many, many ways to use the services and to expose resources to the bottomless appetite of the internet. Maybe Digital Ocean or Linode or the like would be a better option for starting out.
Yes, I use it extensively.
> Is applying separation of concerns to web services really that bad? Look, if you're a small company with a simple product, you can just put your stuff on a few EC2 boxes and pay the monthly bill for that. At that size, your infra costs are going to be dwarfed by your other costs of doing business anyway. If you're a big business, you can literally pay people to keep track of this stuff. You've got the extra money, because now you don't employ data center architects, server engineers, etc. AWS is able to "print money" because it brings a lot of value to many businesses.
It's not bad per se. The real problem I often see is that many people within the typical org aren't aware of how these costs can accumulate, especially when you have multiple DTAP environments, microservices, and so on.
Another problem is the days of "put your stuff on a few EC2 boxes" seem to have been overtaken by the cloud native, microservices, k8s trends of the last few years. Personally I 100% agree this is the way to go for many (if not most) people -- and for this purpose Digital Ocean is just as good -- but what I see more often is people diving straight into the deep end, because "Is applying separation of concerns to web services really that bad?"
I also think even big businesses underestimate the total costs of these all-in cloud infrastructures. You indeed need to pay people to keep track of this stuff -- multiple people. Your cloud costs + your devops/platform personnel costs can get to be a significant % of your total IT costs. Is it really necessary?
Just because your business has a big market cap or significant revenue from digital doesn't necessarily mean it needs any of that stuff. Chances are you can run it all on the same couple of EC2 boxes you ran it on when you were still a small company.
Maybe a good analogy is to think of AWS as C. It's flexible and powerful, but it's easy to shoot yourself in the foot. That would explain why a lot of these "> magical-tool deploy" services have emerged, promising to abstract away AWS.
I am not sure where you get the notion that any serious user goes with the "put your stuff on a few EC2 boxes" approach. Every single AWS project I worked on had a cost calculation phase when we investigated what combination of services could be the most cost-efficient for a certain problem.
>> cloud-native, microservices, k8s trends
It is kind of funny that you try to put on one end AWS the company that made cloud computing popular (and microservices with it) and the biggest player in the segment and "cloud-native, microservices, k8s" on the other. Trends come and go, AWS is here for the long run.
With something like digitalocean my costs are fixed. I might have misjudged how much I need but can scale up easily or down (with effort).
Will Amazon offer fixed pricing? Does Amazom offer fixed pricing?
For a small/startup, the ability to execute quickly without sinking capital into a product that will likely pivot anyway is a superpower.
When you get bigger, stuff gets dicey. Paying a premium for a "X as a Service" made sense when you had no people, but at some point the cost structure may not make sense or your needs are at odds with how the service delivery works.
When you work for a big company or government agency, things change again -- you don't hire a database guy for a project, you have a project team that builds stuff, and an ops team that exerts the minimum viable effort to keep your stuff running or an outsourced ops team that does the same, but worse. At that point, you start looking at how much it costs to pay AWS to provide and SLA vs hiring more competent (ie. more $$$) ops people to keep it running.
The long and short of it is that AWS is in a position to rake in cash up and down the spectrum -- all they need to do is be competent, which they are very good at.
I look at the whole stack and aside from IAM and EC2, and ECS I think you could get by without a lot of those products.
Yes you can probably launch your own cache fleet using a bunch of ec2 boxes - but you don't get all the low level hooks Elasticache gets. You don't get the pricing discounts either (I remember cross zone replication or something being distinctly free for ec2 but you pay for network. Same with dynamo - access from all zones cost the same pretty much IIRC).
If you are going to make the dive, it is worth spending the time. I would do a small overview bootcamp somewhere so that you know what's available and then start digging in for each new service you take up.
There's a level of complexity and nuance in AWS that casual users or those on the outside do not quite comprehend. And honestly it takes quite of bit of experience and interaction to fully understand some of these things. It's not that you're getting charged "twice" for S3, it's that there are separately line items in the CUR for S3 data transfer, S3 storage (and S3 API, S3 inventory, etc.). It's confusing to a layperson but this level of detail is actually quite powerful. You can tune your application to use just what's needed.
The fact that there are consultants who specialize in figuring out AWS billing was, in retrospect, a warning sign.
I wish there were SaaS products that would effortlessly audit enterprise SaaS bills with support for more than just the top 1% SaaS products.
I have a ticket open with MS support. I suppose it's to their credit that they're listening to me but it's been a few weeks with only the aforementioned hand-waving. I started just sending them screenshots showing that their own dashboard showed no usage during the times I'd gotten charged.
AWS you can poke around and figure it out, Azure is a Kafkaesque nightmare of infinitely confusing UI/UX, a permission system that is like trying to hit a moving target in a dark room, being moved by someone that hates you, doing anything seems needlessly complex, and trying to replicate/test anything locally is horrible (looking at you Azure functions).
I'm trapped in this mess because the nature of our Client is that they'd consider anything besides Azure as a slap in the face, but it's been absolutely the most miserable task I've taken on in years.
As a bonus I'm getting side-eyed by teammates who have a naive faith in The Cloud and can't comprehend why this is taking so long and why I can't just give them a cost estimate.
Take this with as much salt as you want but they're really trying to put that in the past. OCI pricing is super transparent, simple metrics, cheaper than the incumbents on every front, there's no "call us for pricing", there's a transparent 30% discount for committed spending (before you even get to talk to a salesperson) and the dev team has been given free reign to act like a startup and ignore the corporate machine. I'm not a marketing person but I do believe we have a very good offering and it's a shame it's being ignored solely because of bad blood.
[1] https://www.oracle.com/cloud/economics/
[2] https://www.oracle.com/webfolder/workload-estimator/index.ht...
You have a company that charges licenses based on the physical core count even if you run the product in a VM using only a single core, but then they also want to be your cloud supplier.
The only strong point of Oracle is legacy product lock in, you would be crazy to pick them for any new initiatives.
Honestly, I wouldn’t touch something from Oracle with a 10 foot pole, sorry. I’m sure the UX is probably pretty good, but I’m staying away from that company if I can avoid it-too many horror stories, way too “enterprise”.
If you feel like you're way overpaying for AWS, you're much likelier to look out for alternatives. So if they help their biggest accounts save some money, it'll net them way more in the long run.
I frequent a local family-owned Italian restaurant/pizza place and my usual order jalapeños, banana peppers, and roasted garlic so I do a build you own. Totally fine, happy to pay the price. But one day I come in and the woman working the take-out counter sees my order before it goes in and tells me that if I order a "meat lovers", remove two toppings and sub the rest it comes out way cheaper.
So now I'm the vegetarian that always gets a meat lovers and she makes the joke every time I come in.
I don't think it's weird or nefarious that a business doesn't have completely consistent pricing and holding it differently can save you money. When we spec out services on AWS we're using the official calculator when deciding whether it's "worth it" so to us at least it genuinely feels like we're saving money.
I've seen that before, and I've also seen the opposite where people buy the veggie pizza and then add a meat or two on top to get a cheaper supreme pizza.
We had originally gone in with clever ways for owners to describe how different changes would affect the price of an item, but ended up landing on just a giant matrix where they could input the end result price for any combination. The owners couldn't articulate their own pricing well enough to model it well.
It's part PR, part long term customer acquisition
* Price things in an intricate way sufficiently specific to actual costs of providing the service for many use cases
* Price things more generally in a way that more or less probably results in about the same revenue
The problem with option #2 is that it's dependent upon some customers implicitly subsidizing others (e.g. any "unlimited storage" backup solution). It can work a lot of times, but sometimes it doesn't, and at AWS's scale, that could change quickly. It'll upset a lot of customers if they then have to restructure pricing to account for a small subset of customers' use cases being too costly to provide.
You always need the head to subsidize the tail (e.g. free trials). You can’t let the head become unprofitable.
And they can also impose things like bandwidth caps to curb some less-profitable customers' usage. But cloud providers, by design, are charging "per use" and expected to scale "indefinitely"... at least for expected use cases.
Prices should have gone down because they've continued to scale up and go into custom cooling/hardware/backbones/etc.
You're basically saying that you're cutting them slack because their margins have gotten much larger and they haven't decided to raise prices to make them even bigger.
The slack I'm giving is for the complexity in their billing, not in the price itself.
Why should that make prices go down? Prices aren't based on cost. They're based on what the market will bear, i.e., what customers will pay. If you think AWS is too costly, you can go find some other solution. If enough customers start doing that that AWS's financial state suffers, AWS will have to rethink its price and cost structure.
Competition in general tends to drive prices down, yes; but competition was not what the post I was responding to claimed "should" make prices go down. That post was claiming prices should go down simply because AWS is investing more resources in building capital as they scale up. That is not true.
> The fact their prices and profits have remained high for so long indicates lower competition, likely due to high barriers to entry from competitors and high switching costs for consumers.
It could, but it could also indicate that this market simply has not reached equilibrium yet. I don't think it's plausible to claim that AWS has no competition in this area, or that barriers to entry are high; there are a number of huge corporations investing a lot in this market, and plenty of smaller players gaining customers by presenting a simpler interface to an AWS back end (e.g., Heroku). Switching costs might be high, not just for AWS but for any provider in this market, simply because there is so little standardization in how infrastructure is specified and controlled. That would lengthen the time to reach a competitive equilibrium in this market.
It is worth remembering that for new technology markets, it can take a long time for a competitive equilibrium to be established. A good example is the automobile market; in the US, for example, it took many decades for car prices to be driven down to marginal cost and for all of the various market players to search out and capitalize on all the possible competitive efficiencies and strategies. Part of that was also that it took decades for the auto market in the US to become saturated, i.e., for most new car sales to be replacing old cars instead of getting a car to someone who had never owned a car before. I don't think the market for AWS-like services is anywhere close to saturated, which means we should not expect a competitive equilibrium; instead, we should expect exactly what we see, large market players trying to capture as much market share as possible during the growth period, just as major automakers did in the US during the growth period in the mid-20th century. Capturing market share in a growth period is a very different game from squeezing out efficiencies in a market near competitive equilibrium.
My point is that their prices could have gone down without them even feeling any pain because their costs have gone down. They don't deserve slack for keeping prices the same. Keeping prices the same just means their competitors either suck or they sit in an oligopoly together.
Well, there are whole organizations in my company dedicated to installation, procurement, provisioning, ordering(and tracking of said orders), of datacenter resources and hardware.
We are spoiled.
It isn't simple at all! Not only is it complex and fairly daunting, you also need pretty deep knowledge of AWS to fill it out correctly. They eventually introduced an improved calculator that dropped the "simple" branding: https://calculator.aws/
I faced a similar issue at the company I work for (where a PM had activated an instance for months and racked up a $9k bill). I've been spending my coronavirus time hacking together a service that hopefully makes billing on AWS more clear. It does cost optimization and anomaly detection right now, and I plan on adding more features as I start to see more use cases.
Check it out if you'd like, and let me know what you think!
www.usage.ai
Why does AWS use such convoluted language? Is it because they're dominant and it adds friction to moving to another provider?
Providing complex answers to simple questions, almost always is due to a need to hide sth.
Imagine the confusion if S3 were called "Amazon Unlimited FTP Server." That gets every word wrong, except that "Amazon" is merely redundant. It's not unlimited (having to pay for a thing is a limit), it's not using FTP, and it's a service, not a server.
Or if VPC was "Amazon Virtual Colocated Rack". A "colocated rack" means your computer in their datacenter. They actually have this service, it's called Direct Connect, because you can actually
Lambda does require you've got some vague notion of what lambda notation is. But "AWS App Scripts" suggests it's for mobile "apps", but it is not specific to those. And it suggests it's only for scripts, but you can run an entire application on Lambda just fine.
Or even DynamoDB. They recommend "Amazon NoSQL." They're not offering many NoSQL databases, just their proprietary one: DynamoDB. They have a service that offers many relational databases and that is called Relational Database Service.
My point is that definitely are named for these concepts but AWS uses brand names which is quite confusing for people who are new to AWS.
Using these brand names might make it easier to evolve them later, or they really wanted nice sounding names because eventually you'd have to write out long phrases with the AWS prefix, and that's probably harder to market.
Which serverless? ECS Fargate? Lambda? Step Functions?
There are tons of AWS services which are technically "serverless". Even S3 obfuscates away the server.
Some of the "WTF, how did they come up with that name" with AWS comes entirely down to the public name being the internal project name, e.g. Snowball. Various engineers and managers have facepalmed hard when marketing decided to go with the easiest option and use that name rather than come up with something meaningful.
Oracle's EC2 is called "compute instances", S3 "object storage", SES "email delivery", Lambda "functions" etc.
And object storage could have been Casper, load balancer Flamingo, or data transfer Rhino.
But, the saying initially was about IBM. The CTOs in the 60s and 70s who chose IBM instead of one of their competitors in hindsight made a good decision. You can still buy hardware from IBM today that can run COBOL programs written thirty to forty years ago. All of IBMs competitors are long dead.
From a recruitment standpoint, you can easily find someone who knows or wants to learn AWS or Azure - Oracle Cloud - not so much.
Like when you think about if a startup had to sell people on a single one of these services, they would have to actually say "like an unlimited FTP server BUT" and thats the only way they could get into board rooms and they would spend years on just that wrong and skeuomorphic branding just to get off the ground
Whereas Amazon doesn't have to do that, and doesn't have to explain the skeumorphic stuff to anyone, they'll just mention it in some conferences about what you can do now, the end.
Yeah, thats really cool. It doesn't mean there isn't a better way, but I can see how it isn't as arbitrary as I thought.
Kubernetes Engine, Compute, Storage, Memory Store, Cloud SQL, PubSub.... almost all of the main services do what they say on the tin.
The only downside is - ironically - it sometimes makes googling for help a bit tricker. Eg. Are you search for generic cloud storage or the Google product with the same name?
Having gone from AWS in my last role to GCP in my current, I can tell you with 100% certainty that for me, AWS' mnemonic device naming convention is far, FAR more effective in helping me remember which service does what.
S3? Storage. EC2? VM/compute. GCP's equivalent? GCS/GCE. I don't do a whole lot with VMs in my role but it takes me a few good seconds to remember "GCE" whereas EC2 is instantly memorable. Don't get me started on Google's many "Data X" services (Datastore, Dataproc, Data Transfer, Data Catalog, Data Fusion, Dataprep, Data Labeling).
tl,dr; the lizard part of my brain very much prefers AWS' naming style, and I have a hard time remembering GCP's services despite the descriptive naming.
I suppose because of this, Google tend to use the full name in most of their documentation. It does make learning how to use Google Cloud as a beginner a lot easier though.
Because it's run by engineers (not a bad thing).
How long has the idea of “ubiquitous language” when talking about a business domain been around?
Everyone technical at my company from the CTO down can have conversations and talk about CloudFront, API Gateway, Lambda, S3,SQS,SNS,DynamoDB, Aurora, Fargate, CodeBuild, CodePipeline, CloudFormation, IAM, WAF, EC2, ECS,ECR, Cognito, etc.
I’ve seen developers come in and within 3 months it’s second nature.
because Amazon can ignore trademarks? That would have been an entertaining lawsuit between Amazon and Microsoft though.
Consumers expect most companies to act like a coherent unit, (purportedly due to Dunbar's Number), and when you don't have enough oversight or leadership everything begins to look schizophrenic.
"Self-organizing" is organized chaos. If nobody picks winners at the end it never stops being organized chaos.
This is the exact reason I prefer Azure. I can use the search textbox to find something and the name is usually pretty explanatory (but they do have some daft stuff, like 3 different queue offerings with pretty vague documentation on the differences).
the names have a logic to them but I agree it’s intimidating to learn hundreds of things at the same time - so you don’t. you learn the bits and pieces you need.
for example:
EC2 actually comes from Elastic Cloud Compute. You have Compute in the Cloud which also happens to be elastic.
S3 is Simple Storage Service. It’s a Service for Storing things. It’s simple because it’s just a key-blob storage.
Route53 is obvious if you know what runs on port 53.
There are loads of AWS services that don’t follow any logic, they’re just distinct, easy to spell nouns.
It's a tongue in cheek about https://en.wikipedia.org/wiki/Redshift#Expansion_of_space and how
> Snowball
It's like Glacier, but it's smaller and moves more quickly. I think this is one of the dumb-codenames-turned-real that crop up from time to time. Fargate is another notable example.
> Chime
A chime is a notification. That's about all I got.
> Route 53
DNS runs on port 53, Route 66 is a famous highway.
> Sumarian
It's a really niche joke about the novel Snow Crash, that popularized the concept of digital "avatars".
Route 53 - DNS runs on port 53
Chime - when you want to talk about something “let me Chime in”.
Snowball and Sumerian. I got nothing. I didn’t even know what Sumerian was until I looked it up and I thought I had at least heard of most AWS services.
https://aws.amazon.com/products/
look up your service. each service has a 1 line description that captures what it does.
eg Athena = query data in s3 using sql
i don’t really get why people cannot be bothered to learn what a service does and want to pretend they get it from the name. you need to learn the ins and out of the service. the name is the least of your worries.
Names matter.
When your first experience with a product is an opaque name or description, it usually tells you one of two things:
1. The people behind it are not good communicators. If so, this will likely show up in many of the other product details: APIs, documentation, complexity.
2. The people behind it deliberately chose to be opaque. Perhaps they want to make the product seem more important than it is. Perhaps they thought marketing jargon would help sales. Etc.
You seem to want to frame this in terms of developer laziness, but I don't think that's right. Both of these are often reliable signals. The criticism here is well-justified.
The tech world is gibberish. The real world is gibberish too, but the comparatively large number of tech products (because of the ease of releasing software) really exacerbates the problem in tech. Everything is a code name, in-joke, or initialism.
If this were limited to public products then you could write it off as overly ambitious branding, but you can tell it's an issue because internal products suffer the same problem. You know it's bad when you can't even navigate your own company's hierarchy because you don't know what half the team names mean.
Unfortunately, this will probably never change; there's too much precedent. Naming things incomprehensibly is fun, it creates a barrier to entry that makes developers feel smart, and the tech world doesn't value communication enough to change it. In a world where stackoverflow serves as de facto documentation for many major tech products because their own documentation is terrible, I hardly expect those same companies to value coherent naming.
I look at the front end web ecosystem and I think it’s all gibberish also and I’ve been using Javascript off and on since it was first in beta for Netscape. But thousands of people take the time to learn it.
The claim isn't that better names help you ramp up faster. It's that they make it faster to figure out which service you need, or if a service is relevant to you. That said, per the point in my previous post, it's likely that if you can't even clearly describe or name your service, you can't write clear documentation either. I've found that's often the case.
> But thousands of people take the time to learn it.
I don't understand what argument you're making. Is it: Since some subset of people will invest time in learning technology even if it's complicated, therefore it doesn't matter if it's too complicated? If so, you'd have to show that the number of people using it wouldn't increase were it simpler, with clearer names, etc.
That seems unlikely. The whole ethos of many successful companies -- Stripe, Mailchimp come to mind -- is to differentiate themselves by simplifying. It matters to the vast majority of users, developers or otherwise.
Well, two points.
1. From a completely selfish standpoint, if people aren’t willing to learn about AWS, it helps me out a lot. I’m able to make more money by taking the time to learn it.
2. There are three credible cloud providers -AWS, Azure, and GCP - and Oracle. Amazon is twice as large as Azure and 5 times larger than GCP. It doesn’t seem to be hurting them.
Just like the obtuse clusterf%%% that is modern front end development doesn’t seem to be hurting the adoption by the $cool_kids.
This is true, and it may even help with adoption among "cool kids." But the adoption could be much, much higher if it weren't such a clusterf%%%. There is an almost infinite amount of marketshare from a wide spectrum of technologies ranging from Wordpress down to "vanilla jquery" that could be eaten into.
Again, you can't look at some successful thing S lacking quality X and conclude "quality X doesn't matter." You have to compare "S with X" to "S without X". Sometimes a direct experiment is possible, sometimes not.
Not sure what that second sentence means...
Doesn't feel like accurate description for Lightsail, nor a useful one. Maybe something like "simplified deployment and billing for some AWS resources, including VPS, databases, DNS, and load balancers"
(listing "storage" as something Lightsail does is kind of weird; of course, it does instance-attached block storage, you couldn't have a VPS without that. critically, it has no S3-like blob storage product, and I think that's what most people would associate the general word "storage" with, but maybe I'm wrong about that).
Lightsail is basically AWS' version of Heroku and App Engine.
(i.e. a PAAS)
I wish Google would also do this. Many times on GCP's website or at Google Next you try to decipher what the product is that they are talking about, then you realise "ah, it is their version of S3, CloudFormation etc". If they just had said that at the start...
Of course, no company will do this, unfortunately.
Look for "Object Storange" for instance and in the row will be links to all the competing services, so you could pretty easily do this to learn about competitors through the one you know... At least for the big players.
As long as I know, aren't those types into Cloud Storage - nearline and coldline object storage types?
Lightsail is not comparable to Heroku or App Engine. Its comparable to Linode or Digital Ocean. Its not a PaaS; its a simplified VPS provider.
Heroku and App Engine operate at the application layer, with limited direct access to the underlying operating system.
Edit: Original quote (seems I accidentally modernized it a bit):
> Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway – Andrew Tanenbaum, 1981
Maybe we're better off making this as github page where users can send pull requests and add/rewrite to these.
(1) https://netrixllc.com/blog/aws-services-in-simple-terms/ (2) https://expeditedsecurity.com/aws-in-plain-english/
For an external crowd-sourced version, I'd like to see something like this with a column for maturity, and whether it actually works.
The classic AWS services are rock-solid, and perfectly sufficient to build a business on. Many of the newer ones are.... much less so. A green checkmark, yellow question mark, and red land mine icon would go a long way towards letting me know what I should and shouldn't use.
Sure, it should be right in the headline of the service's "about" page. The fact that people need this at all is UX problem.
If people are reading your "about" page and nobody understands what the hell your thing does, maybe your marketing, faux-tech, word salad is pointless.
Is it a UX problem or is it intentional deep branding to further promote vendor lock-in? This is one reason I've been opposed to AWS since the early phases--I don't want to learn all of their stupid branded vendor-specific nomenclature.
If you have ever worked in healthcare, the level of “lock-in” that they have to their EHR/EMR and various other third party services. would make you cry.
I’d like that and also the same thing for languages supported by various services and SDKs, which are sometimes supported and sometimes “supported”.
> Kinesis - Collect massive amount of data so you can do analytics (like ELK?)
Based on some of these that I'm already familiar with, I don't think I would rely on these descriptions for the ones I'm not already familiar with.
Storage Gateway: Virtual appliance to couple on-premises applications to storage in the cloud.
(So it's no iSCSI (block) to S3 (object) but Block(iSCSI to EBS), File (SMB/NFS/S3 to S3), or Tape (iSCSI VTL))
Addition:
VMware Cloud on AWS: Bare-metal, automatically deployed VMware clusters on AWS hardware.
https://expeditedsecurity.com/aws-in-plain-english
Seen on HN (2015): https://news.ycombinator.com/item?id=10202286
I may be wrong but from what I understood, it's more or less a way to manage AWS Lambda functions (cgi-bin scripts), Docker, and a MQTT client connected to AWS on your GNU/Linux devices (raspbian on a raspberry pi for example, or a x86 pc).
However you still need Ansible or similar to manage the device so the actual value is kinda low because if you have a setup to manage the device, it's not much more work to manage docker and a mqtt client yourself. About running AWS Lambda functions on a the device, I think it makes sense for AWS to check the box "IoT edge computing with AWS Lambda" but unless you have a huge codebase in AWS lambda, it seems to be a bad idea.
In one sentence : "vendor locked half baked IoT platform".
Check if your devices are supported here: https://www.zigbee2mqtt.io/information/supported_devices.htm...
I would recommend to go opensource and skip the platforms that try to lock you in. There is a lot of solutions depending on how low-level you want to be. Like HomeAssistant if you want high level, or a Mosquitto MQTT broker and NodeRed and some dongles on a Raspberry pi if you want to be lower level.
I agree vendor lock in must be avoided. I also disagree that Open Source is the best approach in all situations. Mosquito and Raspberry Pi are fine for smaller projects, but if you're going to push 100M or a billion messages, you're better off leveraging a IaaS solution like AWS IoT Core. At least, until you have a dedicated 24/7 opps team that can triage and support the solution.
The trick is to create the right abstractions and architectures to migrate your solutions off of AWS and onto another solution: Azure, your own, etc.
IoT Core: Managed MQTT broker, and state management for devices with intermittent connectivity
FreeRTOS: RTOS operating system for microcontrollers to automatically connect to IOT-Core or greengrass.
IoT 1-Click: Manage 1-click buttons that can be connected to other systems like Lambda
IoT Analytics: Clean up and save messages from topics into a data-store for analytics
IoT Device Defender: Automated detection of misbehaving devices
IoT Device Management: Firmware release management
IoT Events: Visually build automation rules based on device data
IoT Greengrass: Run Lambda functions on remote devices, and manage release of new versions.
IoT SiteWise: Turnkey industrial automation platform
IoT Things Graph: Represent IoT devices in terms of connectivity - for example a door sensors connects to a hub, which has an internet connection.
And one bonus definition:
AWS Sumerian - A 3D game engine integrated with AWS services.
IoT Greengrass - Edge computing that can run Lambda functions and ML models on prem.
IoT Core - Managed MQTT broker, state management and rules engine for devices with intermittent connectivity
I'm not sure I'd call SiteWise a "automation platform" I think it's more of a data collection and visualization platform?
> EC2 Virtual Private Servers > Lightsail Amazon’s hosting provider (vps, dns, storage)
Both of these are VPS? EC2 has no storage?
Absolutely not a criticism of your list, more a comment on how baffling AWS is these days. I stopped using AWS once they hit the point where I couldn't reasonably be expected to remember what all the three letter acronyms were. I still occasionally have to use it for S3, Route53 and IAM - but every time I log into the console I find that they've removed them from my "pinned" services in the menu bar and I have to pin them again. Even this tiny detail is enough to make me not want to have to deal with that * 1000 by taking up more of the services.
Just remember if you see an "S" it means simple. That means if you can't understand it you must be an idiot. S3 is one of the least simple things I've ever used.
Azure: https://web.archive.org/web/20190321175020/https://www.exped...
https://news.ycombinator.com/item?id=13442597
Simple Queuing Service, Simple Storage Service, Elastic Compute Cloud, SimpleDB are names that very much make sense. Greengrass, Lightsail, not so much. EKS, Outpost, Ground Station, and Lake Formation are more toward the usefully descriptive side again, I think.
I was surprised by 2 descriptions:
Opsworks: I thought it was using Chef under the hood. Is it really Ansible?
CloudWatch: it's actually so much more than logging, as it also provides timeseries, alerting and even scheduling. Not sure how to summarize this, though.
That one person only needs to devote maybe 2-5% of their time to it (once the systems are setup), so it's still a net gain.
If your goal is to be the next billion dollar company, it probably doesn't matter that much. But if you're self-funding and need a sustainable business model from the beginning, there's a huge growth phase (mid-sized) where the cost savings from doing this sort of thing can be large enough to be worthwhile.
I'm not sure why so many developers seem afraid of hardware. Back in the early 2000s it definitely felt more commonplace for developers to be able to deal with it. Maybe it's because we grew up having to deal with our own in the form of desktops. Nowadays it seems like lots of people just use a macbook. And that's their only exposure to hardware
The fully allocated cost of two ops people in my neck of the woods is a little over $300K. That can buy a lot from AWS.
I'm not sure why so many developers seem afraid of hardware
I want to develop. Yes I’ve been around long before AWS was a thing, I had to manage servers, load balancers, and even an elevated “server” room with a SAN with whopping 3TBs of storage back in the early 2000s. But in my old age, my time is valuable. I’ve done the on call thing before where our database server went down in the middle of the night when the night operations crew was working.
I won’t work for a company that is not on a cloud provider and doesn’t have a dedicated ops team.
But, if you are a small company. Why are you spending a lot on AWS? Two EC2 instances behind a load balancer and a hosted database shouldn’t cost you over $500 a month and that includes a decent amount of traffic and miscellaneous other charges. Heck if you are that small, just use Lightsail and when you grow, peer your Lightsail VPC that you probably didn’t know existed to a real VPC.
I manage a couple dozen servers and it's a small minority of my job.
I agree that it's possible to do that kind of work on a 5-10% basis, but it requires just the right set of circumstances for sustainability. Like, maybe it is your own startup or family business, or one is not particular good at sysop'ing and wants to learn more.
He said okay go for it.
But then I said, or we can just throw some money at AWS and use the more expensive (on paper excluding babysitting time), AWS SFTP service. I gave him both solutions as a test to see where his head was at.
He chose AWS SFTP. We are a small high margin B2B company and sign long term contracts that are worth yearly in the six figures. We are very much focused on outsourcing the “undifferentiated heavy lifting” so we can focus on development and sales.
[1] Autoscaling would not have been used to scale on demand. We set a min/max of two so if one server fails a health check, another instance will be brought up.
If you do cloud native, there can definitely be pricing and peace of mind benefits. The idea that you can deploy a globally redundant database with automatic backups, with zero installation or server/software maintenance, is pretty amazing. Lambda is awesome and powerful, AWS IAM security model is revolutionary vs. so many different network/LDAP security policies.
I have no affiliation with this other than being a Physics/CS graduate with only one Quantum Computing course under my belt.
I would say a better description is something like "pay per use Kafka".
Fast, Reliable, Stream Processing - Flink
Data Science on Big Data - Spark
Reliable Distributed Log Aggregation - Flume
Low Code Data Flows w/ GUI - NiFi
Zapier for Enterprise Software - Camel
A Single API for Batch and Stream Jobs. Execute on Spark, Flink, managed services etc. - Beam
Never heard of anyone use Apex over Flink. Storm community has branched off into Flink, Spark Streaming, Heron and Cloud Dataflow.
Explaining Spark vs Flink is quite hard tho.
Spark was built as a Batch job solution. Flink was built as a Streaming solution. Since Spark needed to adapt, they leveraged micro-batching to operate as a streaming solution.
They are very similar today. Maybe some remnants of the original design make Spark "slightly less appropriate" for pure streaming usecases, or possibly less able to iterate on future features/optimizations, but so-far that line of reasoning has been only speculation.
- Flink guarantees exactly once stream processing through a barrier checkpointing system.
- Flink has very detailed APIs for handling state and doing stateful stream processing.
Additionally, Flink has one of the most active communities in the Apache Software Foundation: https://blogs.apache.org/foundation/entry/the-apache-softwar...
In situations where you want to create a streaming application with persistent state you might choose Flink over spark streaming.
We use Camel, Spark and Beam to do much the same thing at my work, so despite using them all semi regularly, I've never known why you'd pick one over the other.
[1] https://github.com/gunnarmorling/awesome-opensource-data-eng...
triggered
https://web.archive.org/web/20190508145128/https://www.exped...
But I would just name "Lightsail" "Amazon's digital ocean"
> After reading it over and over again, i still have no idea what it does.
> Some quantum thing. It’s in preview so I have no idea what it is.
> in preview so no idea.
I will try to update and correct services as soon as I have more info (or somebody can provide it to me)
The main feature is the on-demand cloud hosted version of the robotics simulator gazebo ( http://gazebosim.org/ ) that allows you to run your robot's code in a 3d physics environment. You can then take that tested/verified code and deploy it over the air via our fleet management features to your actual robot.
Wrong. That should be "Amazon's Cloud-based Contact Center"
> Pinpoint Create transactional emails based on templates.
Pinpoint can do SMS and voice too.
Yes, pinpoint can do more.. Will update
as well as visual representation https://moca.computingarchitectures.com/en/~hello-world/
Seems many of the actual services may have one.
The complexity of pricing and so much different technology, which we absolutely don't really need, will eventually lead to we start hosting things ourselves.
Oh yeah, and privacy of course :)
You most certainly can mount S3 buckets, and its done frequently in data pipelines throughout the industry[0].
>S3 Glacier Low cost storage system for backups and archives and such
Sure but it would be good to include why there is a tradeoff in price, why is it low cost? For Glacier its intention is to provide storage, but rarely fetch data, as it is very slow.
You could also mount gmail accounts as a filesystem.
I have already argumented against it, and provided evidence for my statement [0].
My argument is logically airtight.
You can simply retrieve the page constantly over a residential connection to drive up the costs to over hundreds of dollars
S3 is an object store, not a file system.
Overpriced, unnecessary or both.
I mean in terms of hardware and software stack.
Outbound bandwidth is the real killer, going at ~$90/TB
Not particularly helpful if you have no idea what a VPC is! Of course it takes 2 seconds to search this for yourself, but still.
Amazon simply describes it as a "virtual network"
Simplifying AWS like this only serves to normalize wasteful spending on the "cloud".
This works when the economy is great....not so much when businesses are looking to trim costs.
I didn't knew cloudfront was an Amazon product. I wonder what cloudflare people think of that.
Because of the name? Cloudfront is older then Cloudflare afaik.
Given the discussion few months back, I thought maybe this was one of those lets-copy-our-costumers-business we-already-know-everthing-he-does things.