eightnoteight

> A government agency isn't in a business to make money.

true, but it's not in the business of wasting money either. if they don't take care about economic efficacy, then someday voters are going to judge them for wasting tax payers money

Most of the cost comes from ingestion and running the servers all the time (OpenSearch, Loki, Clickhouse)

But the free s3 data transfer and using on-demand lambda compute for searching makes it really cheap

> Cutting costs by adopting a better architecture was a big thing at one of my previous jobs. People were praised and promoted for cutting thousands of dollars off monthly AWS bills.

I think this is a slippery slope. praising is fine, but aws bills is essentially a non-functional quality attribute of the software. the job is to never even let it become a problem in the first place

what about teams that have built their software in time and with quality, are they essentially losing one of the opportunities to get promoted because they built a better software in the first place?

like who would care if you can bring certain type of knowledge's half life to 200 years. since you can't live anymore for that much time, you don't even need to think about solving that problem. just need to think about how to convert knowledge whose half life is less to the knowledge that has higher half life. (like converting API surfaces to mental models)

while I agree that there is a half-life to certain type of knowledge but I think it would be overstating to reasonably apply to all knowledge.

its very easy to retain a significant portion of the knowledge by building mental models. sure you will forget about the API surface of a technology, but you would surely remember the underlying knowledge models and you would surely apply it in many other contexts.

like remembering the REST API inputs and output data types is also a knowledge whose half life is much shorter. but building mental models of those would stay for a long time. the point never is to remember and include REST API input and output data as part of your skill set, the point is to include their underlying knowledge. and you can't treat the API surface of a library as knowledge, its essentially a volatile memory and supposed to be cleared away

> If it were a battery break through it would take 10 years to turn it into a mass producible product

I think it really did take approximately that much time for LLMs as well. first transformers paper came in 2017, almost 6 years back.

text to code came almost 2-3 years before ChatGPT: https://www.microsoft.com/en-us/research/video/intellicode-c...

so even for software with the same underlying tech i.e transformers, it took almost 5 years to get to a breakthrough that can be scaled.

I really like paulg's observation that "knowledge grows fractally". If you put the scope of chatgpt as all human jobs, it would still seem that its we have only scratched the surface, same in terms of throwing money at it, we are only throwing a very little fraction of the money

> We need to value readable code over DRY, or Design patterns or what ever the next FOTM is. We need to laud people who make systems simpler, who reduces costs, who reshape and extend existing things rather than build new ones and pile more on top because it's "easy" or looks good on the resume.

not just in tech, its always been hard to quantify and reward people based on non-functional attributes of a system's output.

> I am part of the problem too, and I need to stop. I need to stop reaching for that next shiny bit of tech, next framework, next library. Because acting like a schizophrenic ADHD child finding candy hasn't really served anyone.

referencing paulg again, I think this reach for next shiny bit of tech should still happen, but reaching fractally i.e in context of everything that you do, reaching for new tech should be a small part of it but still an essential component to grow

as in the motivation is already there, but there aren't any good solutions to help engineers ensure efficient resources on cloud

its high time that cost is treated as a requirement of software delivery itself. even now cloud cost optimisation is essentially a requirement driven out of top down goals like FOMO, P&L etc,... its not wrong but that is a pretty late metric

I think a serious lack of tight feedback loops and tooling for engineers made cloud cost management into a discipline driven from top down rather than bottom up like how other pillars of architecture are driven like reliability, scalability, security etc,...

I think finops is a pretty great step in the right direction, but it still feels like a lot of it is being driven out of leadership teams attraction for quality rather than building a culture within the whole org every team is attracted to new quality standard i.e running efficient systems in cloud

exactly, a typical REST server spends a lot of compute time in json encoding and decoding. which could be sped up significantly if there is better memory bandwidth

I felt like the real lesson is a missing flight recorder system, if each action and behaviour is properly logged, it would have been pretty clear that the telemetry was lost before chris even began testing the RAT motor

I want to better understand the reasoning behind the "scale to zero" point that many people mention.

only case I could think of is that businesses that want "scale to zero" have very low total expenditure on the database. is this the case for you as well?

because serverless without going to zero still solves a major problem for a lot of companies with some decent scale, where there is some decent traffic and its too hard to implement database autoscaling and most businesses have unequal traffic (daytime and nighttime etc,...)

remember it is aws, "leverage" is built into their DNA, I don't believe that they would go on the route of building a totally new SQL database like google did

I think they are internally using postgres only. aurora uses a custom storage process on host which internally supports the custom storage engine (s3)

given the mentions of "log as a database", I believe that depending on the transaction the storage responds differently. like how mysql mvcc uses undolog to essentially rollback database internally so that transaction sees data consistency. they could be doing something similar i.e regardless of whatever postgres uses, if they can get a reference of transaction and its start time then they can use the custom storage engine to rollback the log and respond in that way

I think we mean different things, it is enabled by default but I'm saying thats not the default IMDS version that metadata endpoint points to for `/latest/`

for example if you run below curl request on ec2 instances launched from aws console before nov 6th then you will get a response

```sh

curl http://169.254.169.254/latest/meta-data/profile

```

but if you run the same command on ec2 instances launched from aws console after nov 6th, then you will get an error that auth token is missing

IMDSv2 was initially introduced after the capital one attack. but aws users will have to explicitly enable it and its not enabled by default.

now they are changing the default modes for instances launched from aws console using quick start process. and sometime in 2024 they will give an option to customers to control the default value for all run instances API calls from an account

actually I think this might be the intention of mirage as well, especially since they put "modularity" in the title

exactly this, debugging is a pretty big part when it comes to operational aspects.

but I feel like similar to kubernetes container injection systems. I suspect in future we would see dynamic injection into the unikernel where you can enable debugging environment instantly.

like the library unikernel exposes a plugin system that you can inject in future, this way the unikernel is lightweight and only gets bloated when you eventually want to debug something.

I think its about efficiency. thats why you may not be seeing "new problems" being solved.

like people used to be able to travel before cars as well, cars didn't solve a new problem and the world didn't have any demand for them too. but once they became available, regardless of whether someone knows it or not, the whole world needed cars. now the car builders control the demand because they can't instantly scale up the supply

AI is similar to that but people can scale it pretty quickly to the whole world, there are a number of efficiency problems being solved in the world using AI. a couple of years ago I had to research and read atleast 1 hour worth of content to find out the information that I needed, google was good but still it required 1 hour. now its hardly a couple of minutes with chatgpt in many cases.

efficiency problems usually don't manifest in demand, people adapt to the status quo

in hindsight, you got the context pretty accurate i.e importance of microsoft in all of this

@sama @gdb joined microsoft

this is a predictable worse latency due to CPU throttling, i.e nothing is suddenly introduced in the system. but the other case is worse i.e a non critical microservice can cause outage on your critical microservice.

imagine some non critical system like blog-service suddenly causing 2-3% new order creation failures

imagine amazon's order history microservice causes outage for ordering creation microservice. regardless of the monitoring and alerting, you never want a non critical system to cause outage on the most critical system you have

Reservations are not limits, they are minimum guaranteed CPU usage constraints. while theoretically, those are minimum guaranteed resources, if there are other busy containers running on the host, you will still see your tail latencies and average latency increasing abnormally

imagine you got 4 core ec2 instance, the latencies you see at 50% cpu utilisation and 90% cpu utilisation are quite different. with reservations similar thing happens i.e even though each container is guaranteed to their reservations, the relative CPU utilisation still increased very high when there are other busy processes on the same host

aurora serverless v2 now supports io optimized mode. which increases the pricing for instances by constant amount but makes the io pricing to 0

https://aws.amazon.com/about-aws/whats-new/2023/05/amazon-au...

most of the benefits of writing are pretty clear to me but I'm still unable to improve the efficiency of writing i.e ability to put down thoughts properly and quickly rather than spending a lot of time on structuring the blog, very rarely it feels like english knowledge is the bottleneck