maxk42

I'm not really worried about the cost of traffic. According to their latest 10-Q [0] they earned 152 million and spent 187 million for a net loss of 35 million. About 76 million of that (40.6%) went to sales, so they're certainly not spending it on the pipes. But despite a large increase in sales spending they're less profitable than they were during the same period a year earlier. They're losing $1.23 for every dollar they earn.

[0] https://www.sec.gov/ix?doc=/Archives/edgar/data/1477333/0001...

There is one: Cloudflare isn't profitable.

Cloudflare is still in growth mode: They're losing money hand-over-fist. AWS, on the other hand, is a money-printing machine.

Personally, I don't trust Cloudflare until they achieve profitability. They're going to have to raise their rates one day, and alienate the majority of their customers.

That's more than a 17% difference. Seems significant to me.

Scaled multiple self-hosted products up to tens of millions of hits / month. Scaled several for clients up as far as the hundreds of billions of hits per month. (Yes, hundreds of billions.) Never seen a kubernetes deployment that was (A) necessary or (B) simpler than some alternative.

Unless you're doing something very slow / processor-intensive, you'll probably never even need an autoscaler before you can afford to hire an expert to do it for you. You'd be surprised how far you can get with a small VM running your whole stack.

It also takes incentive away from the scientist with an unpopular or unique hypothesis. That stifles innovation.

> content that states a false claim as fact

is far from a blanket denial. They will arbitrate individual claims, apparently, and decide which claims are true and which are false. Google now decides what climate science is instead of scientists.

It was just an indentation typo. Swap check should be one level higher.

Because I have a need to optimize:

    // Given array a[1], ..., a[n]
    for i = 1 to n - 1:
        swapped = false
        for j = i + 1 to n:
                if a[i] > a[j]:
                    swapped = true
                    swap(a[i], a[j])
                if swapped == false:
                    break

This should reduce time complexity to O( (n - 1)^2 ) and stop sorting after the remainder of the array is sorted and tested one time.

Reminds me of the time I spent ten days building a memory allocator based on a red-black tree to optimize lookup times for an interpreter that needed to be fast. When I compared its performance against a linear search, it failed miserably. Turns out the fixed overhead in implementing a red-black tree vastly outweighs a naive approach when you're only dealing with 20 terms or so.

I worked for one of the "bad actors" that was terminated by Cloudflare. To this day they have not provided anyone with a reason for the termination. To my knowledge, the site in question was not breaking any terms of service and when I asked customer support why the account had been terminated they pointed me to the clause that said something to the effect of "Cloudflare may terminate an account for any reason or no reason at all."

They also refused to provide a refund. I will never use Cloudflare again nor recommend them to any of my clients.

Implicit overflow is how your processor works. Until that changes it will always be appropriate in some programming languages.

Actually, this is normal in the container shipping industry. Oversupply is the very likely result of this within 24 months. (Likely very much less.). Most of the oversupply typically comes from recycled ships and not new manufacturing. However, even during a downturn new manufacturing doesn't cease because container ships have a very short lifetime. New ships are sold typically in just a few years to smaller operations which in turn resell them again to smaller firms. At the end of their lifetimes, the ships inwill be owned by tiny (typically) East-Asian operators which will spend more time and money jury-rigging them to stay afloat than operating them under contract. The sea is not kind to steel. But when rates are high, even an old clunker can be run profitably, so these small operations with one or two rusting ships which are essentially lemons can continue operating them at a profit until rates dive again.

Not quite true. Although very wealthy people could afford to pay a high price for fuel for frivolous reasons, most people (even the wealthy) would tend to avoid doing so. So if a middle-class person were planning on using fuel to go on a road trip, they might postpone or cancel the event, which would help to ease the demand. Whereas if a relatively poor person were planning on using the fuel to go on a road trip - to attend the funeral of a loved-one - they might decide it's worth any cost and go anyway. Hence the people that need the fuel the most are able to use it, whereas the people who merely would like it are not. This extends to far more important operations: If an organ-delivery service needs the fuel to get a living heart from a donor to someone whose life depends on it, they're able to pay the added cost. In a system where prices are fixed too low to quell demand the heart patient will pass away due to lack of availability of fuel. It's literally a matter of life-or-death that prices be able to rise to quell demand.

In order to help the people that need fuel the most the price of fuel should be increased. It's a basic self-healing mechanism.

"Distributed" teams are teams where everyone works remotely whereas "remote" roles may include those where some workers are in an office together. It's an important distinction.

The way he presented it there's a 50% chance that opening another door reveals a goat. I've of the other doors will always contain a goat The key piece of information that is merely implied by the Monty Hall problem but not explicitly defined is that the host knows which door is the winning door and deliberately opened a losing door. If the host does this purposefully then you should switch your answer because two thirds of the time you will have chosen a goat and the host will be forced to reveal the other losing door. However if the host has opened a door at random and it happens to reveal a goat then changing your answer will not improve your odds. There is a 50% chance that the remaining door is a goat rather than a two thirds chance. He could have just as easily revealed the prize. Explained in this way the Monty Hall problem is not confusing. The wording is what was confusing.

Or rebrand it the 5-ouncer. 5oz being juuuust below a third of a pound, but still more than a quarter pound.

I like this - the hash function is memory-based rather than CPU-based so it's easier on your CPU while being more costly for attackers to spoof en masse.

Good thinking!

Couldn't reply to this comment - but sure enough, the permalink gives me the option. Thank you for the info!

Actually, my reply was to Folcon. HN simply doesn't allow you to reply to comments beyond a certain depth sometimes.

Perhaps mods have the ability to extend this for active discussions and that's why I can reply now?

Then you're not doing a fresh load of the page. There are over 30 images visible on the front page, so your measure doesn't pass the smell test, does it?

Bash scripts and cron. Automatic alerts go out to devs via OpsGenie when resource availability drops so we can get out ahead of it. 0 seconds of downtime in the past 12 months.

(1) Simple beats complex.

(2) You can spend weeks building complex infrastructure or caching systems only to find out that some fixed C in your equation was larger than your overhead savings. In other words: Measure everything. In other other words: Premature optimization is the root of all evil.

(3) Fewer moving parts equals less overhead. (Again: Simple beats complex.) It also makes things simpler to reason about. If you can get by without the fancy frameworks, VMs, containers, ORM, message queues, etc. you'll probably have a more performant system. You need to understand what each of those things does and how and why you're using them. Which brings me to:

(4) Learn your tools. You can push an incredible amount of performance out of MySQL, for instance, if you learn to adjust its settings, benchmark different DB engines for your application, test different approaches to building your schemas, test different queries, make use of tools like the EXPLAIN statement, etc. you'll probably never need to do something silly like make half a dozen round-trips to the database in a single page load.

(5) Understand your data. Reason about the data you will need before you build your application. If you're working with an existing application, make sure you are very familiar with your application's database schema. Reason ahead of time about what requirements you have or will have, and which data will be needed simultaneously for different operations. Design your database tables in such a way as to minimize the number of round-trips you will need to make to the database. (My rule of thumb: Try to do everything in a single request per page, if possible. Two is acceptable. Three is the maximum. If I need to make more than three round-trips to the database in a single page request, I'm either doing something too complex or I seriously need to rethink my schema.)

(6) Networking is slow. Minimize network traversal. Avoid relying on third-party APIs where possible when performance counts. Prefer running small databases local to the web server to large databases that require network traversal to reach. This is how I handled 30 billion writes / day: 12 web servers with separate MySQL instances local to each sharded on primary key IDs. The servers continuously exported data to an "aggregation" server, which was subsequently copied to another server for additional processing. Having the web server and database local to the same VM meant they didn't need to wait for any network traversal to record their data. I could've easily needed several times as many servers if I had gone with a traditional cluster due to the additional latency. When you need to process 25,000 events in a second, every millisecond counts.

(7) Static files beat the hell out of databases for read-only performance. (Generally.)

(8) Sometimes you can get things moving even faster by storing it in memory instead of on disk.

(9) Reiterating what's in (3): Most web frameworks are garbage when it comes to performance. If your framework isn't in the top half of the Techempower benchmarks, (or higher for performance-critical applications) it's probably going to be better for performance to write your own code if you understand what you're doing. Link for reference: https://www.techempower.com/benchmarks/ Note that the Techempower benchmarks themselves can be misleading. Many of the best performers are only there because of some built-in caching, obscure language hack, or standards-breaking corner-cutting. But for the frameworks that aren't doing those things, the benchmark is solid. Again, make sure you know your tools and why the benchmark rating is what it is. Note also that some entire languages don't really show up in the top half of techempower benchmarks. Take that into consideration if performance is critical to your application.

(10) Most applications don't need great performance. Remember that a million hits a day is really just 12 hits per second. Of course the reality is that the traffic doesn't come in evenly across every second of the day, but the point remains: Most applications just don't need that much optimization. Just stick with (1) and (2) if you're not serving a hundred million hits per day and you'll be fine.

> the JS fragments thankfully should load only on first visit and then get aggressively cached by DDG/your browser

According to Alexa you have a 46.4% bounce rate. [1]

When 46% of your users aren't coming back, how does 31 round-trips to your server for 100% of first-page visitors save anyone time or bandwidth? Your pageviews per visitor is 6.8, meaning the 53.6% that stick around view an average of 11.8 pages each. Even if there are zero subsequent js requests on other pages (clicking a random page I see 8) you would be generating 31 requests up-front to save 10.8 subsequent requests for about half of your users. (And again - in any scenario where the number of js fragments transferred on subsequent requests >= 1 even this benefit goes out the window.) How does that save you or your users bandwidth, server load, or other overhead?

The scale is not quite linear, but generally speaking, if you get your number of requests down from > 100 to < 5, you'll be able to handle around 20x the traffic with the same number of web-facing servers. Or alternatively the same amount of traffic with around 1 / 20th the servers.

Would that have a material effect on your costs?

[1] https://www.alexa.com/siteinfo/mangadex.org

My approach to what?

lol

> In general the less moving parts you have in a system the more reliable, secure, efficient and cheaper the system becomes.

100% agreed. This is not my first high-traffic site, nor even the highest. (I built the analytics system for a an Alexa top-10 site in 2010, reaching some 30 billion writes / day off of a mere 14 small ec2 instances.) I've never seen a k8s implementation in production that was necessary.

I will note that my Alexa-2k site is also a personal site (no revenue) and under constant attack. In fact we frequently suffer DDOSes that we don't even notice until reviewing the logs later because it doesn't suffer any latency under pressure.

I run an Alexa top-2000 website. (Mangadex is presently at about 6000.) I spend less than $250 a month.

I have loads and loads of thoughts about what they could be doing differently to reduce their costs but I'll just say that the number one thing Mangadex could be doing right now from a cursory glance is to reduce the number of requests. A fresh load of the home page generates over 100 requests. (Mostly images, then javascript fragments.) Mangadex apparently gets over 100 million requests per day. My site - despite ostensibly having more traffic - gets fewer than half that many in a month. (And yes, it's image-heavy.)

A couple easy wins would be to reduce the number of images loaded on the front page. (Does the "Seasonal" slider really need 30 images, or would 5 and a link to the "seasonal" page be enough? Same thing with "Recently Added" and the numbers of images on pages in general.) The biggest win would probably be reducing the number of javascript requests. Somehow people seem to think there's some merit to loading javascript which subsequently loads additional javascript. This adds a tremendous amount of latency to your page load and generates needless network traffic. Each request has a tremendous amount of overhead - particularly for dynamically-generated javascript. It's much better to load all of the javascript you need in a single request or a small handful of requests. Unfortunately, this is probably a huge lift for a site already designed in this way, but the improved loading time would be a big UX win.

Anyway - best of luck to MangaDex! They've clearly put a lot of thought into this.

Veterans don't tend to be wealthy. It's like saying "Nevada cops use civil forfeiture to steal a teacher's life savings"

I think what we're seeing is the large proportion of the workforce that can get by on one income rather than two doing so.

Many, many workers are working for non-economic reasons. Retired people work for exercise and social interactions. Married couples who can get by on one income no longer need to have both spouses working. And forcing students into attending classes from home has further forced more and more working couples to take a break to supervise their kids who are learning at home since most children (particularly young children) cannot be trusted to attend their classes without direct supervision. Many teenagers, also, who would have been in the workforce to have some spending money have decided the risks of the pandemic aren't worth it. And then there's the unemployment benefits. In short: Many people who can get by without working suddenly are.

It only takes a few percent of the population choosing to no longer work to create a massive labor shortage.

Sextoy.com was a 1-person company for many years. They probably broke ten million in annual revenues before hiring their first employee.

So was plentyoffish.