Hacker News Reader

Comment by rossdavidh | Hacker News Reader

A recurring pattern that I have noticed: 1) a company the size of Amazon, Facebook, or Google runs into a problem that comes from their gigantic scale, or the fact that they're trying to do something very different (e.g. Google Maps compared to old-style web pages) 2) Big Company comes up with a new technology to solve their problem 3) Many, many small companies that will never become big enough to need this, and are doing far less exotic things that require it, will try to make their ordinary web page into a microservice single page app with a NoSQL backend, because that's what successful companies do.

Imitating success often makes sense; after all, we don't want to imitate failure. But your company is almost certainly not facing issues at all like Amazon, Facebook, or Google, and also does not have nearly as many programmers to do it.

View on HN

+100 votes to this. My last job didn’t need to go Microservices and Kubernetes. But we did from the beginning. And the overhead of getting the greenfield project off the ground was immense. I kept thinking a aimple albeit inelegant monolith would have helped us prove the concept better and that we could have used it to then refactor once operational. If I take nothing else from my time at that startup I’ll tske this: sometimes you don’t need all the fancy tech that is hyped from the incumbent platforms with a vested interest in getting you onto their platform, sometimes it’s best to role out that tried and true Ruby on Rails or LAMP stack to focus on the product.

I'm more annoyed by the extremes. Services are awesome (not using the SOA term...) but how about a monolith for most stuff and splitting out a few key things? I guess it's so boring that no one writes about it...

Kubernetes sure makes deployments easy though.

I call this: Headline Driven Development

Another good example are monorepos. A decision maker sees a headline about Google and Facebook using monorepos and mandates his company to switch. Unfortunately, he or she didn't read or didn't understand the actual article that explains that it only works because of strict code reviews and extensive integration tests.

> Another good example are monorepos. A decision maker sees a headline about Google and Facebook using monorepos and mandates his company to switch.

Unfortunately he didn't know that `git` does not scale well with the size of the repo and Google have built their custom repository system to tackle this.

Google used Perforce, which has a somewhat diferent model from git and scales a lot better. A monorepo is also the obvious way to use it, as you can’t (I don’t think?) branch between depots.

We call it Hype Driven Development which is just as bad as Marketing Driven Development... and yes, monorepos, don't even mention it, good heavens.

Another hype that keeps coming back is the magic of trunk based development (with a random po reading some flashy article about feature switches as the wonderous solution for a-b testing AND faster development).

Nowdays I even consider putting react and angular into this pack, since, you know, "if it's good enough for facebook/google then it must be really good" - anyone ever tried to increase performance for a react site (and realizing that precompiling templates was not exactly a bad idea years ago) or hitting the wall with inline async helpers and rx magic might know my voes. But then again, give me a fast and reliable server rendered site over an unsteable trainwreck spa anytime and I will be happy.

>anyone ever tried to increase performance for a react site (and realizing that precompiling templates was not exactly a bad idea years ago)

I'm not sure I'm parsing this sentence correctly. Are you saying that precompiling templates and rehydrating them doesn't cut it anymore? If so, why not? I haven't used React much, but I've done some work in a framework with similar principles and I felt like proper precompiling, with basic HTML fallbacks cooked in where possible, provided all the performance of server rendered sites with the added bells and whistles of SPAs (including that most subsequent requests can load quicker than a full rerender, provided JS is present).

Afaik the render part is purely dynamic and the advocated style of creating closures with fat arrows to avoid scope binding (which is essentially the same if done in the render section) is quite expensive in terms of performance.

React and fiber tries to be smart about rendering with tree section diffing, but unless you use immutable, it's "not enough" to rely on - without immutable even in a smaller redux app there is a good chance that you have unintentional rerenderings triggered, which while may not create new tree branch renderings, still need to be evaluated.

This of course applies to the client, I don't have experience with nextjs or similar tools.

The issue with closures is easily solved, just declare it as class property (or a const variable outside the component if you're using SFC) and then use it instead of declaring it inside your render JSX. Any React-aware linter would not let you declare closures in JSX.

Can you recommend a tsx-friendly linter and its setting? The sites I worked with never followed this rule and whenever I tried to bring it up, it was too late to refactor things (at least from a roadmap/burndown po/sm perspective)

I use tslint and tslint-react, the rule is named "jsx-no-lambda"

Thing is, Google made the monorepo decision back when they were still small.

It seems architects often do it to so they can put that technology on their resume to find a higher-paying job. Your org is their school, and you are stuck with their convoluted buzzword-filled "training" stack when they hop away for a bigger company or project using their fluffed up resume.

I feel similar sometimes and my next thought is usually, “what part of the system is breaking down such that engineers at small companies don’t understand the scope of their projects?”. Is it a lack of experienced/authoritative technical leadership? Is it bad blood from the rails monolith days? Is it a symptom of a high density of top-tier minds mob-engineering solutions? Are engineers just lazy and prefer the industry status quo framework and some config scripts to the work of actually building software?... Is it even a problem that can be quantified (small company x could be here instead of here if they hadn’t spent resources following the giants)?

It fashion driven development.

Many cannot get to the grips that majority of work is just plumbing, yet another CRUD app or just plain maintenance.

So it is cooler to keep pushing for things like those companies, even if there is no need, after all the cv needs to match the new buzzword soup filter from HR.

> Are engineers just lazy and prefer the industry status quo framework and some config scripts to the work of actually building software?

It's worse than lazy, it's the new way of doing engineering that is glueing together ready made components.

Isn't that the method of every mature branch of engineering? Few projects involve inventing new tools, materials or processes in civil or automative engineering, for example.

My point here was not that people shouldn’t reuse good solutions and avoid redundant work if possible. It was that if this approach is “overly” desirable, then we will naturally see solutions deployed that are not 100% tailored to their application. Writing and designing good business is _work_.

I should have added that I don't enjoy fill-in-the-blanks kind of jobs.

I agree with the sibling comment of 'dcow'.

I will add a) innovation is a slow process and b) that a lot of projects take the _fast path_ of "fill-in-the-blanks" tools instead of thinking about a good solution.

To avoid speaking in vaccum, I will only cite 'Ansible' which is a tiny improvement over the previous hype. Obviously, if you consider the whole 'Ansible + Galaxy' there is value but still the wrong solution. All arguments I've seen for Ansible seem wrong to me.

AFAICT that time would have been better spent in functional devops approaches. My favorite being 'guix'. Also, I understand that Ansible galaxy was built by thousands of man hours by hundreds of contributors where a functional approach will require prolly more time and more focused effort.

Some additional root-causes:

* Ex-engineer managers pointing out that technology, then either brown-nosing or incompetent engineers running with it

* Tech debt: throw-away prototypes / experiments forced to production products

* Engineers padding their technical resume (probably intending to transfer before the debt is due)

* Engineers reusing existing knowledge instead of learning

* Premature engineering, small teams building for a big (hypothetical) future instead of building everything that comes in between

* Consultants / Contractors

* Low hiring bar

And plenty more. It's usually a systemic failure, with multiple parties failing to do what's best for the the customers and the business.

> * Engineers reusing existing knowledge instead of learning

Usually I see the opposite. Engineers jumping on hot technology of the day (and making a half arsed job of it) rather than getting to know any one technology well.

Indeed! The good systems I see are the result of tuning and adjusting over time based on domain experience and experience with the stack. Good stacks take elbow grease to tune for a given environment: incremental factoring and cleaning based on actual experience with it. Development efficiency and productive reuse does NOT come in a box. Barring a huge AI breakthrough, the latest buzzword won't do your hard thinking and analyzing for you.

Yes it's a lack of good technical leadership.

I think labor market incentives distort a lot of engineering decision-making especially at smaller companies, where the boss heard that React is hot and decides Solution X should be built with React.

Developers eager to develop their skills, increase their rates and advance in seniority end up focusing on the new hotness because of this.

Engineers who want to increase their impact and scope should focus first and foremost on the skill of making smart tradeoffs between technical realities and the requirements of other business functions--aside from simply writing code, this is what engineering is indispensable for.

Beyond that it pays to go deep in your mastery of a particular platform, the dependencies underlying it, and related technologies--all of these enhance your ability to design solutions.

Unfortunately the reality is there are very real financial incentives to keep jumping to the new hot thing every year.

> Engineers who want to increase their impact and scope should focus first and foremost on the skill of making smart tradeoffs between technical realities and the requirements of other business functions--aside from simply writing code, this is what engineering is indispensable for.

I have recently started contracting in London, and this is 100% what I have found in my younger colleagues. Trying to do everything "the right way" without keeping in mind at all the business constraints has created a total reciprocal distrust with upper management - which in turn isn't at all able to explain how being able to do that makes you a much better developer for the real world, not for some fantasy world where time and resources are infinite.

This doesn’t only apply to technology. It applies to hiring and other organizational processes as well.

e.g. "the spotify model"

Rings true. I once worked a contract at a place with proprietary hardware, that wanted their software extras to be purchased separately, and accessible through an "app store". This seemed to imply they thought somebody else would develop apps for their store, which was odd because if their proprietary hardware (which was quite expensive, justifiably so) sold 1000 units it would be considered successful. Basically nobody made apps for their app store except the people who worked for the same company that made the hardware, and I had a hard time understanding why they didn't use the (much simpler) model of "if you pay us $X more we'll throw in this bit of software". Instead, they had to develop all of the software for running and maintaining an app store, including the issues of making sure that apps uploaded to their store were not malware.

Then, I realized what was going on: Apple had introduced their app store a few years before, and it was a big deal. Steve Jobs was the CEO on the cover of every magazine. The CEO looked at Apple, and said, "we must have an app store", and that was all there was to it, even though Apple sold several orders of magnitude more units, making the concept of an app store way more suitable for them than for the company I was working for.

> also does not have nearly as many programmers to do it

I tried to make this point at the last place because they were insisting that because Amazon et al used microservices, they were obviously the right way. They were planning on 19 microservices to replace the monolith with a dev team of ... 3 perms and 2 contractors.

> 19 microservices ... with a dev team of ... 3 perms and 2 contractors.

I thought the main benefit of a microservice architecture is to break up an application into smaller services that are about the size that a small development team can build and maintain on their own. I'm not experienced in this area, but more than one microservice per developer seems like a major "architecture smell."

> I'm not experienced in this area, but more than one microservice per developer seems like a major "architecture smell."

It's not a smell; it's a hot, steaming turd.

> more than one microservice per developer seems like a major "architecture smell."

Yeah, it's normally that you need N developers per microservice (I've seen N=5 recommended, for example.)

The prefix micro in Microservice is very misleading. It is a microservice in the context of a >1,000KLOC monolith. I do like N=5 guideline as a way to make you think twice before making too many little services.

That's not the main benefit. The reason Google et al split services up into independent servers is for physical reasons. They have large datasets that don't fit on a single machine, so they have to shard the servers that handle it, and they have want to maximise their use of memory and CPU, so running unrelated tasks on the cores is a bad idea - at which point why link in the code? Before you know it you've got "microservices" except they aren't micro, they're machine sized.

Somehow this got mangled into "microservices are good because they help teams work together" which isn't what it's about at all. At least at Google the smallest hello world server was something like 50mb when I worked there, just because of the huge dependency graph. It was common for servers to pull in hundreds of modules maintained by nearly as many different teams, all in a single binary. Each server was essentially a "monolith" except for the stuff that had to run in separate backends because of resource constraints.

Exactly my feeling. Also, when I hire people coming from the startup industry I have to explain why we don't use kubernetes (we manage 2 servers for our production) or redux (good old publish-and-subscribe works great for us). Some of them just don't get it. Others feel relieved.

Ok, fair point - but redux is great even if your app is small.

Exactly that. Our app is >60kloc of typescript/react, we target android,ios,macos,win32 sharing >85% of the same source code with react native and react web, many abstractions, a lot of local storage and sync with remote server, and so on. Not a small app. Still, we don't use redux, just good ol' publish and subcribe with appropriate separation of concern. My point is there are many ways to manage an app state, redux is one of them, and people can't get to understand how there are many other possible software architectures that provide maintability and scalabity of the source code, and yet does not involve what is seen as the all-incompassing pattern/tool called flux/redux.

I agree, it does sometimes feel that everyone is taking crazy pills. Why did everyone start caring about mutability all of a sudden?

I really like MobX ATM. It's a lot easier to explain to new people as well. Redux is like... too much...

People started caring about mutability out of two reasons I would say:

- It’s a source of thread-safety bugs. With multiple cores even in phones and async programming rising this has become a bigger issue.

- It’s a source of general program logic bugs.

I wouldn’t say that it has come all of a sudden. Like other modern techniques it has been standard in functional programming for years. In mainstream programming languages it’s also not totally new, for instance immutable strings were a design decision in java.

You are so right for the thread-safety, immutability is a major improvements. However, I am pretty sure OP was referring to immutability in javascript, which is single-thread by design, and yet many javascript developers use immutability like a no-brainer and is seen as best-practice to the point where people don't even remember what it solves. Just that it must be used.

Well this is quite bit of generalisation. My business eradicated servers and containers and replaced them with AWS Lambda Functions composed in microservices architecture backed by DynamoDB tables. The result was an order of magnitude reduction in operational complexity, increased stability and reduced cost. This is not fad. This is real.

Lucky you.

There are very few established businesses which can 100% squeeze themselves into the constrains of serverless architecture.

> also does not have nearly as many programmers to do it.

Nor the underlying infrastructure, such as, for example, physical server hardware that's a fraction of the cost of cloud.

Kubernetes comes to mind.

Thank you for saying this. Its a personal battle preventing me from working on my own startup trying to imitate the likes of Google, Amazon.

I hear this quite often but the case is that if you are looking for financing the main technical concern of the investors is how (and if) your application does scale - talking about SaaS. If a SaaS gets popular the new user load can be quite heavy and in the past there were a lot of story's about companies that needed to rebuild there system because of this (I think pipedrive was the last example I heard of). Although it makes sense to ignore scaling at the beginning you need to design your application in a way that you can easily transition to something that can scale.

> the main technical concern of the investors is how (and if) your application does scale

How often is this borne of ignorance, though?

> in the past there were a lot of story's about companies that needed to rebuild there system

More specifically, are investors merely remembering the struggles of the dot-com boom, when transistor density was one thousandth that of today?

Internet usage (number of users and number of services each one uses) has grown, too, but I haven't seen statistics that suggest this would be more than 100x of 20 years ago.

> Although it makes sense to ignore scaling at the beginning you need to design your application in a way that you can easily transition to something that can scale.

This presents a false dichotomy. One doesn't have to ignore scaling at the beginning to make the decision to trade initial time-to-market for eventual "horizontal" scalability (usually what is meant by "can scale" [1]), but that may be what happens in practice.

[1] even though one can buy a remarkably large single server 3-6x price premiums over mid-range single servers, as well as use other, more traditional techniques, before "needing" to rebuild the entire system.

The investors I know and have to work with are mostly no technicians. They scan the news for investments and reported problems in growing business. If you can convince them that you thought about scaling issues and know how to solve them then they come up you should be fine :)

That belies the previous assertion that they're concerned with the "how" of scaling. What you're desribing is merely the investor (usually VC, which has been discussed here before) behavior of demanding that founders have a confident answer to potential challenges, even if it's bluster.

That's a situation fairly far removed from a VC refusing to invest because the company isn't imitating a FAANG's architecture.

I've seen this so many times unfortunately. Additionally, you don't need a big data architecture when you get 20GB of data/year.

> you don't need a big data architecture when you get 20GB of data/year

Probably no even if it's 50x that at 1TB/year, unless it's in very small chunks and/or requires an outsized amount of (possibly realtime) processing.

Current commodity servers will scale up to 12TiB of main memory, and if you're willing to use previous generation CPUs, those servers can twice the RAM (24TiB). Under half a megabuck without storage.

Wet Streets Make Rain(tm).
Not the same thing, but the psychology is related.

Ref: Michael Crighton; Gell-Mann

Adding to 1): more often than not, the "scaling" issues are 95% social and 5% technical.

Reply on news.ycombinator.com