CUPID – the back story
dannorth.net
dannorth.net
This change in outlook of existing thoughts/ideas is how many crafts grow, such as martial arts, painting, philosophy, etc (instead of stagnating). Sometimes we need to frame things in a more modern manner, and sometimes we need to discard them completely. In this case, I think re-framing the concepts is helpful, and I found it to be an interesting point of view. I agreed with a good amount of it, but I don't think we need to discard SOLID principles just yet.
What counts as a "responsibility", for instance. Where I see one responsibility some people see two and vice versa.
The Single Responsibility Principle is not a rigorously defined Scientific law, where the "responsibility count" can be exactly measured and minimised.
It is a subjective design guideline, with elements of experience, of context and of "I know it when I see it".
This does _not_ make it useless. It is still very useful indeed. But you do have to understand that disagreements over it are not always about "who is objectively right" but "which style makes the most sense in context".
If a set of principles lets them both think they're both correct and the other one is wrong, what exactly is the point of those principles?
This isn't just a coding thing. It's also, say, why society follows a body of laws rather than something like the 10 commandments.
That's why, if I am on a team project, I much prefer working in opinionated frameworks with strong idioms. It actually doesn't matter if I think I could do it better, the framework has chosen a different way and that is fine. We all have to do it that way, and we can all compromise and collaborate. No one bickers about best, and we can get real work done.
Different when it is a project of my own, but if it requires teamwork you need a framework of collaboration as much as a framework of code.
This is my other beef with SOLID: no trade offs.
I'd even extend your point to say that there are many times when code is not ideal but fixing it isn't worthwhile.
It would be nice to have a set of principles which recognized costs rather than promoting a vague, idealized standard for developers to fight over.
Guidelines can be useful even when subjective. Vitriolic disputes over whose (subjective) view is "correct" is an example of toxic behavior, not a problem with the guidelines being used to justify such behavior.
What would you think of someone who loudly insisted (without humor) that putting pineapple on pizza was wrong as a matter of principle? Is such expression in any way useful?
Some developers will say a (data structure) Controller is a class obeying the SRP
Some others will say the class can manage itself and not need a controller, so M and C can be one thing.
Some other will argue that it's better to make 2 controllers, one for complicated thing 1 and another for complicated thing 2, all based on the same Model.
The principle draws focus to certain (arguably important) design aspects. Multiple possible approaches are identified. Various concerns are raised in response. A good solution needs to balance these concerns against one another in context.
The principle is just one of many cognitive tools to employ when thinking about the problem.
With any paradigm, there is often ambiguity with certain elements, because those elements should be dynamic. What SOLID aims to do is say that these main points are not something you should dedicate brain cycles towards, as they are best spent elsewhere in the design.
It's because, unless you're careful, human language is insufficiently precise by its nature for many domains.
This is why mathematicians communicate with mathematical notation, for instance. It's why lawyers use special lawyer only terms (or why interpretation of the law is an explicitly separated function).
With SOLID the lack of precision renders the whole exercise rather pointless.
You're supposed to be able to use these principles as something you can agree upon with other developers so that you have a shared understanding of what constitutes "good" code.
However, it doesn't happen. Instead we argue over what constitutes a single responsibility.
SOLID isn't the only example of this. "Agile" is actually far worse.
If your assumption is that design guidelines can have mathematical precision, then you're going to be perpetually disappointed.
> Instead we argue over what constitutes a single responsibility.
I'm sorry that you can't reach consensus. but IMHO, the false idea that there is a single, mathematically correct answer (and if people differ, therefor someone must be be Wrong, capital W) is often part of the problem, of what stops an agreement on how to move forward pragmatically being reached.
I don't think that there is a set of design guidelines that can be applied with mathematical precision, but there are a set of code "costs" (like cyclomatic complexity) which can be measured with mathematical precision.
That is to say, all other things being equal, if a pull request non-negligibly reduced cyclomatic complexity I'd be happy to claim that it increased code quality. I don't believe that this idea is false.
"Responsibility" is IMHO about intent and meaning, which as a human concept is not reducible to measurements of this kind. Cyclomatic complexity is all fine and well, but it cannot tell you if an intent is being met or not. It is silent as to meaning. It's relevant only if you can compare two pieces of code that do the same thing; it's not talking about what that thing should be.
I tend to agree with that, and the original article in general. Mr Martin's approach helped originally, but the dogma now can and should be moved on from.
For example, an Entity part of the persistence layer in the app, has the responsibility to persist the state of the Entity to the database, but the responsibility to know the information is with an object that is part of the business logic layer. The information that is stored in the Entity to be able to persist it to the DB is just a cache of the information in the Business Object, the Entity is not responsible for it, it just holds a cache. If the same object would be responsible for both holding the information and persisting it, it would have 2 responsibilities.
This might sound somewhat confusing and useless, but it isn't so. Imagine a future where computers have a form of RAM that is not volatile. There is no need for a database, in the classical sense – whatever is in RAM when the computer is powered off/rebooted will still be there when the program resumes running.
I read no such thing. (Speaking of straw men...) instead, I read him saying that the alternative to SOLID was “Write simple code.”
I think there’s room to criticize this essay, but that’s a bizarre one to lead with.
The part you missed is in the same sentence as the three words you quoted.
"Instead I suggested to write simple code using the heuristic that it 'Fits In My Head'."
The obvious criticism here is that now any developer who wants to defend their code will simply claim a given spaghetti incident easily fit in their head. The author even seems to acknowledge this line of attack in the next paragraph:
> You might ask “Whose head?” For the purpose of the heuristic I assume the owner of the head can read and write idiomatic code in whichever languages are in play, and that they are familiar with the problem domain. If they need more esoteric knowledge than that, for instance knowing which of the many undocumented internal systems we need to integrate with to get any work done, then that should be made explicit in the code so that it will fit in their head.
But that just protects against spaghetti made from esoteric know of internals. For example, in C a "big head" would still be able to justify using global variables (after all, that's idiomatic C), rolling their own buggy threading approach, and deeply nested side-effect based programming.
I'd much prefer pointing big heads to the rule of "do one thing" than to ever read a response of the category, "Well, it fits in my head."
The thing is: that's just a bad defense. If the people who are going to be to maintaining it and working with it are saying it's bad code, it's bad. Even if this is the only developer who is going to maintain it for now (which itself is always a terrible idea), eventually someone else will take over, and the closest thing you have to that "someone else" is the rest of the current team.
I'm having a hard time imaging someone seriously making this argument, who is not also:
* Trying to establish themselves as the sole possible maintainer, so either the company can't fire them or is completely screwed if they ever leave, or:
* Is a saying the rest of their team is just too stupid to work with
In either case, this person is a massive liability, and the longer they are employed the more damage they'll do.
What if everyone but one says it fits in their head?
What if you're just talking one on one, and have no idea how many team members it fits in the head of?
What if the person who is criticizing the code will begrudgingly admit that it fits in their head, but still thinks it's too messy?
"It depends". If the one person is a junior it's maybe not a code issue (unless junior-level people are expected to maintain it). On the other hand if it's a skilled, experienced, senior person saying they can't follow it, there's likely an issue with the code.
> What if you're just talking one on one, and have no idea how many team members it fits in the head of?
> What if the person who is criticizing the code will begrudgingly admit that it fits in their head, but still thinks it's too messy?
The greater point here is is that it's about code maintainability. "Fits in your head" is merely one proxy for that, not the actual goal. There are also hundreds of other things that lower code maintainability -- including being "messy", having bad names, poor or no inline documentation, no unit tests, and being "clever" -- and all of those are things that should be discussed and fixed.
Presumably everyone involved are professionals and working in good faith. If they really can't come to consensus: sleep on it. Involve more people. Suggest alternatives. Pair up.
Someone consistently on the losing side of these discussions -- whether they feel they're constantly dumbing down their code, or that the rest of the team is writing unmaintainable spaghetti nonsense -- should probably reflect on if this is the right team for them.
The "bigger heads" types will use SOLID or any other argument to justify a higher number of abstractions. While the other type will do the same to justify a lesser.
At the end of the day, there is no right solution, but what is so called "cultures" that group people with similar taste and cognitive fingerprint, and provide best practices.
If I can't understand it, neither can the next guy.
slaps on hood
Care to elaborate as to why those are strawmans?
Without that, this is a no-argument, which is probably worse than a strawman.
>and then suggests that instead we should make software as complex as possible but no more complex than the coders own comprehension
The combo (go and make "software as complex as possible but no more complex than the coders own comprehension") is suggested nowhere in the post.
The second part of the combo (make software "no more complex than the coders own comprehension") is indeed said, and is very sensible advice.
It is, however, combined with the inverse advice of "make it as complex as possible" which you claim the author combined it with: with the advice to keep it simple.
The strawman here is the fallacy that SOLID (SRtC) is clearly not saying that a software, composed of "code", should only do one thing. By that reasoning, SOLID rules out any software that provide multiple capabilities. Your editor, for example, has multiple capabilities. It can save. It can highlight. It can cut. It can paste.
So, a reasonable reading of SOLID naturally is not ruling out composing complex software using "simple single purpose code". However, OP is assuming a ridiculous reading (the strawman) that basically can only be valid for software that has only one irreducible capability.
One can read SOLID SRtC in terms of capability as "compose complex software using single purpose code".
One can read SOLID SRtC in terms of change as "changes to code should consist of one, or a sequential set of, single purpose changes".
That's not what TFA says here. It doesn't claim that SOLID says that software as in 'a full program' should do one thing.
It just says that SOLID says that a piece of code (as in a function or a class) should do one thing, which SOLID does say, and which the author of TFA disagrees with.
Perhaps the use of the word "code" (or "non-trivial code") is confusing, but the author doesn't imply the whole program with that, but the same as SOLID does (a unit of code):
"Code should fit in your head at any level of granularity, whether it is at method/function level, class/module level, components made up of classes, or entire distributed applications".
As long as you manage to keep artificial complicatedness out of your code, you will always have the complexity of the problem mirrored in your code. A common problem of ideas about object oriented programming like SOLID or Clean Code is that they have a focus on classes. If you keep your classes very simple, you will instead end up with a complex architecture, where you might have zero responsibility layers or functions that just pass on the call further to a layer down.
Code is cheap to write, and mostly debt. A software product - a working system that meets some particular need - is not. The distinction is that building a software product is much more than banging out code; it's the experience of figuring out what exactly that need is (gathering requirements, getting a system out for users to test, getting real-world feedback). Sometimes you capture that in documentation, test cases, ADRs, comments, commit messages, etc. Sometimes it's in your head, which is okay as long as you're still there. (Of course, if it's in your head and you leave, then the next set of programmers will be scared to change the system until they redo the work of figuring out what the software is, despite the code being in front of them.)
If you have that understanding about what the software does, ideally in the form of automated test cases, you can rewrite the code. So you may as well bang out the code in a way that gets you a working system and remains comprehensible. Once you're making enough changes to it that you're worried about it getting incomprehensible, proceed to rewrite it. Probably the world has changed in many ways sine you wrote it - maybe you can run the system for a lot faster and cheaper with containers in the cloud talking to a SaaS database than with your expensive IBM mainframe talking to DB2. Or perhaps the requirements are changing significantly, and starting with the old code doesn't give you much of an advantage.
Trying to keep a codebase comprehensible over the long term makes it, if you'll excuse the pun, solidify - it becomes increasingly hard to make changes that weren't anticipated by the original design, and it also requires more and more effort to just do anything. You might be able to swap out the relational database for another one, but you probably can't switch to a design where you're using a key-value store with totally different performance characteristics. And even if you do just want to change the database, you have to figure out how ten classes get dependency-injected into each other before you can start coding.
All the time you spent making the code "future-proof" so it remains forever comprehensible could have been spent simply not doing that, delivering business value, and writing new code as the need arises.
This has nothing to do with tooling, but the fact that pulling the rug under an established code base could have very unintended effects, compared to simply adding and extending functionality without touching what is already there.
By doing as the author suggests you’ll end up with either 500 broken tests or 5000 compiler errors in the best case, or in the worst case an effectively instantly legacied code base where you can’t trust anything to do what it says.
I once had to change an entire codebase’s usage of ints to uuids, which took roughly 2 whole days of fixing types and tests, even though logically it was almost equivalent. Imagine changing anything and everything to “make it do something else”.
And how long would it have taken for the original author to use an IdentifierFactory instead of ints and write meaningful tests for it? Less than two days?
OCP though more commonly refers to: 1. Building small, universally useful abstractions first 2. Extending behavior of that abstraction or system by writing new code rather than changing published code directly.
This is trivial when you have a few patterns under your belt. Template factories, builders, strategies, commands. I mean, while it’s not the best idea in most cases, even just inheriting a parent class and giving a new concrete new behavior is still better than changing something fundamental to the system.
Like has been said 999 times in this thread, software isn’t black and white. You have to make choices about where you go concrete and where you abstract, and gauge that against risk factors. A somewhat complex class you expect to go away in a couple months? Make it a god class that anyone who wants to can scan through. A fundamental class that will be used by hundreds of developers and underpin most operations in a production system where 5 minutes of downtime costs tens of thousands of dollars? It’s worth the upfront cost to build with these standards.
Honestly, if you cannot change your code, you have a problem.
https://drive.google.com/file/d/0BwhCYaYDn8EgN2M5MTkwM2EtNWF...
Flipping it around, if base libraries were to be always open for modification instead of extension, then instead of writing your feature logic in your own codebase, you might be tempted to submit a pull request to React.js to add your feature logic. That sounds ridiculous but that's the equivalent of what I see a lot of new engineers do when they try to fit the feature logic they need to implement anywhere it makes sense, often in some shared library in the same codebase.
That's still a matter of tooling. With type-checking, static analysics, and test suites, changing code doesn't have "very unintended effects".
Back in the day, without those, things were much more opaque.
... and, you haven't ever seen a business requirement of what used to be a "software" to become a framework / plateform instead ? it happens all the time
But I've never seen it happen in any way close to resembling what the original architect thought would happen, and as a result, all those abstractions and generic implementations not only added time to the mainline development, but in the end actually got in the way of the abstraction that was needed.
If you can decide on a language, framework, critical libraries etc. then you should be able to decide on a database. It's probably more important than your application anyway.
At the same time, substituting Oracle with a lightweight DB in an environment where full-time DBA was developing a database with loads of stored procedures and DB-specific stuff wasn't something really feasible - no abstraction layer would solve that.
But as you said, some of the problems were when it got really DB specific. A simple layer that could be swapped for a simpler (and not performant, but didn't matter with little data) variant locally was nice.
This also improves efficiency in operations, not necessarily development. If you used a library/framework for database access anyway, it's not an extra expense. There's ultimately a portability concern even if "vendoring", it only imposes a cut-out to permit control of necessary change).
After a few unpleasant experiences I endlessly advocated we should always use an interface to access populated data objects and not interact with the database directly, not even running queries directly but always using at least lightweight IOC. I also advocated for testing where known result sets were fed through mock objects. After all, saved result sets could also be used to test/diagnose the database independently after schema/data changes. My experience predates a lot of ORM and framework responses.
Unfortunately later frameworks (intended to abstract these concerns) became ends in themselves, rather than a means to an end. These were used to satiate "enterprise-y" concerns (sacrifices to the enterprise gods). If you could afford to deploy operational Oracle, you would't necessarily flinch at the cost of the extra (often pointless) layers of abstraction.
DRY and YAGNI are two basic concepts I've always tried to work with. Do I need this piece of code more than once? Should be extracted/created in a separate place. Would I need this in other projects? Library.
YAGNI is most likely directly conflicting with most of what's mentioned, if the developer simply asks themselves: do I really need this? Will this project benefit from having separate layers of abstraction for things that are most likely never going to change?
I always think twice before writing a single line of code, with the main point of focus being if future me (or anyone who reads that piece of code) will be able to understand and change things, if needed.
Just because there is a block of code that’s being used in two different places doesn’t mean it should always be abstracted out. There’s a subtle yet mindful consideration whether these two consumers are under the same domain. Or if it should exist as a utility function across all domains. And if that’s the case, changing the code to make it generic, simple and without side effects is ideal.
I’ve seen too many of these mindless DRY patterns over time, and they eventually end up with Boolean flags to check which domain it’s being used in as the codebase becomes larger.
I also propose LAWYD - look at what you've done, a mandatory moment of honest reflection after drying out some code where you compare it to the original and decide if you've really improved the situation
If they absolutely have to be the same, wrap'em in a function. If they just happen to be the same, leave them as separate copies. If I don't know, make an educated guess.
And it sure is easier to just not argue about it and abstract everything. The real pain only comes later, after all.
Without DRY, this would be a perfectly acceptable practice. DRY gives me something I have in my head when I see this and refactor into something that is manageable. DRY gives me something I can point to and say "please for the love of god don't perpetuate this folly".
You can argue about that just the same as when client needs a feature X.
In my case, I was very often getting away with YAGNI or 'how about we implement nightly sync from pg to oracle' but not always.
Long-lived OSS software is a relatively stable bet, but the point still stands.
I feel the industry has a long hangover from the 80s and 90s in terms of the holdup problem. Oracle, basically, created a massive externality of anxiety about vendor lockin that continues to impose drag to this day.
In the majority of cases, I think a simple “runs on Postgres” is even more valuable today (unless your product is a database abstraction layer).
Exactly, so make the changes when you need them, otherwise you are relying on hitting the abstraction lottery
This is a clear example of where YAGNI applies, I think.
Extra work plus extra boilerplate to maintain. No payoff.
That said, extra code + boilerplate is a great way to treat riskier software.
One is that you're using two SQL databases and you're not using any advanced features. You started dev on MySQL, and then the company says "Thou shalt use Postgres" (or whatever). You don't need anything fancy in your own code to handle this. You're still making SQL queries, you're just swapping out the database engine. Technically, this is an example of dependency inversion (depend on the SQL abstraction), but you also didn't set out to do it - basically any programming language you're likely to use has the common database libraries use a common abstraction for sending queries. And you didn't specifically make sure you were writing generic SQL, you just happened not to need anything.
More commonly, you're switching databases for a specific feature. Maybe you realize PostGIS (or whatever) is going to solve a problem for you very well. But then you're changing how you model data, what your schemata are, and even how your code is architected and accesses things in the database. You're deciding to move certain logic from your code to the database engine - might even decide to move certain logic from the frontend into the backend, or change how request routing works, or something. This is a fantastic reason to move databases, but no amount of abstraction can prepare you for it, because you're fundamentally changing what the abstraction is. And you're deliberately abandoning SOLID because you're picking up a dependency on a concrete database.
But the real case I've seen is where you're switching databases (or data storage layers, more generically) to a different model - MongoDB to not-MongoDB, a C/P database to an A/P one, a relational one to a key-value store, etc. This is the above case but even larger. There is no abstraction you could possibly write that could encompass the old and new cases. It requires rearchitecting how your code works.
And then there are the most boring of cases - the ones where a database swap sounds doable in theory and the code is supposedly using an abstraction layer, but no one has ever verified that the code doesn't make assumptions about what database it's on and the code has gotten too big, so we just get an architectural exemption from "Thou shalt" and we run our own instance of the wrong database, because the overhead of running our own DB costs the business less than getting the swap wrong.
(Some public examples of these sorts of database migrations that come to mind: https://slack.engineering/scaling-datastores-at-slack-with-v... is about how Slack couldn't move away from MySQL and the architectural assumptions they made about it and had to rule out migrations to non-relational databases out of hand, and https://about.gitlab.com/blog/2018/09/12/the-road-to-gitaly-... talks about how GitLab moved from NFS storage to an RPC service, requiring a lot of refactoring of callers.)
FWIW, every single db abstraction I’ve ever witnessed was worth it - if only so that one could run tests in a sqlite and run prod in something else, or as a way to contain vendor lockin in the code (I’ve seen projects successfully migrate from a plsql-heavy system to mysql because the code was well segregated, and I worked at a startup that literally imploded bc the database was metastasized all over the place)
Anyway, as I put it, abstracting data storage is a no-brainer for me, and it saved my skin every single time. I don’t expect to convince anyone here to go do it. :-)
How do you avoid the issue where SQLite doesn't support all the things your real DB supports? (i.e., why is it worth running tests on SQLite as opposed to a local instance of the same DB software?)
I'll read some of these public posts....
WRT sqlite, the obvious advantage is test setup doesn’t require any local infra, and it forces your system to NOT use vendor-specific features. That school of design is essential for places that want to avoid lockin for whatever reason. This isn’t possible in every case - some companies have a great DBA team, for instance, or strong specialization around a specific db (cough Oracle shops cough), so obviously that flexibility wouldn’t be advantageous. As everything else in our industry, it depends :)
What were the reasons for the swap and what were the outcomes of it besides that it actually happened? Costs, performance, scalability etc?
I think hardly anyone would argue that those transitions are impossible, because they are indeed possible. The main question is usually if your abstraction can leverage the benefits of underlying implementation or it is just a common denominator. I can imagine a clever technical strategy in a startup, foreseeing future growth and starting with a simple DB before moving to a more complex in maintenance but scalable solution. This may work. Often it doesn't.
Abstractions are a way of trying to find a common ground conceptually. Agreed, abstractions should not be multiplied unnecessarily, but in the same way it's easier to learn Newton's Laws than read Kepler's data, a few well chosen abstractions help others organize and understand what's happening in a code base.
What's always amused me, in a tortured sort of way, is that SQL is already the abstraction. Then people go and layer an abstraction on top of it, such as some ORM flavor-of-the-day, which is obviously much less universal than SQL. In "SQL" I'm including vendor extensions too (Oracle, MySQL, etc.). It's easier to read up on vendor extensions than learn yet-another ORM.
Which do you think has had the longer shelf life: MySQL or some random Ruby ORM? If you've been doing MySQL since the '90s then it's largely the same as MySQL of 2021. That Ruby ORM? Probably hasn't seen an update since 2008. I can't even remember the names of all the ORMs I've had to use over the years.
class User {
static get(id) {
embed your SQL here
}
}Literally impossible to get any new clients to agree to run on oracle
That's what I mean when I say simple. Don't obscure your code with unfalsifiable assumptions reified, as related in the typical obscure vernacular...
I have had to deal with RDBMS-swapping for fairly large and transactionally intensive applications. I appreciated code that embodied SOLID to the extent that it reduced the amount of code to inspect and improved the quality of sizing up what needed to be done. However it was much easier to "lift" up systems to the new objective than "unravel" previous efforts to insure portability without a defined and testable objective of portability.
Ultimately the goal of SOLID is to be able to change any important aspect of the software independently from any other. If the DB is going to outlast the business logic you’re writing, there’s no problem having it depend on the DB concretely.
Separating the what and the how does have the effect of changing how you approach reasoning about a program though, so SOLID is not without penalty.
Edited for clarification.
I wonder if others here have applied some sort of domain driven design (domain models agnostic of database) without going down the whole repository route.
How can you be fed up with something that will never occur?
> “dependency inversion has single-handedly caused billions of dollars in irretrievable sunk cost and waste over the last couple of decades”
Oh please. Is there anything in programming that hasn’t had the “Irreparable harm to humanity” sticker attached to it by now?
Not sure that's the right attitude.
Unit testing is the only reason pretty much all of my code depends on interfaces. Some people seem to consider this a bad thing/over-engineering, but it's how I've seen it done in every place I've worked at.
How do you do it?
More interesting is if your code relies on the outside world, then instead of abstracting out the connection with the outside world abstract out your business logic and test it separately.
So instead of a database repository being injected into your domain services, make your services rely on pure domain objects which could come from any where be it tests or the database.
Make a thin outer shell which feeds data into your domain logic from the outside world and test that via integration tests if necessary.
I'll admit I don't have the full picture here, but I have used this technique to good effect. The core idea is don't embed your infrastructure code deep inside your architecture instead move it to the very top.
This the thing. People have a tendency to overuse mocks. The point of automated testing (whether it's unit tests or something else), is to enable refactoring. That's really the only reason. Code that you don't touch doesn't suddenly change its behaviour one day.
In a previous jobs the developers started to go crazy with mocking and it reached a kind of singularity where essentially if a function called anything, that thing was mocked. It definitely tests each function in complete isolation, but what's the point? It makes refactoring impossible, which is the entire point of the tests in the first place!
This excellent talk completely changed the way I approached testing. Every developer who writes tests needs to watch this now! https://www.youtube.com/watch?v=EZ05e7EMOLM
> Code that you don't touch doesn't suddenly change its behaviour one day.
This can and does happen all the time, when the platforms and abstractions your code builds on change underneath you. This is why a compelling environment / dependency management story is so important. "Code rot" is real. =P
Haskell programs tend to have this structure because pure functions aren't allowed to call impure functions.
That's what industry converged on, that a function/method = a unit, but it apparently used to be that a module meant to be a unit.
It can be that both interfaces and modules can be considered a "bag of functions".
Seems like a lot of confusion about unit testing, mocking and DI stems from this historical shift.
I believe that interface/method level testing is too granular and mostly results in overfitted tests. Testing implementation, not behaviour. Which can be useful for some algorithm package for example, but probably not so much applicable to typical business logic code.
TDD: where did it all go wrong. https://youtu.be/EZ05e7EMOLM
The end result is the same as if you'd created an interface or abstraction with two implementations (real and fake/mock), but you skip the part where a separate interface is defined.
The upside to this is that the code is more straightforward to follow. The downside is that the code is exposed/potentially coupled to the full API of the real implementation, as opposed to some interface exposing only what is needed.
I strongly recommend reading "Unit Testing: Principles, Practices, and Patterns" by Vladimir Khorikov.
That's going to answer your question and will unveil a whole world of testing without mocking (not that it's better or worse than with mocks). It might be a single best book about unit testing in general.
What you actually want is fast and self-contained integration tests. Have an API endpoint that returns some filtered data from the DB? Spawn a DB server locally, load some fixtures into it (e.g., phrased in the form of a migration), start up your application server, and curl it and see if the data is filtered. Then shut it down. In most stacks you can do all of this in a fraction of a second.
The reason most shops love unit tests is that it saves them from having to figure out how to run those integration tests. It's easier from a Conway's Law perspective to use your language's built-in testing facilities to test the function you're writing, let the ops team figure out how one starts up a database server, and let the frontend team figure out how one makes HTTP requests to your application. But you will deliver better software if you spend a bit of time figuring out all those things and stick them in a tiny shell script.
Some programs are nothing but Infrastructure (DB backed microservices) but some have complex Logic (complex problem domains). In the latter case the most important parts of the system can be tested without DI.
You still need DI but not pervasively.
In the case where there are mocks, the unit tests are almost always literally testing "can my programming language call a function and return a result" or "can we store data in a database if the database works?" They are functional tests disguised as unit tests, and are testing only non-production fake functionality.
Write functional code instead. Liberally use the compiler and the type system to make mistakes impossible instead of unit testing.
Unit tests are for when you can't express something in a way where the compiler and type system will save you from errors.
Everything else is a functional or integration test and should be testing the production system, not a mockup of the production system that works differently.
Basically, write Haskell programs in whatever language you're working in. Use unit tests as a backstop for when the type system isn't as good as Haskell's.
To summarize, you want to set up your code into two distinct categories, data and pure functions. Your data is at the top level of your code, so if you need external data that needs to be at the top level of your module. You then feed that data through a pipeline of pure functions that return with some sort of answer back to your top level module that can then pass into some other function(or through a quick lambda) that does the actual transformation at the top level. The key is that a function can either be a pure function or it can be an impure function. Never both.
As an example, let's say you've got code that has to read from a couple text files. Currently your code does this through a couple nested loops, with the loops passing in what folder it is in and what its file name is, but your code is filtering through some that you don't want to read.
To re-envision this code, you would create a pure function that comes back to the top of your module of which files to read. Then that top module would read those files, and pass them to some other function that does transformations on the result.
You now don't have to create mocks, just create data and pass that into your pipeline and then check if it gets the expected result. One of the interesting side effects of doing this, is at a certain point you stop unit testing for correctness(as the code is so easy to reason about written in this style), and start unit testing primarily for documentation.
[0]: https://blog.ploeh.dk/2013/12/03/layers-onions-ports-adapter...
This is a classic example of software-pattern-being-language-defect.
It is trivial to mock out upstream or downstream dependencies in Python, without any changes to the code under test.
This is basically what dependency injection is doing, but working around the fact that modular programming didn't really catch on.
Finally create decoders that instantiate those types from different sources.
https://fsharpforfunandprofit.com/fppatterns/
And
https://fsharpforfunandprofit.com/posts/13-ways-of-looking-a...
I'm still learning on how to apply these ideas in my own work (which doesn't use f sharp or functional programming) but it's a rich source of ideas
You can complain about “automate testing theatre” without being against automated testing.
The author isn't actually demanding you change your tests (unless they are unnecessarily complicated).
> Unit testing is the only reason pretty much all of my code depends on interfaces. Some people seem to consider this a bad thing/over-engineering, but it's how I've seen it done in every place I've worked at.
Unit tests require 'seams' to divide the tested code into units but those seams don't have to be interfaces. Some languages don't have interfaces at all - Ruby, say - and they unit test just fine. The pattern of every Foo having a corresponding IFoo is a bad thing - you should replace IFoo with smaller role interfaces. Having an IFoo if it isn't necessary to test Foo and there's only one implementation is indeed over engineering.
> I wish to know this as well. And what about functional programming languages?!
Unit testing's much the same in functional languages, though it tends to be easier because pure functions are easier to test. But how code gets its dependencies tends to differ a bit. It may help to think of dependency injection as parameterisation. Or, perhaps, parameterisation where you don't use the parameter immediately.
Suppose I want to test a function foo that depends on another function bar:
fun foo() {
return bar()+1
}
Well with objects and interfaces, I can do this: interface IBar {
bar()
}
class Bar(): IBar {
fun bar() { return 2 }
}
class Foo(val bar: IBar) {
fun foo() {
return bar()+1
}
}
And then I can make a test double for Bar in my test, and when I invoke foo, then it will call the bar of my test double.Now, of course, in this example, I could test with the real Bar and I don't need a test double. That's partly because the example is simple but also because foo and bar are pure functions. But let's ignore that and see how we can do the same in a functional language.
fun foo(n: Int) {
return bar()+n
}
The question is, how can we control the value of bar in our tests?The simplest answer is that, since bar is varying (between test and prod), then bar is a parameter:
fun foo(bar: ()->Int, n: Int) {
return bar()+n
}
Now the reason we don't do this in OOP languages is that we won't have bar at the call site. That is the production code calls foo like this: foo(n)
and not like this: foo(myTestBar, n)
That's why our test code looks like this: val myFoo = Foo(myTestBar)
assert(myFoo.foo(3)).isEqualTo(something)
though, if foo takes its dependency as a parameter, the test could just look like this: assert(foo(myTestBar, 3)).isEqualTo(something)
But the prod code won't look like that, it only passes n. So we need to pass bar before that. That's why we pass bar to the constructor in the OOP version.OOP languages generally expect you to pass all the parameters to a function when you call it but some functional languages don't require this. That is, you can pass bar on its own - a mechanism known as 'partial application'.
You'd use it like this:
// Do this when 'wiring up' the application
val myFoo = foo(myTestBar)
// now I only need to pass n
myFoo(n)
If your language doesn't provide partial application, then you can do the same with a function that captures the dependency: // Do this when 'wiring up' the application
val myFoo = {n-> foo(myTestBar, n)}
// now I only need to pass n
myFoo(n)
Now, notice that, since bar is just a function (of type ()->Int), I didn't need to create a separate interface IBar. For this purpose, functions "just work". They are equivalent to interfaces containing a single function (Java's SAM recognises this, if that's familiar to you).Notice also that an interface of a single function is as small as it can be. Often, we'll see a class Bar with, say, ten methods, and a corresponding IBar also with ten methods.
SOLID's Interface Segregation Principle says to prefer small 'role' interfaces over obese interfaces such as the ten-methods IBar. And, of course, if you're using functions, that happens naturally.
Hope this helps.
My point being, do you have more references for studying about tests?! As a NodeJs and Python developer today (with almost no tests) it's really hard work with these legacy code :|
If you can pair with someone who habitually works TDD, do so. Sounds like getting experience in a team with effective modern practices (see "Accelerate" by Forsgren et al) could change your life for the better.
If you want to test your code thoroughly you needs these techniques.
2. "...coupled with the way most developers conflate subtypes with subclasses..." Speaking as somebody who both likes SOLID and could write this essay/present the deck, I think there's a lot of confusion to go around. There are a lot of guys coding classes using rules better suited for types. There are a lot of guys applying OO paradigms to functional code and vice-versa. In general, whenever we swing from one side to the other on topics, it's a matter of definitions. There is no such thing as "code". There's "thinking/reasoning about code" and there's coding. You can't take the human element out and reason abstractly about the end product. Whatever the end product is, it's a result of one/many humans pounding away on keyboards to get there.
3. My opinion, for what it's worth: as OO took off, we had to come up with heuristics as to how to think about grouping code into classes, and do it in such a way that others might reasonably get to the same place ... or at least be able to look at your code and reason about how or why you did it. That's SOLID and the rest of it. Now we're seeing the revenge of the FP guys, and stuff like SOLID looks completely whack to them, as it should. It's a different way of thinking about and solving problems.
ADD: Trying to figure out who's right and who's wrong is a (philosophical) nonsense question. It's like asking "which smell is plaid?" Whatever answer you get is neither going to provide you with any information or help you do anything useful in the future. (Shameless plug: Just submitted an essay I wrote last week that directly addresses reasoning about types in a large app)
Indeed. This clickbaity style of laying out arguments is not terribly constructive. Software is not black/white. It's entirely grey. And there's a lot of room for contextual nuance everywhere.
Principles like SOLID (and DRY, and YAGNI, etc) are principles. They are not laws. Principles are guidelines which can help you make solid (heh heh) decisions. They are subject to context and judgement.
If good software design were as easy as memorizing a couple of acronyms, we'd all be out of a job. But it's not. It takes practice and experience. Writers and academics can make things easier by presenting accumulated experience in principles and guidelines, but there are no silver bullets. It's unfair and pointless to expect SOLID (or anything else) to apply in any and all cases.
I think that's a big part of the problem, and how we end up with articles like this. A lot of developers do expect SOLID to apply to every case, and I've seen fine code get rejected in reviews because it wasn't SOLID enough.
Prescriptive principles without informed discretion. That is the problem, and it's not just software.
"A foolish consistency is the hobgoblin of little minds."
It's terrifying how easily a reasoned and seeming sensible polemic may be interpreted into a foolish oppression, even by otherwise rational folk...
entirely? There aren't parts that are black/white? That's seems a bit black/white.
Principles, heuristics, "best practices," and just generally good ideas are not absolute truth. SOLID is like hand washing. Please do it! Unless you have a good reason not to.
The root of the "disprove a heuristic by a single counterexample" problem is a misunderstanding of logic. A heuristic is not a statement that universally all hands must always be washed. It is a milder claim that generally handwashing has proved useful via inductive means, so you should probably wash your hands if you want to minimize your risk.
Any expert in a given field should know times when not washing hands has been justified. But by the same token, those people know that they should still recommend hand washing to the general public because they won't know when it's not justified.
Wash your hands, please.
The post does seek out where the SOLID principles came from. And it's not really debunking them; just saying they're not absolutes. Which, yes, the title is click-baity, but I've certainly found people who treated them as absolutes, or tried to talk about code from SOLID perspective, and I've certainly never found that useful.
In fact, I've not found -any "best practice" to ever be absolute in a generalizable sense, and it's never been useful rhetoric to bring them up in a design discussion because of that. In fact, they sometimes run counter to each other. "DRY would say we should combine these together" - "Yeah, but Single Responsibility; while the code is mostly the same, they fundamentally are dealing with different things in different contexts".
Learn the heuristics, then deal with the subjective realities each project brings; anyone who tries to treat a given codebase as having an objectively correct approach, rather than a nuanced and subjective series of tradeoffs, is not someone worth talking or listening to.
As originally coined, DRY speaks of ensuring every piece of knowledge is encoded once. If pieces of code are "dealing with different things" then those are two pieces of knowledge, and DRY (per that formulation) does not recommend combining them.
I agree that there is a prevalent notion of DRY that is more syntactic, but I find that version substantially less useful and so I try (as here) to push back on it. Rather than improving code, it's compressing it; I've joked that we should call it "Huffman coding" when someone tries to collapse unrelated things in ways that will be unmaintainable.
Note that it's not just that syntactic DRY sometimes goes to far - it also misses opportunities where the original DRY would recommend finding ways to collapse things: if I'm saying "there's a button here" in my HTML and in my JS and in my CSS, then I'm saying the same thing in three places (even though they look nothing alike) and maybe I should find a way to consolidate.
There are, of course, still tradeoffs - maybe the technology to unify the description isn't available, maybe deepening my tech stack makes things less inspectable, &c.
In my experience, misguided "could this be DRYer" is usually motivated by superficial similarity, and a focus on knowledge is clarifying. (Although as I mentioned, "yes, it's repeated, but given tradeoffs that's the best in this case at least for the moment" is still possible.)
heheh I've made that joke too, also "let's un-complicate this into a pointless complexity," when its goes over their head.
PS. Interviewing senior devs, team leads and tech leads atm. And so far none (!) have been able to properly formulate this (even after I'm hinting that it's not only about code deduplication) and 75-90% believe it's all about code deduplication. Imo quite scary, and tells you a fair bit about the shape of the software dev industry...
Well, duplicate code is the typical manifestation, as the sibling comment [1] relates:
>>> if I'm saying "there's a button here" in my HTML and in my JS and in my CSS, then I'm saying the same thing in three places (even though they look nothing alike) and maybe I should find a way to consolidate.
"and maybe" it's just otherwise hard to get the point across, as you've discovered. Isomorphisms are easier to distinguish (and validate!) versus homeomorphisms, but unnecessary pedantry usually results in MEGO. We shouldn't expect senior developers to automatically embody the virtues of mathematicians or philosopher-kings...
yeah, I am nitpicking worse, but after 40+ years in the industry and suffering through tens of thousands of marginally relevant distinctions in "gotcha" interview questions, I am without shame even though my head is nodding in acknowledgment of your points...
[1] oops, I meant https://news.ycombinator.com/item?id=26532424
This is imo not a gotcha interview question. It's A) Intended as a way for me to check if the person has an understanding of when to deduplicate code or not (I don't care if they know what DRY is, and I try to help them without giving them the straight up core part of the answer, give examples, etc). I believe this is one of the fundamental parts of writing maintainable code and B) Able to formulate this so that others can learn from them. Both are critical skills for the role.
In my experience, by applying DRY wrongly you're creating a really dangerous code base (maintainability wise). It's probably even worse to deduplicate code wrongly than not deduplicating at all.
>It's probably even worse to deduplicate code wrongly than not deduplicating at all.
I strongly agree.
>A few questions: You're quote of the parent doesn't seem correct? Where did you find that quote? MEGO?
pardon me, I meant a sibling [1], not the parent comment. Otherwise I'm just relating my experience. MEGO is an acronym for "my eyes glaze over", I see searching on it leads to a LEGO clone, thanks Google. Trying to explain why a homeomorphism is potentially a DRY violation is the kind of explanation that leads to MEGO.
I don't like those "hinting" questions any more, in retrospect it seems more about exploiting assumptions of superiority at being obscure rather than a valid assessment of suitability. Also seems to waste precious time (interviews cost time, more valuable than money). Better to be direct and practical, principally because misunderstanding is more likely the default state of human transactions. It is probably better to first settle on a mutual understanding of some expected conceptual requirement (e.g. understanding of DRY), then produce some specific examples for the interviewee to judge and interpret. Since these are mostly heuristics, judgement with respect to application of heuristics is the key facility I would attempt to elicit, InMyHumbleOpinion (no more TLA, CamelCase from now on!).
In short, the reasons for the principle matter, but if you know to look for and understand the reasons, the principles themselves are obvious and do not serve as useful heuristics.
Principles and concepts: The reason for naming things is to be able to efficiently communicate things, make them memorable, etc? For example when communicating things in an interview? Why do we name things generally? Empire State Building? Or just the building on Xth street/avenue in NYC. Then of course, if everyone is misinterpreting the Empire State building for the Statue of Liberty then either the naming was off, or the teachings of the name....
A senior dev was creating an admin tool that took 7 or so different things, and treated them the same. They were very similar in how they operated, but there was some complexity.
Cue two months of MTTF never changing; squash a bug, introduce a new one.
I, as a junior, opted to rewrite everything. Got manager buy in. Did so, largely just separating these areas, treating them as unique things. Voila. MTTF started dropping.
Everyone agrees if it's the same thing, don't repeat yourself; no one thinks repeating themselves is a virtue. So once you understand that it's worth trying to find the things that are the same bits of context, knowledge, etc, as part of basic understanding of the domain, you understand the reasons for DRY, and the phrase ceases to need ever be uttered (and in fact, becomes counter productive, because it is to try and say "best practice would be to not repeat this!" without actually addressing the real issue at stake; are these the same thing?)
Wait another 10-20 years and FP will suffer the same fate.
We just like mental shortcuts. We ask should we use SOLID/whatever, instead of asking why was SOLID/whatever used and does this why apply to us.
I don't know - some ideas just turn out to be completely mistaken and any success while using them to be actually down to something else entirely.
I haven't really seen a strong push against unit tests?
Regarding a push against unit tests, the Frontend world, for what its worth, has a rising school of thought that favours integration tests based around what the user sees and interacts with.
You’ve never seen like a User class that only encapsulates fields and methods relating to a User abstraction?
Here is Bob responding to OPs link: http://blog.cleancoder.com/uncle-bob/2020/10/18/Solid-Releva...
Stating:
“SRP) The Single Responsibility Principle.
Gather together the things that change for the same reasons. Separate things that change for different reasons.”
That’s a pretty clear delineation IMO.
The want for change part is hard thing to define. Personally, I much prefer "separation of concerns" when deciding what to place where. I guess you could argue in some sense its around the same thing shrug.
I agree with that push, as long as there's some other way of validating requirements. In my team, that's done with integration tests and our core principle is: code that is integration tested does not require unit testing. That principle surprisingly covers a good 70-75% of the codebase, leaving us with a few core unit tests for the secret sauce of our product.
There is nothing particularly magical - one assumes that the top 10% of any MIT graduating class could be taught to do the same over a decade or less.
But I am guessing that it needed a organisational effort to ensure the bits added never got beyond one person, and that he was comfortable saying "stop".
Every railway and new or refurbished bridge needs to be signed off by a railway engineer - who can and should refuse if it is sub-standard. Keeping our software under the same degree of control will need similar levels of handing the keys of the kingdom over to a small number of devleads. And ensuring they do not encounter unmanageable conflicts of interest.
Thus it is unfair for us to try and contradict his arguments, it's like talking to a conspiracy theorist, they will glue random facts trying to support a pre-decided conclusion.
Having said that, this post is very shallow. Even if there is merit to criticising the SOLID principles (and I would be very interested in such a criticism), you can't just do it an a couple of sentences.
But also you seem to be denying two important possibilities: 1. Something can be both flippant and insightful. 2. A "couple of sentences" is sometimes all that is needed to trigger a light-bulb moment in someone else's head.
We're not writing legal treatices here. We're trying to transfer insight from one conscious brain to another using an imperfect medium.
If that is the context and tone, then I must admit I missed it.
> But also you seem to be denying two important possibilities
I don't disagree this can happen. Let's see what the future will bring. Still I feel the writer could have done a better job going into more detail.
He said something like "Why do all the game developers write code like that?".
He handed me Uncle Bob's Clean Architecture. Uncle Bob is something of a meme to me. I read the book and couldn't figure out how to explain how I thought this style was inferior to what I had done in games before.
Ive been thinking about that ever since it happened. How could I explain to this guy my programming philosophy?
Last night, back in games, after refactoring the same piece of layout code for several hours, I finally arrived at the irreducible, simple, clean result that I wanted. I slotted it into our code base, and mentally noted how I would onboard the team to it on Monday, and how they would instantly understand because the resultant code is so simple. Then it came to me:
"Unit Tests are like pouring concrete on your code base."
Same goes for using "the wrong abstraction". Sometimes you want concrete to encase your precious technology. The problem is when people are writing sub par code, then they pour all this concrete on it via Unit Tests, Inheritance and Abstraction Patterns. It can become harder to improve the core logic of the system if it's distributed across many files, and enshrined in Unit Tests.
I've dedicated over 10 years to coding, and only now do I think I can write clean code that is worth encasing in concrete. I think there might be a general flaw in the idea that everyone should be writing tests. I'm not certain about this, it's just my general intuition based on my experience.
They are pros and cons to every design decision. I acknowledge that they are many pros to these kinds of designs, I'm just enumerating some cons that have arisen from my experience.
My experience is the exact opposite. Tests make me feel safe to perform code changes a bit more aggressively than simple refactors. If behavior coverage is slow, I spend more time double checking I'm not breaking stuff along the way.
Sometimes changing implementation will mean changing tests too which might seem to be defeating the point, but this is because the test isn't good yet so it's an opportunity to improve the test and stop leaking implementation details. To me, it pays off handsomely in the long run.
If you only write tests of the higher abstracted units while also using the lower abstraction ones, you are not really doing unit testing anymore but what some people call integration testing.
In my opinion they both have their place. Most of the code we write is relatively throw-away code that will be used in one project/occasion and is prone to be changed because of changing requirements. You should test your requirements and functionality with integration tests, unit tests will cost too much time and will stand in your way. In rare cases we write code that is really reusable in a bigger scope and acts like a foundation set in stone that rarely changes and is widely reused. In general you should put way more effort in those and also use unit tests there.
If they're very closely coupled to your code, yes.
I'll just say there's a reason why they're called "unit tests" not "class tests": because a "unit" is not necessarily a "class". You get to choose how big a unit is, and if it impedes change - like pouring concrete - choose differently.
I've always felt that "good code" is far superior to mediocre code with tests. I'm sure a lot of this is because I've never been impressed with the "testing culture" at any place where I have worked, it's always just a tacked on bullet point on some sprint board check list.
I guess for me, I feel like I have developed an intuitive sense for good code that has mostly come through real, lived, expensive, mistakes. I know it makes no sense as a programmer philosophy, but I truly think I care way more about every line of code because I have no unit tests. There is no safety net. I've been writing code like this for a very long time, and I think I've developed a certain skill that has been empowered by these high stakes. Pushing bad code and having GitHub tell you about a failed test before it ever hits prod is not nearly as powerful of a feedback loop as watching a crash rate sky rocket, having the Google Play store ranking tank and ultimately cost an unknown, large amount of revenue. I'll never forget that merge.
I guess I'm not against "unit tests", I'm just speaking about the kinds of lessons I've had and how they influence literally every line of code I write or read. Ultimately, yes, I've made big mistakes, but at this point I think I write better code, at a faster pace, than my peers who haven't had these "trial by fire" experiences. I know it's not for everyone, but I feel as though it has made me stronger.
Yes. Let me say that the conclusion that reached above about "unit != class" is one that I was forced to reach recently, by team members assuming the opposite.
In a lot of shops, the tests are done by rote, doing the opposite of what I said above: closely-coupled class tests on each public method. "pouring concrete" is a good description of the outcome.
stopped reading after that.
Fitting your code around a load of "best practices" will basically always increase complexity because you've expanded the list of requirements the code has to conform to.
And how does fitting around practices necessarily increase complexity? If you’re a jr engineer used to write a mess of code to solve a problem, and have to decompose it following a set of principles, you may end up with something SIMPLER very easily.
The whole point about principles is they lead to better outcomes. Problems arise when you confuse them with “rules that can’t be broken” or (as in this post’s case) just refuse to understand them and come up with your own half-assed heuristics - either to justify your code, for lack of experience, or just... to talk at a conference :-)
The article does not present an alternative to SOLID, that's the next article.
The analogy to accounting is still useful and I believe correct. Assets depreciate (lose their value) and have carrying costs (storage, insurance, interest etc) that are booked as expenses.
Put another way, a phrase like "all existing code incurs costs" is an accurate analogy. So too is "all code is an asset", because it implies the former.
Of course. That's true of many assets, usually this is called the "replacement value" of the asset. You use replacement value and depreciation to calculate whether replacement is the best option, or whether to extend the life of the asset you already have.
Tockey's Return on Software is a fairly readable guide to some of these topics: https://www.amazon.com/Return-Software-Maximizing-Your-Inves...
At a more personal level, the DIP section of this article hit close for me, as I've (again) lived the described scenarios a mere week ago. Sometimes `main()` is all you need and all you'll ever need. The extra tooling is also debt. If you choose to incur it, it must palliate to an actual problem that you actually have, else YAGNI.
> When I look at SOLID, I see a mix of things that were once good advice, patterns that apply in a context, and advice that is easy to misapply. I wouldn’t offer any of it as context-free advice to new programmers.
Off course! I wouldn't offer any OOP architectural/programming principle as context-free advice to new programmers. It's easy to misinterpret it if you don't contextualize it, show how it's used in real projects, discuss advantages and disadvantages, and so forth.
> The Single Responsibility Principle says that code should only do one thing. Another framing is that it should have “one reason to change”. I called this the “Pointlessly Vague Principle”. What is one thing anyway?
This gives me the impression that the author indeed misinterpreted this principle, just to name one, oversimplifying it. If you like the author have trouble understanding the Single Responsibility principle, take a look at "On the Criteria To Be Used in Decomposing Systems into Modules" by David Parnas [1].
It seems to me that the author also oversimplified the other principles as well, maybe on purpose, for using controversy to attract attention to his post.
[1] https://www.win.tue.nl/~wstomv/edu/2ip30/references/criteria...
The criticism of SRP hits home for me - I try and write SOLID code when I can and SRP is challenging to follow. I struggle jumping between "Does this really fit SRP?" and "I can probably justify SRP here." I usually settle on the author's "Fits my head" and justify that when doing code reviews with my co-workers.
The author's comments on OCP don't accurately reflect my experience with the field of modern software development. I've worked on many projects (even green-field projects in the last 3 years) that have been very expensive and risky to change, where work absolutely needed to be additive because you couldn't trust refactoring tools to handle everything that needed to be changed.
Single Responsibility, honestly I like 'separation of concerns' much more. People tend to think single responsibility means use tiny classes and single line functions. Open closed principle stating "should be open for extension, but closed for modification" infers to many it's about inheritance only. Open for customizability would be much better. Liskov Substitutions seems like its talking about inheritance when it really also applies to interface usage in general. Dependency Inversion principle is interpreted by many to be use dependency injection everywhere etc...
I wish someone would go and update these definitions for the modern world.
Is Liskov's substitution principal really one of the 5 most important pillars of software design, or is it just the best one that starts with an L?
The example given by OP is about ETL. Is that one thing or more than one thing? It depends on the abstraction level. If you are talking about a data access object, sure let it do all those things because when considered on that abstraction layer it does one thing, access data.
But if you are looking at individual functions or methods, that is one abstraction layer further down, then S in SOLID says you should not mix extraction and transformation, for example.
It all depends on context and SOLID are principles not law.
Our head may very well fit two pieces of code which are disparate and which would be use to separate into distinct functions.
----
"Open-Closed Principle... was sage advice in an age where ... we hadn’t figured out refactoring yet ..."
Well, as far as I'm concerned, we haven't "figured out refactoring" yet. I'm currently working with a group of people 75% of whose' work is just gradually and slowly refactoring an existing codebase.
"Nowadays, the equivalent advice if you need code to do something else is: Change the code to make it do something else! It sounds trite, but we think of code as malleable now like clay"
That may be true when you're writing code for your own use, or the use of a very small group of people (the large "user base" doesn't work with the code, they interact with what the code does). When you're writing a library, or code with which near-strangers need to work, you can't just willy-nilly change things. Moreover, you must strive to make your code good enough so that it doesn't need to be changed all the time; an ever-changing codebase is difficult to rely on (unless somebody takes a snapshot, and is back in the case of never-changing code).
---
... but I do appreciate some of the points made. Defining what "one thing" means is definitely not trivial and obvious, for example.
99% of developers are working in boring business software with 2-3 other developers.
99% of codebases will be touched by maybe a dozen developers in its entire lifetime.
Except it doesn’t really say that. Without understanding the S, the OLID is bound to seem like mindless pedantry.
They are reasonable principles and often do apply, but there are times when the concept of Locality of Behaviour is better for overall system simplicity and stability:
You cannot tell a junior developer to just “write simple code” and then expect any kind of sensible result. But then, you also cannot tell them to write SOLID code. Indeed, you need to teach the basics first: How to understand the requirements. How to write any code that fulfills them. How to write legible code that fulfills them. How to write good code that fulfills them.
If you dump these principles onto developers that have not yet learned the proper developer mindset, the outcome will most likely be disaster. It is very hard to repair the damage.
Only after a developer can actually write good code can you teach them how to take it to the next level. How some libraries are just so awesome to use without having to check the docs. How to conceive code you’ll look at in 6 month and think “damn, that’s some shit”.
Why every single clean code principle is a recipe for disaster – if told to juniors.
Regarding SOLID principles being outdated, I disagree. The high level concepts of SOLID are good and just as relevant as ever. While the author proposes that code is now less risky to change, I would argue that the asynchronous, distributed systems that we often work with today are significantly more complex and error prone. The world also got hooked on dynamic languages for the past couple of decades, although it feels like that shift is slowly reversing. Software bugs have become a routine part of daily life, partly because there's more software but also because we build that software more haphazardly to achieve deadlines for ever expanding sets of features. Today, a large number of developers lack proper education or training and have relatively little experience. We work in teams where there's a lot going on besides writing code and there's more distraction around us than ever before. To me, there's no shortage of reasons to worry about changing a line of code. I spend a lot of my time trying to understand and reduce that risk so that I and everyone around me can be more productive in the long run. The SOLID principles are a useful guide in that effort.
That said, you may want to update your interpretation of SOLID within the context of your latest coding practices. For me, that meant applying SOLID to functional programming. At first this seems odd, since much of the language around SOLID implies the use of classes and object-oriented programming. But it turns out there is a lot of overlap between the philosophies of SOLID and functional programming. [1]
1: https://dev.to/patferraggi/do-the-solid-principles-apply-to-...
> "... So what would I do instead? I thought there might be a one-to-one correspondence for each of the SOLID principles and patterns, since there is nothing inherently bad or wrong with any of them, but as the saying goes, “If I were going to Dublin, I wouldn’t start from here.” "
As anything, SOLID is a guideline, not a dogma. The purpose is to encourage developers to approach the design of their code mindfully.
Personally, in my memory the advent of SOLID principles coincided with widespread promotion of better approaches to testing. These principles indeed helped design for better testing. And better testing provided more freedom for changes going forward.
If another set of priciples help one's projects better attain the set goals, well, sure, embrace these by any means.
There is definitively room for new models and new paradigms when writing code, and I welcome that. The click-bait article is just that... click-bait. It silly and reminds me of the "climb that tree" quote.
Consistency has always been a big deal for me. Different code-bases needs different principles, standards and models. If you are consistently wrong you can consistently fix your issues.
My experience is that if you are inconsistent, you usually will end up with the dreadful spaghetti of things.
As with all things, there is a cost (slightly more code duplication for rarely-solved problems) but also huge benefits in having really solid abstractions built from lots of experience.
SRP never really recommended how large those responsibilities should be, but that's almost impossible to do given how widely programming is applied (especially for a quick acronym parroted in interviews). Had it tried to specify this size, we would likely be dealing with "OLID" instead of "SOLID", since it would be much easier to criticize.
It comes down to how many tabs you want opened in your IDE and how good you are at moving between them and keeping the object relationships in your head. If I have a god object to handle one responsibility, I only have to look at one file, but it might be thousands of lines long. Or I can reduce that down to tens of files open with hundreds of lines each. Or it could be hundreds of files with only around ten lines each. The god object allows me to mostly forget about any object relationships or abstractions. But I pay some price by having to learn the entire god object before being able to sensibly work on it.
If I have some object inheritance like:
IPipeline -> AbstractPipeline -> SqlPipeline -> PostgresPipeline, SQLServerPipeline, MySqlPipeline
IPipeline -> AbstractPipeline -> NoSqlPipeline -> MongoPipeline, CosmosDBPipeline
(every architect dreams of this)
It's easier to change things per database without affecting the others. Since each of these products change independently of one another (Mongo doesn't consult Microsoft about SQL Server), it makes more sense to do it this way as opposed to having a god object when you know you must support these other databases.
On the other hand, if you know you will only ever use Postgres, IPipeline, AbstractPipeline, SqlPipeline/NoSqlPipeline become mostly useless abstractions.
This also varies greatly whether you're trying to build building blocks for other developers or just trying to ship a product. Principles aren't always going to apply to both of those, and SRP might be one of them. It wouldn't hurt to add "depending on how your team defines responsibilities" at the end of SRP.
> The Single Responsibility Principle says that code should only do one thing. Another framing is that it should have “one reason to change”. I called this the “Pointlessly Vague Principle”. What is one thing anyway? [...]
If a change request comes in to change the format of the logs to include milliseconds and you're running a search and replace over the code base, you've broken the SRP. We can argue that is a good thing or a bad thing, but that's what it means. I think it's a bit of both, a trade-off, like most things.
But overall I think the article (if we were to take it seriously) basically represents a nihilistic point of view that says "there's no point trying to have principles - code is either subjectively good or it isn't".
I'm as sympathetic to heterodox criticism as anyone, but this desire to throw away any received wisdom at all definitely seems misguided to me.
(For what it's worth, I think the single responsibility principle and the open closed principle are particularly valuable. I'm open to persuasion on the others)
I very much like the “fits in your head metric” though and I wrote about it at length here:
Now that that original sin has been committed, it is super easy to take shots at it.
Better code is the one that takes lowest time in total to get from point 1 to point N by mediocre developer and in the same time result in lowest amount of help desk tickets
Its the essence of MIT vs New Jersey design style (i.e. worse is better)
The problem is, its impossible to measure that (unless we tap in parallels universes) which is why design is a form of art in real world.
https://en.wikipedia.org/wiki/SOLID
As a pure mathematician, whenever I find myself reading an acronym-heavy paper I know I'm in the wrong part of town.
What you have is a series of anecdotes from people pushing particular strategies saying that they think the thing they're saying is a good idea and other things are a bad idea. That doesn't tell you very much about whether the code worked in context, especially on teams of large numbers of developers. It doesn't tell you very much about whether the sample code on a blog that is (allegedly) more readable or understandable is actually more readable or understandable to others.
And many of the folks pushing strategies, giving talks, etc., seem to be better known for being consultants about how to program than for programming per se. I have, I think, access to more empirical evidence about good and bad programming practice than any blog or talk that a SOLID advocate has given - but it's all internal to my employer's source control repo and you can't see it. If I could, I could easily point to code that has tried to follow "proper" design and has turned into Enterprise FizzBuzz and I could also point to short, readable code that doesn't follow any of the SOLID principles but it doesn't matter because you can read the entire thing in 10 minutes.
Now I admit that I'm not offering evidence either, and as much as I agree with this article, it's also a first-principles argument. It says, for instance, that in our modern era recompiling a system from scratch is cheap and you don't need the open-closed principle, which I strongly agree with, but I can't immediately point at an example of a system that followed the open-closed principle and accreted into cruft, nor at a system that intentionally keeps its internals easy to edit and has worked well.
I think we ought to do something about it as an industry. I don't have great ideas about how. I suppose we could look at open-source code (does Git follow SOLID? Chrome? TensorFlow? OpenStack? what about systems that do the same thing but with very different architectures, like GCC vs. LLVM?). But open-source code is generally subject to different development pressures and expectations than internal code, and the question many of us are trying to answer is, how do we write code at our day jobs that works well?
All that said, I will say this article matches my (anecdotal) experience - and I would go further and agree with the comments that you want to test your system in its entirety, including running the actual database in tests instead of swapping it out. Just like it's much easier to recompile something now than it was in the '90s, it's much easier to spin up a database in a cloud VM/container/etc. than it was in the '90s. Dependency inversion and injection is a workaround for environments where unit tests are easy but integration tests are hard, and making integration tests easy both solves this and has a range of other benefits for software quality (and development velocity) too.
(And actually there's a potential piece of empirical evidence here: software that doesn't have per-unit licensing costs, either open-source software with no costs or cloud SaaS with continuous billing based on usage, has been beating traditional enterprise licensing. I think you could argue that a big reason for that is that you can spin up that software in a CI environment or on a developer's machine. I can run a local MySQL much more easily than I can run a local Oracle DB, so I'm much more likely to do it, and we're much more likely to go to prod with MySQL, in turn.)
But good practices in a craft don't necessarily need to be quantifiable to be useful. If you learn a group of concepts on how to improve your work, you can ask yourself: "Is any of the SOLID principles applicable here?" and "Is the resulting code better if I apply them?"
If you answer "yes" to both questions, you're using the principles in practice to improve the code without having empirical proof that they worked, just gut instinct.
The underlying problem, I think, is that design is a human problem, not a technical one. Design is about making code is to understand and modify. How can its quality be measured?
Without an objective criteria for design quality (and stuff like cyclomatic complexity are far too simplistic) then no study of design principles is possible.
"Join us next week when I will introduce a new acronym, whose interpretation will be superior to this interpretation of SOLID".
Is SOLID any good? Who knows? Who proved it? Heck, who even did tests?
But, I don't think this author gets it (what the SOLID elements actually are, nor how they are moot when using languages that solve the same issues they are solving).
sadly, he fails miserably at it.
Writing simple code is great. For when you can write simple code. For everything else, people invented things like SOLID. We need and want to do complicated things and maintain complicated codebases. If you want to do simple things simply, I'm not even sure OOP is a great model, let alone SOLID, which is an attempt at simplifying people's OO constructions. For all intents and purposes, SOLID is how, in fact, you write simple code in complicated projects.
"Single responsibility" is a much better heuristic than "fits in my head" because code expands and gets bigger. It fits in your head now, but no decently sized, actively developed project stays simple. When do you break it up? When it starts taking on multiple responsibilities. Yes, that isn't objective. But the alternative proposed is even less objective. Additionally, "fits in your head" is inferior because it assumes you already understand the codebase. If you are new to the code base, trying to weave through the functions and objects called to figure out what it does, looking at code that does multiple things is confusing and hard to synthesize without first understanding the whole. And you can't understand the whole until you understand the pieces. Single responsibility is much more specific and identifiable than the author's alternative.
The author mis-attributes the Open-Closed principle to a performance optimization. The open-closed principle allows me to make assumptions about what a class is doing and how it behaves. This is because I know it hasn't been modified. Wikipedia notes that the original formulation of the term, a closed class or module "has been given a well-defined, stable description." The point here is that if you don't let something be fundamentally changed during runtime, then you can reason about it. It is definitely not a performance hack.
The author states that the Liskov Substitution Principle is the principle of least surprise, which is surprising as you cannot substitute one principle for the other. This suggests to me that the author does not actually understand the meaning of this principle, especially as he fails to give a substantive critique and unhelpfully declares that you should write simple code. The Liskov Substitution principle is, again, how we reduce the apparent complexity of code so it allows us to reason about it more effectively.
This next one is baffling. The author of this article recounts the anecdote showing an instance where the interface segregation principle would have been helpful in designing the code. However, the author appears to be unable to gleam from that anecdote the underlying principle being advocated and is fixated on the fact that in the story the code was refactored. Since the anecdote describes an implementation, the author is unable to see it as anything other than a design pattern.
The author claims this principle has cost billions of dollars in damage. He does not feel the need to explain this or provide a citation. Just, you know, billions of dollars in damage. Like we just all know that this powerful decoupling principle just set a large fortune on fire and this is plainly obvious to everyone. Next he says "the real principle here is option inversion." No, it isn't. I don't know where the author got this from. But I suspect given what he's said previously that it just sort of made sense to him and then decided that must be what this principle is about without any further discussion. But the author is none the less actually correct that you can over-use this principle to over-complicate code before it really needs to be that complicated. These are, of course, ultimately rules of thumb that work well together. Common sense and experience are still required.
Robert C. Martin has an unusual habit of giving advice that sounds kind of stupid at first glance, until you actually try it out for yourself. His "rules" often have hidden reasons behind them that only become clear when you write code with them. I suspect that the author did not give SOLID a serious shake as his criticisms all sound strongly like someone who read about SOLID, failed to internalize their meaning or give the technique a try, and is now attacking a strawman with advice that, quite unlike Robert C. Martin's style, sounds okay at first glance, but you quickly realize does not translate into actionable advice when you're coding. I've heard "fits in your head" before. It sounds like what you want, but it does not allow you to identify problems early and predict what sorts of changes you might need to make to the code to prevent those problems. Especially if you're a beginner, try writing according to SOLID principles. See if it helps. After you understand why those are the principles and why they all go together by seeing what sorts of decisions it causes you to make in your own code, you're free to ignore it having at least learned what the advice actually is.