A good developer has a natural, almost visceral aversion to complexity
sigpwned.com
sigpwned.com
Moseley and Marks disagree with his premise in the paper Out of the Tarpit [2]. They propose an approach based on functional and relation model to minimize accidental complexity.
Even if you don't agree with the conclusions or some of the premises in this papers, both provide an excellent read on the subject.
[1] http://www.cs.nott.ac.uk/~cah/G51ISS/Documents/NoSilverBulle...
-- Brian Kernighan of "K&R C" and AWK fame
If you're just quasi-trolling/shitposting, please go back to Reddit.
"Magic" is definitely a problem when it comes to debugging, but eliminating magic isn't a silver bullet for making code easier to reason about, particularly when there's errors. Rails still, almost without exception, throws obscure errors that give no indication to what actually may have gone wrong, and there's still quite a lot of magic left that can behave in odd ways in ActiveRecord.
The real interesting question is not whether you should strive to make your code complex (clearly not) but rather what "less complex" actually entails. Does this mean less moving parts? More parts that are simpler? More verbose code that is more explicit? Less verbose code that is easier to read at a glance? More math? Less math? Code that looks uniform? Code specialized to the given domain (and therefore less uniform)?
I have strong opinions on these questions, but I have no answers. I've met others with strong opinions exactly the opposite of mine. They don't have the answers either. It is, ultimately, not an easy question.
I once inherited a module written by a more junior developer. He indiscriminately used several dozen global variables that were used and modified in various parts of the program without much of a system. I spent about two months trying to understand how the module worked without much success. Ultimately, I had to completely re-write it (with great difficulty) to get rid of global states, and bring the number of entities affected by each function to under five-to-sever.
This biological limitation affects our development tools. That's why we have object-oriented programming. The goal is to keep the number of items we have to deal with at any moment under that magical limit of five to seven. Incapsulation is a key component of OOP used for this purpose. Without it all will be lost.
Sometimes I think of an alien race capable of keeping track of 100 items simultaneously, and try to imagine what their programming languages look like.. :)
The real reason for such limitation is that number of possible connections between objects is growing very quickly.
For example, 2 entities connect with each other by 1 connection.
3 entities connect by 3 connections.
4 entities connect by (4 * (4-1)) / 2 = 6 connections
5 entities -- 10 connections.
7 entities -- 21 connections.
10 entities -- 45 connections (4.5 times higher than for 5 entities).
Number of possible connections between entities defines complexity of the model.
Of course the specific number is highly affected by human biology.
Especially given gorelik's observation that complexity really derives from the combinatorial explosion of connections, are objects with internal state the best way to manage that complexity? If every call of a public method from another object is a connection in the graph our entities are an (object, message) pair, and a situation involving 3 objects with 3 public methods each exceeds our limitations.
Contrast this with a pure-functional approach. If you can express your algorithm with immutable data and operations that create new transformations of that data, you could (hypothetically) have fewer things in mind at any one time: the data being operated on, and the single operation being performed. It doesn't break down quite so cleanly in reality, but use of pure functions where possible largely obviates the benefits of state encapsulation as provided by most modern OO languages.
OOP (when done properly) enables us to focus at any time on a sub-part of the system containing five entities or less, so that we can understand that particular slice of the system. The slice can be very low-level (a private helper function of the smallest class), or very high-level (invoking a large subsystem through a facade interface, http://en.wikipedia.org/wiki/Facade_pattern). A typical REST API would be a good example of the latter.
I like benefits of the functional programming too, but I think functional and OOP work together rather than being in contradiction to each other.
Oh absolutely, I was responding to the statement of "all being lost" without OO more than making a serious argument for the opposite extreme ;)
...
"I like benefits of the functional programming too, but ..."
Your response makes it sound as though you believe OOP is the magic that allows encapsulation or modularization.
Functional programming also enables us to focus at any time on a sub-part of the system. You don't need OOP for that. In fact, you don't even need functional programming for that. For example, you could get by with nothing more than subroutines and functions (whether they be first class or not).
But there are many apps that have complex internal states. Subroutines and functions don't say anything about how to deal with it. This state ends up being global variables, which is a big issue.
I once worked on a complex desktop app written with functions and global data. It was nearly impossible to modify it, or understand what it does. A click on a button could execute 1,000 lines of code modifying dozens of variables, resulting in effects throughout the application. Debugging a seemingly simple issue could take an inordinate amount of time. We had to re-factor the entire app into OOP and model-view design to stabilize it and move forward.
The app had about 30,000 lines of code. So, based on this experience, I'd say it's too large for the functional style. When I start a new project I often don't know how large it'll get, so the safest approach is to default to OOP. If the functional approach will prove more appropriate down the road, it should be relatively easy to integrate functional design in those parts of the app where it's required.
In certain cases, you may be able to reduce the size of working memory through recursion or related techniques: "The red-black tree T without node X is the subtree on the left without node X if ... possibly rotated based on the color of the top node in subtree T' ..."
But sometimes, it really is simpler to just do things imperatively with mutable state: "First recurse into subtrees until you find node X. Then, unlink it from its parent. ... Rotate if its color is red. Repeat with parent node ..."
There are many things that are strictly simpler with mutable state. Most prominently I think are hash tables, but there are others. Not all programs are best-described as a composition of maps, folds, and filters.
It's also worth noting that even if you believe you know how to improve/simplify/whatever, it may be worse than you think and sometimes you don't figure out the extent of the complexity until days have gone by.
This, like many things, is a trade-off and not a set rule. In brand new code, by all means make it small and pretty and documented and whatever else would win it an award; but in established code bases be smart about what you "fix".
It was hard to criticize, though, because he pretty much single-handedly got the company off the ground by quickly implementing features people were willing to pay for. This code was good enough to get the company up to $2 million annually.
The downside is we spent about a year re-writing the software to get it to a state where the company could scale to higher levels. During that year we had to stop adding new features.
I remember getting very frustrated with the code quality, but I often wondered whether that was just a price we had to pay to get cash flow positive.
Dirty code not always has to be refactored. Most of dirty code just dies together with useless functionality (wrong guess on business side).
— David Gelernter
This resonates with me. The article makes a fine point about removing unnecessary complexity, but in the real world you can't be perfect in that regard - not for a large system anyways. So maybe we should allow a little more complex code if we can use the beauty of the product to hide it.
For example, if I had in some way, the catalog of all the possible solutions to problem X ready in my mind, I'd pick the most elegant one. Not necessarily the shortest, but the most elegant, the one where you can spot the _solution's outlines_ easier.
I think seeing and appreciating this outlines would make you a connoisseur of fine software. You would strive for beauty and elegance, and appart yourself from unnecessary complexity.
There, fixed that for you.
'Make it as simple as possible, but not simpler' (Einstein, as quoted by Niklaus Wirth.
Amusingly you seem to support the author's argument that running around making changes when you don't really understand whats going on leads to lower quality code.
I actually use that exact Einstein quotation in the article, attributed to the big man himself. One of my favorites.
You can have a simple library or util module with the standard function API instead of building a DSL. They work just as well in many designs.
It is a tradeoff when deciding to use DSL or not.
I guess what Im saying is that the programmer need some help from the language to make the software non-complex. Even trivial things often becomes complex in low level languages.
When you have a large project I will always take the compile time checking that tends to resolve a lot of bugs over unit tests to verify the same thing. Especially when it comes to large re-factoring efforts.
An old hand programmer once told me he could figure out whatever social/psychological pathologies from the code of the programming shop that wrote it.
Lots of musicians go through a stage where they are trying to see what they are capable of and make music as involved/complicated/ornate as possible. Many of them mature into musicians who try to find a few elements that work together.
There (probably) are areas at the extreme ends of the spectrum where things are too complex/simple, but there's a big range in between that might do it for certain people. I don't believe that just liking some form of complexity is objectively bad. It's not all just complexity for complexity's sake.
In terms of artistic works that people still actively experience, the tendency is for the classics to be simpler works. Complex works tend to be classics in the Mark Twain sense.
...does that make King Crimson listeners bad people?...I don't believe that just liking some form of complexity is objectively bad. It's not all just complexity for complexity's sake.
I don't believe that just liking some form of complexity is objectively bad either, in general. In the case of coding projects, though, I have to note that the scarcest bandwidth is often reducing team lead bandwidth. Exactly what this means is highly contextual. The tricky part here is in the "but no simpler" phrase of the Einstein quote.
You might say that I like minimalism because I'm lazy. Minimalist cuisine means fewer ingredients to prep, less cleaning later. Minimalist syntax means fewer things to keep track of and lower barriers to tool-smithing. I don't think that makes me a bad person either. The question to ask is, what is the cost/benefit now, and what's the cost of changing your mind? Sometimes the more complicated system reduces the latter cost. YMMV
It's jam sessions all over again. Much easier if the basics of the musical form aren't too complicated (e.g. 12 bar blues) or everybody knows the same songs already (e.g. Irish folk). Getting a random bunch of people together for some Free Jazz is a bit harder.
Perhaps the musical analogies just break down here, I'm not sure that they are helpful...
But yeah, the analogy is easy to abuse, due to the multitude of artistic styles, and active rebellion against the "mainstream". For every Piet Mondrian out there, you can find a Chuck Close.
This brings out a problem with the OP and the discussion. We readily dismiss things as complicated that are merely unfamiliar. The converse is also true: we don't see complexity once we've become habituated to it. And we become habituated surprisingly quickly.
So how do we distinguish between complexity and unfamiliarity? Without some reliable distinction, this is all in the eye of the beholder.
There are some cases where you can, for example, radically simplify a complex algorithm by using an unfamiliar toolkit (Feynman diagrams, Lisp, etc.) but there are also cases where a solution is obviously more complex than it needs to be even within its existing framework.
Most judgments of complexity we currently make have a lot to do with familiarity. (Is the 40-line spreadsheet program at http://www.nsl.com/k/s/s.k complicated or simple?) You assume something similar when you say "obviously more complex than it needs to be". When is complexity obvious? When it exceeds what we habitually consider ok.
What we really need is an objective measurement. Do we have one? We sort of do, insofar as the research on this seems to point in one direction: shorter programs are simpler. That doesn't answer every question - you still need a way of measuring program length and defining what a program is - but it's the beginning of an objective answer.
What's harder is to evaluate the effect of adopting a certain pattern or framework will have on the cost of future refactorings. I don't think people do even a back of the envelope estimation of even just the first number. The number of statements put in can probably be used to estimate the cost in man hours.
Here's an overview of the spreadsheet program that's implemented and a walk through a bit of the code:
I was also reminded of an old a comment from Guy Steele: http://www.youtube.com/watch?v=4LG-RtcSYUQ&t=1h21m11s
- you might not have seen enough cases be able to characterise the true problem, e.g. in some situation a constraint exists, but you've only seen one instance of that situation
- reusing standardized components can enable you to solve the problem more quickly (even though one-size-fits-all doesn't really fit)
- Kolmogorov complexity doesn't account for efficiency, space nor time, only that the eventual answer is correct. Efficiency often requires ugly hacks
I would put an aversion to complexity in the same box as an attraction to it. I.e. atheists are fundamental in the same way that christians are. Agnostics know what's up.
Refactoring: Improving the Design of Existing Code http://www.amazon.com/Refactoring-Improving-Design-Existing-...
Disclaimer: I'm the author of the post, so that's not actually a different recommendation, just a more elaborate one. :)
I think education in general rewards complexity, getting bonuses in English class for using long passage of flowery words with many synonyms or extra credit in CS for adding 200 lines but reducing O(n^2) to O(n log n).
That depends on who your teacher is. My teacher taught that good english should be understood by as many people as possible while retaining all of its meaning. The goal of English is to communicate.
He was a good teacher because the top 65% of our class got into the top 35% of the state for English, while our school was below average in just about every other subject.
Now I focus on simple solutions. That is why I've come to enjoy Ruby.
Note that during refactoring you will have twice the logic, which is the worst state, so you can't stop in the middle and say you'll finish later.
Introvert != Misanthrope.
Maybe "trivial" is good.
If you can "write something in a weekend", it's easy to maintain. If you lose the code, you can rewrite it. And maybe it will be better the second time.
When a project becomes so complex you cannot easily rewrite it, is that a good thing?
No, you can't really. You can just say "simplify" but it does not convey the same message.
That is, it conveys a suggestion to simplify (the first part) but it doesn't convey that simplification has so extreme importance that it's worth to repeat the message (the second part).
Fuck, if I have to be a pedant, I'd better go all the way: well, they might be, but
1) they don't use * as the multiplication symbol, but ~ "x".
2) they don't multiply words and numbers... Repeating a string this way is something programmers do.
I don't think anyone will have difficulty grasping that 'Simplify Simplify Simplify' and 'Simplify x 3' can be considered equal.
It's not like I thought of it as a serious question, I just added it for the benefit of anyone confused by the contradiction.
"One of my most productive days was throwing away 1000 lines of code." -Ken Thompson
So what are they? Great?
Seriously, how do we describe developers who have a visceral perversion toward complexity?
There are a lot of developers who love complexity.
Regular expression complexity or AbstractSingletonFactoryImplementationBeanStoredProcedure complexity?
1) You don't get them and think they're magical
2) You learn regular expressions, and find all kinds of places to apply your knowledge of greedy modifiers and positive lookbehinds.
3) You learn enough to realize that regular expressions tend to be extremely brittle and try as hard as you can to avoid them.
There are a lot of very dangerous developers stuck in level 2.
Here's a practical example. I spent half a day coding this algorithm (ASCII string in STEP format to an array of Unicode code points) in 70 lines of C++ without regular expression. It took me 15 minutes to recode it in Perl 6 using regular expressions:
while $step.chars > 0 {
given $step {
when /^ '\\S\\' (.)/ {
@unicode.push(ISO8859toUnicode($page, 128 | $0.ord ));
}
when /^ '\\P' (.) '\\'/ {
$page = $0.ord - 'A'.ord + 1;
}
when /^ '\\X' [ '2' | '4' ] '\\' (<hexdigits>+) '\\X0\\'/ {
@unicode.push(:16($0));
}
when /^ (.)/ {
@unicode.push($0.ord);
}
}
$step = $/.postmatch;
}
This (admittedly completely untested) version is drastically shorter and much less brittle.A lot of developers seem to have a serious disdain for anything that might be construed as the "lowest common denominator". I don't get it.
Somehow I find a tremendous versatility in the LCD. It works in so many environments. I can rely on it to work, no matter what the size of the input.
But then, if you go buy a book on RE, what do you get? An explanantion of the most ridiculously complex and difficult to maintain RE. As if the author is just showing off.
Mastery of the basics is too boring I guess. Too simple.