Programmer Competency Matrix (2016)
sijinjoseph.com
sijinjoseph.com
The next improvement I can suggest is to remove the implications that knowing the FP paradigm somehow makes you a better developer, or that only a good developer can use FP. FP is a useful tool just like OOP or imperative programming. Generally speaking, a good developer should be able to compose multiple paradigms together where they fit in the problem space. Maybe if this section of the matrix were reworded to say something along the lines of "Able to use multiple paradigms in their equivalent problem space including OOP, FP, and imperitive programming to create quality software." it would be better.
I do like how the author included a section for communication skills. Too often we, as developers, forget that we are ultimately only as good as the team around us is. We should work our hardest to improve our team in order to increase the quality of our work. That starts with effective communication.
I think a bigger issue, though, is that some of these skills are very valuable, and some of these skills are the ones companies hire on, and those two sections are not always the same. Personally, I see myself as being somewhere around the n level on the algorithms stuff, but well into the log(n) level at the engineering/communication stuff. I've harped over and over on the weirdness of prioritizing theory knowledge over practical skill in day-to-day jobs where the practical is what matters - it's nice to see all the different dimensions so clearly laid out and given (roughly) equal attention.
Only in the open source world and a few SV darlings.
I am yet to do a project at a Fortune 500 among our customers that isn't based in Subversion or TFS.
I have been using Git only on personal projects.
In the last year I've built internal-only systems for two Global Fortune 100 companies, both of them in Git.
I believe you are very incorrect.
They are questioning whether that generalization follows from your observations (using observations of their own).
Writing statements about Git's adoptin on HN posts don't make them into facts, unless they are baked by actual numbers.
The fact is that both of us are wrong, because neither of us have the data of those 500 companies, only anecdotes.
That would make both of you undecided. But one of you is right and the other wrong.
> Only in the open source world and a few SV darlings.
This statement is _factually_ incorrect. None of the companies I've worked for/done work for/communicated with were in SV and _all_ of them used Git. AKA it is in fact _not_ just "a few SV darlings," because the companies I've mentioned that use Git were neither of those things.
From a technical standpoint I'm constantly amazed at the sheer diversity / museum of stuff that'll get deployed to prod in the Fortune 100. Sure, 80%+ of the stuff made in the past 15 years is all Java crud, but the nice thing about that is that dropping wars into an application container is a very solved problem. I don't have to really think during another 90s-style deployment where more difficult problems are metaphysical like if I'm actually still myself from 20 years ago dreaming very hard that I'm living 20 years in the future and this is a side-effect of my 90s frame of mind.
You'd be surprised. The vast majority of enterprise programmers have neither used it nor care to.
Write a small toy document database with an integrated ORM. Tables/documents can be modeled as objects and the query API can be built using a functional paradigm. Doesn't need to be fancy. Could be a simple key/value store really.
Or how about an HTML generation library?
Or how about a toy web server?
How about a command line argument parser for your favorite language?
How about a small library for geometric calculations?
There are many languages that have multi-paradigm support. One of my favorite multi-paradigm languages in still in it's growing phase, but it's fun to learn: https://nim-lang.org/
Some problems are just more inherently imperative and stateful. I use FP code when it makes sense, and I use imperative code when it makes sense.
There Is No Silver Bullet.
data Data = Data { thing :: Int, that :: Boolean }}
addToThing :: Int -> State Data Int
addToThing n = do
x <- get thing
modify (data -> data {thing: x + n})
return x
class Data {
constructor(
private thing: number,
private that: Number
) {}
addToThing(n: number) {
const old = this.thing
this.thing = old + n
return old
}
}
To me it doesn't really look like jumping through hoops, but we're up front about the mutation so no-one's ever going to get a surprise.
There's no silver bullet, sure. But I don't think pure FP is really all that complex. It's mostly just different.
I can't help but think that people who say this, yourself included, just don't deal with the problems that I face on a regular basis.
I write a lot of games and game engine code. I might have 60 dynamic objects in the game, each of which with its own behaviors, state and physics. Object oriented code means that the functionality for each object has both inherited and specialized behavior.
And when you're working with those dynamic objects, the "information hiding" aspect of OO programming can be very useful. Some objects have a member variable; others have an accessor that pulls the value out of a child class. I use that to good effect in my current game.
Or I might be parsing a map that I need to iterate over a rectangle in the map. But sometimes it's a hex map; ever tried to iterate over a hex map? Saying map.iterateRect(iterator, x,y,w,h) or equivalent, and not having to know whether the map is rectangular or hexagonal is very nice.
Functional programming is great when you have a lot of generic functions that you can apply to lots of different kinds of data. Object oriented programming ... is good for UI and game development, and anything else where the functions you're writing are very tailored to the data, where polymorphism can map well to the problem, and where you have lots of variations of kinds of data, but fewer (or less complex, or more specialized) things to do with the data.
None of the things you listed are particularly hard in an FP context. Classes/traits and parametric polymorphism will allow you to do these things just fine (Haskell code):
class RectIterable map where
iterateRect :: map -> (Point -> a) -> Rectangle -> [a]
// define map data structures
data TileMap = ...
data HexMap = ...
// make them instances of the RectIterable class
instance RectIterable TileMap where
iterateRect = ...
instance RectIterable HexMap where
iterateRect = ...
I don't know, but React begs to differ. A pure function (state -> UI) is basically one of the best UI programming paradigms we've come up with so far, the other probably being FRP.
React is good for certain classes of UI, granted. I use another FRP-style library (not React) to render UI in my current game, and it works well -- except that it doesn't mesh well with me trying to control it from an OO game, but I've created an interface that works.
But imagine creating Adobe Photoshop, Maya, or Microsoft Word entirely in React, and tell me again that FP would be the ideal choice.
For any particular dialog that pops up? Absolutely.
For the entire app? You'd end up with a "god object" that would make the app unmaintainable. OO gives you compartmentalization that you don't get with FP.
FP is great for problems up to a certain complexity. After that it falls apart. Thing is, 98% of apps don't come close to that level of complexity, so there are lots of people who never even work on an app that won't work well with FP.
In the RectIterable example you gave, I'm afraid I don't know enough Haskell to comment on it intelligently.
Is "instance" adding an interface to TileMap/HexMap so that either can be passed in as a parameter that requires a RectIterable? Can other interfaces be added that way to TileMap/HexMap?
In my editor, can I say something like myTileMap.iterateRect after doing this, or do I need to know that iterateRect exists and remember exactly what it's called?
The last point is salient: Discoverability is a huge part of the UI of a programming language, and memorizing all of the exact function names that are appropriate for a data structure is not fun.
Well, not quite; take a look at the Elm architecture [0]. It's basically what Redux gives you, but more explicit.
The idea is, that yes, while you have a god object in the top-most component, child components don't know anything about it; they operate on their piece of state completely independently. This makes UI elements perfectly composable and reusable, since in the end, a UI element is just a pure function and nothing more.
Not to mention other nice side-effects (sorry :)) of pure UI architectures, like time-travelling debugging, sane logging and so on.
> FP is great for problems up to a certain complexity. After that it falls apart.
I disagree completely. FP, and moreover pure FP, is especially well suited for complex problems. Pure functions are testable (if you haven't heard of QuickCheck / generative testing, you've missed out) and as I said above, composable.
If anything, I've never seen a big OO program made out of truly reusable components. Because classes and methods are often impure, before calling a method one must make sure that the correct environment is set up in the right way (if you've ever forgotten calling initSubsystem() before subsytemDoSomething() you know what I mean).
One can rip out a pure function from one codebase and reuse it, as-is, no modification necessary, in another one without any problems. How often does this happen in a big OO project?
> Is "instance" adding an interface to TileMap/HexMap so that either can be passed in as a parameter that requires a RectIterable? Can other interfaces be added that way to TileMap/HexMap?
Yes, and yes. In Java, when you have a class like this:
class A implements B, C {
...
}
In Haskell, you can:
instance D A where
...
I can't give you a complete overview of the differences between ad-hoc and parametric polymorphism, but there are plenty of resources online if you are interested.
> In my editor, can I say something like myTileMap.iterateRect after doing this, or do I need to know that iterateRect exists and remember exactly what it's called?
I can only speak for Haskell here. It's a static language, so things like autocompletion, show type at cursor, jump to definition etc are not a problem. The tooling could be a whole lot better, no doubt, but it's not bad. The REPL helps a lot, too.
TypeScript, which is my current OO language, will use signatures to match to an interface, so it's even more open: You can create an object with the right members and pass it in to a function expecting an interface without even explicitly identifying it as "implementing" the interface.
> If anything, I've never seen a big OO program made out of truly reusable components.
Agreed. The point of using OO in a giant complex program isn't just the reuse, which you point out is exaggerated. The point of using OO in a giant complex program is the isolation you get, so that a team of 1000 people can develop the app in a reasonable way.
I think the cost of FP as the complexity of a program goes up is that the cognitive load to think about the structure of the program goes up exponentially faster than OO. Debugging is a huge issue when things get complicated, and the last I heard Haskell debugging is still in its infancy. A debugger where you just can step through the code is really, really important for some kinds of analysis.
It's all Turing Complete, so you can do anything in FP that you can do in OO. It just feels harder to decompose problems into FP units than OO units, at least at a larger architectural level. Not everything is easily isolated: Sometimes you need a control over here to poke into the state of a control over there, and you don't want that information in some "god object" at the top level; you want a direct connection and mutability.
I have heard good things about Elm, and will probably give it a try at some point. But my 30+ years of programming experience have given me a (possibly incorrect) strong intuition that FP just won't work well for some problems. Can they apply to them? Sure.
> [...] pure FP, is especially well suited for complex problems.
Which is why it's so popular for solving complex problems? Looking at a list of Haskell's uses in industry [1] and applications [2] I don't see a single example of Haskell used in what I would call a "complex problem". Lots of machine learning, parsing/compiling, and transformation tools. Some games written by amateur game developers -- getting a game working is about 10% of the problem. A very few more complex apps that I've never heard of, but nothing that hits the complexity of even a Microsoft Word or Adobe Photoshop, much less a 3d editor like Maya. There's an order of magnitude more complexity in an app like those.
Haskell is 27 years old. Over the last ten years it's been incredibly popular. Totally enough time for at least a few high profile applications to have been created. Where are they?
Maybe Haskell would be a better platform, but it's just harder for many programmers to think pure-FP. But even that's a real limitation: If you're putting more cognitive load on the developer, then you're just changing the problems, not preventing them.
Row polymorphism is indeed very useful. Purescript (a Haskell descendant compiling to js) has it and I wish Haskell did too.
> The point of using OO in a giant complex program is the isolation you get, so that a team of 1000 people can develop the app in a reasonable way.
> Not everything is easily isolated: Sometimes you need a control over here to poke into the state of a control over there, and you don't want that information in some "god object" at the top level; you want a direct connection and mutability.
The above two statements directly contradict each other. Yes, OOP preaches isolation and programming against abstract interfaces, in theory. In practice, more often than not one is tempted to do something the easy way and "poke into the state" of another object. This has disastrous consequences: you never know what will happen where when you call a method. Everything is fair game. It's like changing the lock of the basement door in a skyscraper somewhere only to have it fall because the steal beams depended on this exact lock to hold some state, since it was "faster that way". It's insane.
I've seen so many OOP clusterfucks that I seriously can't believe that you haven't in your much longer career.
Sometimes FP is not a good idea - mostly when performance, memory or deterministic runtime behaviour is a must. But even there languages like Rust show that it may be possible to have one's cake and eat it too.
Other than that, I'd be genuinely interested in hearing what kind of problems you think FP wouldn't work well for.
> Which is why it's so popular for solving complex problems?
It is, actually. I've worked on a very complex codebase in Haskell at my last job (a pathfinding and ticket price calculation backend with ridiculous specifications full of special cases and exceptions that changed almost every day) and I was able to be productive almost from day one. We could refactor with almost absolute confidence because the language is pure and type safe, and we had to because the specs changed so quickly.
Facebook's spam filter and Jane Street's OCaml stack come to mind as other examples.
There aren't as many applications written in FP languages simply because these languages aren't that popular. But popularity isn't a metric of quality.
> If you're putting more cognitive load on the developer, then you're just changing the problems, not preventing them.
I absolutely find the opposite to be the case. Haskell reduces the cognitive load on the programmer, because it is pure (i.e. no need to keeping track of dozens of interactions at once) and type safe (i.e. a function marked as pure can't call a function with IO).
Not to mention things like an actual working software transactional memory implementation that makes concurrent programming as easy as writing a single threaded program. No other language, with the notable exception of maybe Clojure, has a practical STM library. But anyway, STM is just a bonus.
In short, I would say that FP actually delivers what OOP set out to but couldn't. Many FP ideas that were almost unheard of just a couple years ago, like immutable data, pure functions, composition over inheritance, non-nullable types, algebraic data types and so on are slowly but surely taking hold in the mainstream. Pure FP languages just take it to the next level :)
When I said "poke into another object" above, I didn't mean literally mutate the data of another object. Private data is a good thing. I meant grab onto a handle to the object and changes its state through its public interface.
None of the "complex" problems you're talking about hit that level for me; they're mostly the kind of algorithm you can easily approach from multiple directions. Pathfinding, ticket price calculation with a million special cases -- these are broad-but-shallow problems for the most part. A spam filter is one or more algorithms applied to a data stream. I don't know Jane Street's OCaml stack, and a quick Google isn't explaining it to me, but it looks like some kind of parser?
A game might use pathfinding, but an A-star or one of the many newer algorithms is pretty straightforward. Not the kind of complexity I'm talking about. I'm talking about needing to deal with the state of a hundred or a thousand items that have various ways they can interact with each other and that are expected to have emergent behavior. Like you'd find in games, but as you may also find in stock analysis (speaking of a domain where speed can be critical).
I've avoided the worst of OOP I think because I've avoided enterprise work, except for one greenfield project where I got to create an app for the retail space -- and I ran into some pretty awful code that was wrapped in a giant black box that only ran on Windows and required about 160Mb of RAM per user...to run a cash register. No I'm not exaggerating. It was insane.
The problem with using concurrency in FP is that the entire point of concurrency is speed, and on the problems that most need speed you get more out of an imperative language.
There was an attempt to code an image convolution algorithm in Haskell that I read about. A PhD tried his hardest to parallelize the algorithm and beat the speed of C code handling the same problem. I turned out to be impossible; the asymptotic speed of the Haskell code as you added cores never ended up faster than the C code on one core. Amdahl's Law is a hard limit for a lot of problems. [1]
But I actually wasn't asking about pure vs imperative, but rather FP vs OOP. I haven't heard of a pure OOP language, but in that context I think those properties could be seen as more or less orthogonal.
I think OOP is a better way to build an ORM. That said, the query constructors might benefit from a FP approach.
I think FP is a better way to build a mathematics library. That said, data structures might be better represented as objects.
It really depends on the specific application.
I also want to clarify that OOP does not necessarily require inheritance and such.
data Address = Address
{ city :: String
, street :: String
}
data Person = Person
{ name :: String
, address :: Address
}
That said, single level inheritance can be useful if you need to have multiple different objects that require very similar business logic but contain vastly different data.
A good example of this is Django's class based `View` class. I'm not talking about all the mixin views Django offers, just the simple generic `View` class. It handles routing to specific methods based on HTTP request METHOD. That eliminates the need to write the same ~100loc over and over. The alternative would be to use function based views. However, with function based views you end up basically recreating what the class does anyways after checking for each HTTP request method.
No, it just assumes that all kinds of programmers, including front-end programmers, will be much better for knowing those things, which is true.
My only complaint about the matrix myself is that I only scored a level 1/partial 2 on the "books" section. I guess I don't own enough books. I don't like learning from books, though.
Otherwise I hit level 3 on almost every single row, so when I see the folks complaining about how this or that skill is irrelevant, well I have to say that every single row of the matrix has good information, and they should hit the (virtual) books if they want to level up.
2^n: Writes a blog post with intentionally high width and doesn't test it on mobile devices.
I found it more useful to evaluate competency based on activities instead of particular data structures / algorithms / languages: http://science.raphael.poss.name/programming-levels.html
It also declares the intended use of a the matrix, unlike the topic article, and I find it agreeable.
You can possibly argue that in this scheme, ability to think about the whole system falls under "systems decomposition" -- but that's just one box of many, and still doesn't quite relate to building things.
Ultimately, I guess I'd be happy to ignore any and all of these things if someone can build what I want or need.
... by looking at their accomplishments?
Ultimately, productivity is not about how many tools/frameworks/languages/etc. you claim to "know". It's about what you can do with what you know.
Absolutely agree, but with the caveat that the software world currently seems to be on something of a "teamwork is everything" kick which can sometimes obfuscate what individuals are capable of. So judge results -- but be sure everyone is getting a chance to spread their wings at least some of the time.
I get the same sense of revulsion, but I know exactly why. Because almost everything in that matrix can be learned within a day or two. Take any experienced programmer and pick out one of those concepts in the matrix that he/she hasn't used before. It would take very little time to grasp the concept. This isn't like teaching yourself homotopy type theory with only a calculus background.
VP trees — have you ever heard of them? Neither had I until I needed to quickly search and partition a set of 3D points with periodic boundary conditions. It took about a day to become an "expert" on these trees and implement my own library. Isn't that a much more valuable skill than pushing yet another kind of data structure and its properties onto an "interview-ready" list of useless facts?
Smart and gets things done is a better measure and harder to BS.
I think it has value as a framework for someone doing hiring, basically just as a memory-jog for what to talk about in an interview. This would be only after going through and editing it - refining a criterion here, redefining one there, adding a row here, deleting one there, to make it fit the situation. But even then, it shouldn't be taken too rigidly. It's an approximation.
(Of course it doesn't. But maybe it should...)
But seriously... In general, "strategic thinking" for junior people is basically understanding your role in the team, the team's long term strategy, the team's collaborations with other parts of the company, and stuff like that.
edit: and in the comments of that earlier post, someone mentioned that it was posted in HN in 2008 already ...
(edit: Mods, can we get a 2008 on the title?)
This just seems to encourage enterprise-y ridiculous verboseness, which does not help with readability either. By this measure, people like the original UNIX authors would be deemed incompetent; and the opposite would apply to the architecture astronauts.
It's been said many times that naming things is the hardest problem in programming.
That said I don't agree with many of the criteria for the levels. And in general its a bit too theoretical rather then practical.
The next thing that is important is how do these criteria map to your organizations needs?
I do notice as someone who has built tools for just about everything imaginable I lack of awareness of UX and its subtleties. Also optimization, GPU, graphics in general. He has obviously never worked as a game developer. The "systems programming" section should be a section in and of itself. Having "code organization" a sibling of system programming is a joke. But its a start.
Here are some previous submissions of exactly this, nearly this, or closely related items:
https://news.ycombinator.com/item?id=4626695 (122 comments)
https://news.ycombinator.com/item?id=1949915 (106 comments)
https://news.ycombinator.com/item?id=9341146 (51 comments)
https://news.ycombinator.com/item?id=1022394 (40 comments)
https://news.ycombinator.com/item?id=554338 (33 comments)
https://news.ycombinator.com/item?id=232192 (33 comments)
https://news.ycombinator.com/item?id=5905330 (13 comments)
https://news.ycombinator.com/item?id=12081124 (2 comments)
https://news.ycombinator.com/item?id=2823912 (1 comment)
There are also submissions with no comments:
https://news.ycombinator.com/item?id=8966791
http://sijinjoseph.com/2008/04/30/programmer-competency-matr...
Also of note is that he explicitly says it is biased towards non-visual (I think he means non-GUI) developers.
In python level 3 error handling is unpythonic.
I've written libraries on top of apis i haven't memorized.
Also I'd argue that the ability to efficiently start using a new api is a bigger skill.
There's a difference between "is this person capable?" and "is this person personable?
You kinda want both.
http://www.construx.com/uploadedFiles/Construx/Construx_Cont...