Hacker News Reader

Being able to solve people's problems doesn't mean you need to be able to program.

What is a "real programmer", anyway? Is it knowing how a CPU works? Managing memory? If you rely on the garbage collector, do you really know what you're doing? If you write a Rails app without fully understanding HTTP, are you just plumbing?

Does it matter?

The reason we build tools and abstractions is to allow us to accomplish higher-level tasks without worrying about lower-level ones. Does it help if you understand lower level ones? Sure. But for millions of apps and websites that aren't Facebook or Hacker News or Firefox, the only problems are high level.

I think that it matters if you can dig into anything. Given reasonable timespan being able to learn x64 machine code, debug and fix buggy driver if necessary, for example. If you're program has GC issues, being able to read papers, read GC source code if necessary and find right set of parameters or change some lines of code. If your Rails app has vulnerability because of underlying HTTP issues, being able to learn more about those HTTP issues, read RFC if necessary, read Ruby http code if necessary and provide required fix. If your manager heard about Meltdown issue and wonders whether our services are vulnerable, real programmer should be able to read papers, understand vulnerability and whether it's important (to shut down services, for example) or not.

Abstractions are leaky. I've yet to encounter one which doesn't leak.

Though if you're working in a team, it's not necessary for everyone to be real programmer. One or two is enough. Those tasks which require real programming are rare, most of tasks are mundane.

I'd say the world 'real' muddles things up and injects too much value judgment, and with little purpose I can see other than making oneself feel special (or inferior).

I'd much prefer 'highly skilled' or 'advanced', if at all necessary.

Sure, I'll concede all of those points! But I think they fundamentally change the question, because now you're presenting a different set of problems to solve.

Understanding one level of abstraction doesn't mean that you understand the levels above or below it. It's perfectly possible (likely, even!) that a team will have someone who can build a good Rails app and someone else who can make sure the HTTP code and infrastructure is secure, yet those people won't have many skills that overlap.

Yes, there are definitely good programmers and bad programmers. IMO, that has to do with how effectively you can solve problems you care about, not (as the comment to which I was replying suggests) the level of abstraction you're comfortable with.

Without low level knowledge you can't tell which feature requests are easy or hard to implement. https://xkcd.com/1425/

The XKCD is a bad example in this case, as it is mostly about domain specific knowledge of computational theory or standardized algorithmic approaches to the problem spaces presented within those domains, whereas "low-level" is something different ... primarily referring to common layers underlying many domains.

Most programmers never have to write low-level code.

This is a good thing.

It doesn't mean they can't, it just means we've moved on from wasting our time re-implementing solutions to problems already adequately solved for a general case. Finally.

Manual memory management is frankly insane outside of extreme edge cases today.

I'd put it differently. Software development severely lacks any objective metrics of performance and quality. We just haven't invented any(at least any practical enough to become mainstream). As a result, we quite often misjudge our(and others') skill and make bad decisions.

It's the goal of software to provide more and more features. The problem is, we usually achieve those features by abusing abstractions we are using. Once the problem becomes apparent, somebody writes a library, which allows us to write a couple of times more crappy code before everything collapses. Rinse and repeat.

Since we are unable on scale to choose right abstractions when necessary, we quite often get ourselves in situations, when it's actually preferable to rewrite everything using lessons learnt. Rewriting comes with its own set of new problems and cycle is complete.

As a result, we are fundamentally unable to reuse already written code on scale, crumbling development and putting the whole industry in constant early-stage.

>>It's the goal of software to provide more and more features.

That's the goal. The key question is: are we writing features that users find valuable or are we writing features that developers find valuable? They're two different things. What's happened is that the community has become incestuous. Instead of focusing on value to the user we're focusing on selling frameworks to one another.

>>Software development severely lacks any objective metrics of performance and quality.

Agreed here as well. But this is related to my first point. Nobody wants to focus on the users. It's far too easy to focus on the technology (or other developers). You write a feature for a user, you can instrument it and see whether users use it or not. You write a feature for a developer, even if nobody uses it you can argue that somewhere, somehow, that feature is going to be critical. It's an abstract value argument -- which is intractable.

>>As a result, we are fundamentally unable to reuse already written code on scale, crumbling development and putting the whole industry in constant early-stage.

It's a sad state of affairs. I suggest that your conclusion continues the broken thinking I'm describing above. We shouldn't be striving for reusable code at scale. We should be striving for people who have strong basic programming skills and are experts in some business domain. Then we would shoot for the minimum amount of technological abstraction necessary for these programmers to solve problems in that particular domain. Because that's really what the whole point is, not creating large codebases that last twenty years, creating tiny bits of code almost instantly that solve business problems for twenty years. We've got our head stuck in the wrong bucket.

> That's the goal. The key question is: are we writing features that users find valuable or are we writing features that developers find valuable? They're two different things. What's happened is that the community has become incestuous. Instead of focusing on value to the user we're focusing on selling frameworks to one another.

I think a deeper problem is that we're not really doing such a great job writing features for developers either. I'm amazed at how much of what I do is still done in the terminal or a bare-bones REPL, not so different from how they worked a few decades ago, when there's so much that could actually improve my day-to-day in small but very noticeable ways.

For example, I make almost constant use of the autocomplete feature when I work with the BEAM REPL (Elixir). I was amazed to find out that this was a relatively new feature. I can't count how many keystrokes it saves me throughout the day.

Of course, I still can't use VIM-style keybindings, and I'm being forced to write my REPL-code line by line instead of a Chrome Devtools style snippets. But, as you say, I do have tons of frameworks to choose from that all do mostly the same thing, and yet none of them do the relatively common thing that I need, so each of them requires me to read documentation and figure out how to configure it 'just-so'...

> we are fundamentally unable to reuse already written code on scale

Actually I disagree - it's just that when something does become reusable it immediately vanishes from people's consciousness. You can see this both in things that become standard library features, and open source components that end up ubiquitous. The list of "incorporates software from" in licenses gets ever longer as people embed copies of SQLite, logging libraries, ORMs, serialisers, and so on.

One of the original article's points was that the things he loved in Lisp became features of other languages, and so ceased to be special.

http://uk.businessinsider.com/how-many-lines-of-code-it-take...

That's a great point.

I watch teams a lot. When I listen to them, I notice how much they talk about tooling instead of solutions. The more they talk about the tooling, the more they're sucking wind.

The truly good stuff just disappears from awareness. That's how you know if a framework or abstraction layer is working for you (instead of you working for it.) It's invisible.

Good point. I would like to point out however that all of your examples appear to do well in abstracting mostly technical problems, not business problems. Technical problems are problems common across our field and natural areas of interests of those '10 %'. Business code is something written by majority of us(or at least I would expect it to be so) and it doesn't appear to scale and grow so nicely.

Most marketing solutions are abstractions of business problems. CRMs, tracking, automated reporting tools, etc.

Of course, they also require a fair bit of custom code that is purely plumbing to hook them up (translate data from one source and stick it into the tool).

Sorry to interject, but I'm curious: as someone who is in their first semester of a CS education, could you elaborate on what you mean by abstraction?

So far, I just know it to mean making your function more widely applicable. I'm assuming there's a better definition I'm missing?

Nope, that's pretty much it. It just have very far reaching consequences.

To illustrate, let's use a simple example. Let's assume you need a logging feature in your application and your language/framework doesn't provide one(pretty much impossible nowadays).

The simplest solution is to just append to a hard-coded file. It solves the imminent problem. It's also very bad solution unless you will never write log again.

Instead, you build a Logger class/module/whatever. The logger provides 'write' functionality, which is abstracted away - client(i.e. code using logger) does not know or care how data is going to be logged. It's a responsibility of the logger to do that and make decisions on how to do that. Now you have abstraction - you depend on ability of Logger module/class/whatever to do its job, without knowing/caring/influencing actual implementation.

Now, let's talk about leaking abstractions. Let's assume you're need to differentiate between different log levels(info/warning/error). Logger is provided as a library(so not easily modifiable) and only has generic 'write' method. Proper solution would be to create a new abstraction, LoggerWithLevels, which would provide methods like 'write_warning', 'write_error' etc. Underneath, it would probably call just call Logger, prepending severity string to logged message. The important thing though is the fact, that we still base our code on abstraction - we don't know how our new logger represents different levels - we use appropriate methods and we assume it's handled alright.

What will likely happen though is that programmer will not create new abstraction layer. Instead, he will start manually or semi manually format strings sent to a logger. Effect will be the same, but abstraction will be now 'leaking', because severity level is now dependent on implementation(prepared log messages), not on logger API. Therefore, any change in the process(like saving logs to online service instead of disk file) will now require changing code in multiple places instead of just one.

This is not what I understand a leaking (or more commonly 'leaky') abstraction to mean.

Instead, assume the previously mentioned Logger does internally write to a file. Now, when the disk becomes full, Logger throws an exception and the caller must do something about it. The abstraction of just writing anything to log is broken and the underlying complexity leaks through.

This example also demonstrates the difficulty of keeping abstractions from leaking. You can't exactly make the code automatically free up disk space or sign up to a remote logging service when disk space runs out.

Well, you're right, I misused the term a bit. I used 'leaky', because it well describes that abstraction cannot hide implementation details(and in the second example those implementation details are used to achieve new functionality). However, it is because of the user abusing abstraction, not because abstraction is inherently leaky in that regard(and that is the reason the term was coined).

I definitely need a new word for situations like this.

Objective metrics of performance are easy to come with: speed, memory usage, latency, throughput, energy efficiency, depending on type of your software. I can launch Windows Task Manager and those metrics are right there. Those metrics can be measured and compared. Quality isn't hard either, just count bugs. Many customers don't want to pay for performance or quality, they want features, delivery speed, shiny UI and listen to marketing too much. Slack is a joke when it comes to memory consumption. But it has smiles and marketing campaigns, so everyone's using it anyway even if chatting was a solved problem 30 years ago in kilobytes of memory.

Don't agree. Metrics you provided allow to compare two projects. They also allow to see progress in the project(or regression).

What they don't allow though is assessing code quality in terms of single project(i.e. judging if it's good or not). Since in many cases we do not have a good baseline, we are unable to consistently evaluate work. That leaves us with aforementioned problems.

It's rather disappointing how the industry hasn't made any progress on CRUD application development productivity in the past 20+ years. Microsoft Visual Basic 4.0 allowed low-skilled developers to build working client/server CRUD applications far faster than any modern web development framework. Of course it's easier to distribute web applications than thick client Windows applications, but other than that we haven't gained anything.

We're trying to move this forward at https://anvil.works - the usability of Visual Basic, with the ease of distribution of the web. Biased I may be, but I'd call that progress :)