Hacker News Reader

> OOP isn't good or bad, Exceptions aren't good or bad, Scrum isn't good or bad. It makes me mad.

It's all true though. Do whatever you want, all that matters is that you do it well - you just can't simplify a good software engineer down to techniques like these.

I recommend reading this piece from a guy who studied software development methodology for over 20 years: http://alistair.cockburn.us/Characterizing+people+as+non-lin...

No. There are very objective reasons why e.g. exceptions do not work for projects of advanced complexity. In short, they add additional, hard to see code paths and a redundant method of result handling (with its own noisy syntax, and new semantics) that brings grand-scheme structural complexity (now EVERY component must be exception-safe, every added feature's behaviour with respect to exceptions must be validated) that affects the whole project. Once you realize it starts to be a problem you cannot fix it anymore because it affects the whole project. This is why bigger projects use less and less features.

Using only few features might slow development a little in the small, and you might even make more errors in the small while developping. But you can FIX them.

I realize this is not hard science, and I have no motivation to do any on it, like collecting and bending statistics. But as long as all people have to say "XY is not good or bad", "things will get better", "XY has just a very steep learning curve", "you just didn't dance the right dance" and all evidence seems to suggest otherwise, then you've just lost the argument. End of story.

Exceptions work excellently for business-level server software; typically they make the server transaction all or nothing, with transactional semantics. The stateless nature of HTTP, and transactions provided by the database, usually add up to make this easy. Business logic throws an exception, everything unwinds, the transaction rolls back, no state was modified, and the server returns an error code to the client: all good.

When you need partial updates to state in the case of an error, then you're in trouble. When you have multiple transactions to coordinate, but no distributed transaction coordinator, you're in trouble (but you are anyway whether you use exceptions or not). The more state you need to preserve in an error case, the worse exceptions are for your use case. Ideally, exceptions are never caught except at the topmost loop: server request dispatcher, UI event handler, or command line driver. When execution must continue with recovery, exceptions are an inferior tool.

A http request handler is not what I mean by server software. I mean a real server - a long-running process with complex state. Yes, exceptions work for simple programs and short scripts (where you don't handle them at all), or as a plug-in mechanism where there is a catch-all handler at the boundary. I kind-of value exceptions as well for writing python scripts.

To a first approximation, most servers with distinct codebases are HTTP app servers ("request handlers").

And not just simple programs and short scripts; very complex programs, as long as they keep state in a transactional store, and a request has approximately the same lifetime as the transaction.

Sure, if you want to think of the thing that is formed by all combined requests from all clients over the lifetime of the database, as a program - then that might (or might not) be a complex program. But the unit in which exceptions are confined with this architecture is the simple, dirty request handler script.

Which likely doesn't even try to handle any errors, and may actually be a bad, faulty program - the edge cases just don't matter that much for businesses. Maybe an e-mail will be sent out twice in a few circumstances, etc: nobody cares.

Transaction management is implemented in the database server, and very likely that doesn't employ exceptions. And more complexity is implemented by the OS (the "process server" if you will) - if your program dies it specifies how to handle the condition, etc. Another part of complexity is dealt with by the language runtime, for example garbage collection - no exceptions to be seen there.

> Sure, if you want to think of the thing that is formed by all combined requests from all clients over the lifetime of the database, as a program

The thing that serves responses to requests is a program. And it's not just a "simple, dirty request handler script," it's often enormously complex. Entire companies exist with their product exposed via HTTP APIs.

The reason you don't see exceptions in databases, operating systems, etc is because they're often written in C++, which has notoriously wonky semantics around exceptions, or C, which doesn't have exceptions at all.

The thing is, what is the alternative to a scrum process? I see so many shops with zero defined process constantly re-invent project management wheels that were solved a long time ago. Start from a known place like scrum and then adapt it to your team and organization.

We standardize around protocols, frameworks, programming languages, etc because adding constraints and processes make a problem much easier to solve.

If not scrum, what? Where do you start?

Kanban, gantt, unified process, extreme programming, trello, rfc.

There are plenty of methodologies and tools that can replace or complement Scrum.

Or even the engineering design process. Every ABET accredited engineering program teaches it, and while there are minor variations in the specifics it's always broadly the same. It's worked well in the engineering disciplines for decades & produces reliable products. There's no reason it can't work well for software, only an unfortunate case of "not invented here syndrome".

A fascinating read, thank you for sharing. These gems of knowledge are what I love HN for.

>This paper outlines the work I reviewed that made it clear to me that ‘people’ really are a first-order, and not incidental, project success driver, the characteristics I currently use as predictors.

My personal experience is similar - methodologies can make things incrementally better for a team, but cannot do anything with the ‘people’ factor.

this is a fantastic piece you've linked, i'm impressed how well it holds up after 20 years, and it admits on multiple occasions that it's been all known 30 years before it's been written. amazing.

>Software is not a contest in being liberal and tolerant. There are methods and techniques that are objectively better than others in reaching most of the goals that are important.

Oh please. Dogmatists make the worst software developers. The "objectively better" criteria is highly situational and not nearly as clear cut as it looks most of the time.

There are plenty of methods and techniques in software that work well for some goals and absolutely suck in others and it's very often not obvious that something that is working well or badly for you is doing so mostly because of your situation.

My favorite example of this is microservices - which apparently worked really well for Martin Fowler once or twice (and I'm 90% sure he didn't understand why) and then when he promoted it, it got applied dogmatically by teams all over the world in situations where it was manifestly an idiotic idea.

The same goes for most hot-button issues - type systems, functional programming, nosql, etc.

Believe me, I know that microservices suck for almost anything, because of the added software and infrastructure management complexity.

And so it is with most other hyped technologies. The complexity and loss of control is very rarely worth it. I'm not a dogmatist, but have seriously tried many different approaches. And that's why I do mostly C (well, not for web applications, which I try to avoid as well :>) with only very simple data structures, and I love it, and it works, and it's maintainable, modular, and very performant. Although I know that C has flaws - I have been trying to find ways to improve (and failed so far).

>Believe me, I know that microservices suck for almost anything

woosh

They work really well in large organizational environments where inter-team coordination costs are very high and language flexibility is valuable. Where these benefits outweigh the additional infrastructure headaches it works very well.

I've likewise worked in environments where they were a terrible idea (e.g. startup team of six).

Like you, I used to be dogmatically anti-microservice. I was wrong.

Yes - that may be, so I'm sorry for omitting the context that I don't work in large teams where you can throw lots of money and developers at problems. (I think that's a sad state of affairs, though). Again, I'm not dogmatic, but have actually tried it, and I know (and have described in my comment) the problems they bring.

It seems to me that you are young and just starting your career.

It’s ok to only want to do certain kinds of work (for example C code that runs close to the metal and that is performant and easy to reason about). Other people in the world though enjoy doing different kinds of work and going about it in different ways.

My advice to you is to be humble and watch people as if they had something to teach you. Whether they are doing scrum, waterfall, using this or that language or architecture, there’s always a ton of things you can learn by watching how someone works.

If you pay closer attention, you will see value in what other people are doing and be able to benefit from what they have discovered.

Very true. There's an alchemy between the software you already have, the tools and technologies your team are familiar with, the problem domain you have, and the solution you implement. There are multiple stable equilibria, multiple local optima. In a more complex environment, microservices can let different teams find local optima better suited to their combination of factors.

I'd still be suspicious of people who start out saying they're going to implement a microservice architecture (especially greenfield) without having an understanding of why, though.

I'm suspicious of anybody who starts out with a clear picture in their head of "what the architecture should look like".

I've almost always found that the earlier on in the process I made hard-to-reverse architectural decisions, the worse the architecture ended up being. The more the architecture came about as a series of gradual, iterative refactors that took in to account the various trade offs that became evident, the better it ended up being.

I'm an anonymous architecture astronaut, so usually I have those "should look like" high-level pictures in my head.

But that's just a fucking picture. A plan. That usually does not survive contact with the changing requirements and implementation reality.

Anyone incapable of admitting that there will be changes should be taken upstate to a nice farm, then shot behind the barn. Usually these people are on the client's team. :(

>> Scrum isn't good or bad >I must have heard this at least a dozen times in the last months.

The thing about any agile methodology is that they are meant to be adapted to the situation at hand, not to be a fixed set of rules that you can just apply to get consistently good results. But many are trying to turn scrum into that - a fixed set of rules that allows micromanagement. I would say that that is definitely bad, but, is that really scrum?

> I do not know of any success stories using Exceptions on Kernels or RDBMSs or even most server software.

I think you need to get out more.

Exceptions are actually pretty standard in the database world. ScyllaDB, arguably the top dog in terms of DB performance, has the "throw" keyword strewn throughout its code base. Oracle PL/SQL's error handling is done entirely through exceptions...

> Oracle PL/SQL's error handling

Is this about the database engine itself? Or just the query parser? Or even just exceptions in the query language itself?

FWIW, Scylla is not an RDBMS. I don't know about its architecture and am not interested in digging more. And that thing works reliably? I don't know - not my cup of tea. Why am I reminded of Node.js? Not cherry-picking here...

    future<file> sstable::open_file(component_type type, open_flags flags, file_open_options opts) {
        auto f = new_sstable_component_file(_read_error_handler, filename(type), flags, opts);
        if ((type != component_type::Data && type != component_type::Index)
                        || get_config().extensions().sstable_file_io_extensions().empty()) {
            return f;
        }
        return f.then([this, type, flags](file f) {
            return do_with(std::move(f), [this, type, flags](file& f) {
                auto ext_range = get_config().extensions().sstable_file_io_extensions();
                return do_for_each(ext_range.begin(), ext_range.end(), [this, &f, type, flags](auto& ext) {
                    // note: we're potentially wrapping more than once. extension mechanism
                    // is responsible for order being sane.

In the 60s or so, people came up with this thing called scheduler. It allowed them to run things (virtually) at the same time, while not needing to make a mess of their code... They also invented cooperative multitasking, where the scheduler only kicks in when explicitly called.

   f = open_file_or_schedule(filepath);
   read_or_schedule(f, buf, size);

I think this is a common misunderstanding: Scrum is not just a tool, it's a toolbox. You don't use the whole toolbox for every job, you pick the tools you need and leave the rest.

BTW, Exceptions and OOP are used conceptually in most kernels, RDBMS, and any bigger piece of software, it just doesn't make sense to use languages which force their use if a project has too many edge cases.

I agree: A few explicit function pointers / vtables and well-considered use of longjmp or cooperative multitasking (coroutines, switching call stacks) is the way to go in some situtations. The point is "explicit".