Hacker News Reader

Comment by bubblethink | Hacker News Reader

I feel that there is an inherent conflict of interest in the OSS support sales model. Hypothetically, if your software were really simple and robust (think standard unix utilities), nobody would pay for support. On the other end, if you have to deploy openstack, kubernetes, or any other stack with a lot of moving parts, you need support and personnel. So in a perverse way, it's in your interest to make complicated shit. In reality, it is perhaps not quite as bad, but I definitely feel that with a lot of projects for which RH is the sole upstream, the quality or elegance isn't quite there when compared to more traditional linux or unixy things which have more diverse upstreams. This manifests in systemd, freeipa, glusterfs etc. too. These are generally hard problems though. So it's not quite black and white.

View on HN

> nobody would pay for support

That is really, really not the case. You don't pay for someone holding your hand. You pay for support because you either simply have the resources to do so or your business requires you to pay for the things you use.

If you need help setting up Kubernetes, glusterfs or something else you get help from a third party contractor. That's what we do.

You pay for support because your want someone to keep sending out security patches even if upstream should roll over, and you pay for Red Hat specifically because you want to have an upgrade schedule your business can adhere to (as opposed to when upstream feels like it).

Put another way, you pay for the privilege of having someone to sue.

You also pay for support so that when your senior engineers all pack up and start their own business together you can ring somebody to keep your infrastructure running while you find more people with the right level of knowledge.

Except that's not really what break-fix support covers. Specifically, support doesn't typically touch your infrastructure; it provides advice, documentation and patches which a local team needs to actually implement.

Advice is the key here. If you're in a situation where literally your entire team goes then you've probably got bigger problems, but when you lose some key people having professional support available to guide your juniors is beyond useful.

>> you pay for support because you either simply have the resources to do so or your business requires you to pay for the things you use.

A nuance: many big businesses require that you pay for software you use, and part of what you are paying for is an indemnification via the vendor, which makes sense to purchase for big businesses with deep pockets https://www.scl.org/articles/3030-indemnity-and-limitation-o...

Companies also pay for support because after a few years the version of the open source product has moved on and nobody realizes the true cost of ownership does not end with the acquisition cost the company doesn't have the budget to upgrade. Thus they're 5 levels back and nobody in the Open Source community is interested in fixing old software. As an example, RedHat offers up to 9 years of support for Openshift. That's why companies pay for support.

https://www.ibm.com/support/knowledgecenter/en/SSC623_11.7.0...

edit: clarified acronym

Exactly this. A dev comes all bright-eyed and bushy-tailed and says we can save a week on the project if we use this open source library instead of writing it ourselves. But he or she doesn't see that they are committing the organisation for the lifetime of the project for someone to keep on top of patches, resolve any breaking API changes, even fully maintain it if the original developer loses interest - or ditching it and writing our own anyway. There is a lot of great open source around, but it is by no means "free" when you look at the big picture. This is one of the key differences that marks a senior from a junior engineer. You can see the junior's eyes glaze over when you try to explain it and you know they'll be back next week begging to use something else they read about on Twitter or HN...

I find this attitude really tiring. I'm a contractor and I work with people all the time who feel like they must avoid improving anything because if they do they'll be out of a job. It's nonsense. It's the same as the luddites that think robots will tale all the jobs. There will always be something new (and more fun to do) than maintaining broken systems and processes.

I have been in a couple slightly dicey situations but overall have found that there are always other things I’d like to have time to do if I didn’t spend so much energy babysitting a shitty tool or process.

You find out frequently that people who use the tool have ten things they never asked for because the tools don’t even handle basic common sense concerns. It feels like asking for a pony, possibly from a neglectful parent.

You fix a few things and get labeled as clever or useful. People come to you with more stuff and it expands your grasp of the organization. For me this strategy has opened up leadership opportunities.

You can still be laid off but everybody else will wonder who was the idiot who got rid of the person who was keeping the ship afloat.

> fix a few things and get labeled as clever or useful. People come to you with more stuff and it expands your grasp of the organization,...

Yes, but it's not really a satisfying work experience!

> who was the idiot who got rid of the person who was keeping the ship afloat

the same that chose the shitty tool for a shitty strategy in the first place

> Yes, but it's not really a satisfying work experience!

I dunno, I like it when people come to me with problems they need solved. You get to work with a stimulating variety of issues, get to meet more of your coworkers and earn their gratitude, and there's a real satisfaction in knowing that people see you as a capable problem-solver. I'd actually really enjoy having a job as "the guy who makes the tools work better for everyone else."

> Yes, but it's not really a satisfying work experience!

I think that comes down to your disposition. I find meetings and scope negotiation very energy draining but one in one problem solving doesn’t tax me that much. As long as I’m not spinning plates when you come up to me I can usually stop what I’m doing and get back to it later.

Hard to solve problems often conceal architectural problems or irreversible decisions and I’d rather be involved long before it becomes intractable.

Indeed, solving problems is my favorite part of my job; the issue is more along the lines of not being able to spend more time improving things, and wasting time fixing other people's git mistakes, carrying boxes of server components up 5 flights of stairs because all the elevators are broken, sitting through meetings re-explaining the same things as 6 months ago, because no one takes notes.

there will always be something to do because once software A is so easy to use anyone can do it, you can jump to support software B that isn't there yet.

But if you poured resources in A with the hope of making supporting A your bread and butter, and A is so easy to use no one needs help, well, you kind of wasted your time and money, UNLESS you managed to make it so popular that the 5% of people who still need help anyway are enough to keep you afloat. That's hard though.

This isn't necessarily true in the enterprise. At our company we buy support for everything we use, whether we need the support or not. It's more like insurance, which also helps to break out of the O(N) scaling that people think goes along with the support business model.

Yeah I really think this is the most important point. In an enterprise environment it's almost more important to be able to point at who is responsible for supporting a product as it is for the product to actually work. The due diligence of any purchasing decision will always involve 'and what if it goes wrong'- especially since most enterprise companies will involve legal when getting approval for the use of OSS.

Large companies sometimes don't actually 'do' any of their own IT work. There are layers of SLA's across data centers and applications, spread across multiple vendors, contractors, etc.

Everything must have a legal contract that specifies support terms, penalties, etc. Large company's motives are risk avoidance to ensure profits for shareholders.

Basically companies will pay a premium to have someone they can call and yell at if something goes sideways. Usually corporate finance departments have no logistical way to accept a reimbursement from a vendor for missing a SLA but they like to put clauses like that in a contract.

The other part is the professional services arm tied to the sales process. RH can provide experts that only they can provide - they are the ones writing the code sometimes. Other companies like Oracle have professional services but I doubt they are committers on the products being sold.

One of the ways I poke fun at the Apache foundation is to point out that their online documentation is vague and uninspiring, and then poof the lead maintainers write an Oreilly book that explains all the whys and hows and whats.

Documentation doesn't exist to explain concepts and give tutorials so much as document functionality and possibly show examples of use.

It really doesn’t take that much extra work to explain why someone should care about the existence of a function. But it makes a huge difference in how approachable the library is, and how easy it is to learn (for instance, is this even the function I’m looking for?)

Why not? Document functionality and possibly showing examples is great (in general I'm a big advocate for examples), but those don't preclude explaining concepts.

Some of the best documentation I've experienced covers concepts, some of the worst doesn't.

I think one of the best reasons is because for users that have some experience with the technology want to be able to quickly find the documentation they're interested in. Can this API do X, for instance. And if it can what is required, what is optional and what are the ranges for values.

I don't want to wade through lots of text and other clutter. I'd much rather have a tutorial or training be a different thing I can use when new to the technology.

Those are valid points, but that just means the API documentation and the concept documentation should largely be separate documents, not that the documentation doesn't belong. In the best cases a link over to relevant concept docs from API and vice-versa is great, and occasionally enlightens even hardened devs ("wait, there's an API to do this directly?", "Wait, this concept was added and streamlined in the 3 years since I first used this product?", etc)

I think there can be different types of documentation (tutorials, reference, explanation, how-to guides). I came across this article a while back that goes over different categories and always re-read it when I know I'm going to have to write a bunch of documentation for something and always find it helpful to really think about the type that I need to write. In regards to the poster you replied to I think Oreilly books are in the explanation category of documentation instead of the traditional reference kind found online.

The article I'm referring to: https://www.divio.com/blog/documentation/

Documentation is a moving target. Depending on your project, you often have to mix background explanations with code samples. Fields evolve. New people show up. Documentation can never really be good enough. Books provide a way of filling in certain gaps in a structured way that gives people more background. Often times, people expect documentation to do more than this. I tend to agree with you, but it's not what end users end up needing a lot of the time.

Books are incredibly cheap, you pay for the editor &c more than for the author for most tech books.

Kind of like professors writing difficult to reproduce papers, then explaining everything in the textbook (which they also assign to their students...)

Do you have any specific examples of this? It doesn’t seem super common that a paper that would be significant enough to make it through peer review would also appeal to the wider/beginner-to-intermediate-level audience of a textbook.

Oreilly author and project lead for an open source eclipse foundation (apache style project though!) project that runs a company. Happy to answer questions here about the how/what/why this happens.

OSS funding and documentation quality are problems I think about semi-regularly. A few questions I have are:

1) Is the model of keeping software open but education material closed a necessary compromise to fund the project?

1.b) If so, have you explored other funding models that weren’t viable?

2) How is the experience of publishing through OReilly? Do they take considerable profit shares? Do they provide editors, advertising, and/or other valuable services? Essentially are they worth it?

1) Yes I think incentive structures are a part of it. For us as authors,I know I typically think about the website very differently than the book. Having an editor peer review also helps keep me focused. What you find with a lot of open source coders, they tend not to always be good writers (me included). My book is 530 pages and took 2.5 years to crank out in between new releases. My company funded the book's development more as a loss leader. Authors don't make crap from the royalites. You're better selling it as part of a larger package such as training or support.

2. You have to know the right people. Go to the conferences and ask them. It took a long time. The profit shares aren't much. I more did it for the exposure at their conferences. We found it to be worth it as part of a larger package and networking with others in the field more than anything else. The notoriety has really helped as well. It's a lot of work, but definitely worth it if you can get it. Just be ready to put in a metric ton of work.

Making complex buggy software is the default. It is really, really hard to write simple, robust software.

wrong angle. Would you prefer to be an unknown/invisible/unappreciated manager of a small team writing simple robust software or to be a highly visible and valuable (S)VP of an organization developing a buggy complex equivalent of that simple robust software? I've observed the same answer to that modern Hamlet's question everywhere around me through all these years at the BigCo-s :)

In my experience, all developers sincerely believe in writing simple, robust software, and firmly believe they are implementing simple, robust software.

However, almost none of them (myself included) succeed at it, not even close.

When you assume inherent complexity exists, and competition exists, it works out fine. The consumers will always require some support for any complex OSS, but they want to minimize it (by finding OSS with reduced accidental complexity). OSS are incentivized to increase accidental complexity, but they’re capped by reputation and competition.

Of course, you can work around the core incentives (eg by marketing), and a lack of competition changes everything (consumers take whatever they can get, producers sell as high as they possibly can without bankrupting the consumer), but in a well-behaving market it should be fine.

Your RH-upstreams examples can probably be sufficiently explained by lack of competition, due to a captured market. Which isn’t behavior specific to OSS, but to every market. Notably, breaking RH’s stranglehold would be naturally easier than say apple’s

But that exact conflict of interest exists in closed source support models too. It exists for any company that wants to sell a support service. And it relies on the supporters being small-minded enough that they'd rather have a 1% market in 100 people than a 0.01% market of 1,000,000.

The incentive you identify only goes away if the software is a product with no support - and then businesses won't buy it, because there is no support.

> Hypothetically, if your software were really simple and robust (think standard unix utilities), nobody would pay for support.

Yeah but reality shows that standard unix utilities are not used by the majority of people. So many people use unix based systems, especially macOS which has much more simple/bare-bones tools than GNU/Linux and still few people do read man pages for instance

Disclaimer: Your concern is exactly what pays my salary. Take what I say with a grain of salt.

I typically see this attitude at the developer level, but the line of business never sees it like that. Devs don't have incentive to care about these things. They also just want to deal with the vendor.

What you sell is more "insurance" and a guaranteed timeline on bug fixes/releases.

As someone who supports OSS, why should you be able to demand I ship something on a certain schedule if you're not paying me? If it's a business transaction, then there's a contract and aligned incentives on both sides.

Taking this a step further, this is usually not enough which is why we then see open core business models like elastic search, gitlab, (and also my company skymind) selling a combination of support + licensing.

I think of it the other way. "I get paid X amount a month to guarantee support. How do I reduce my support costs? Oh I know, I'll make my product easier to use." Kind of like insurance companies who spend effort to figure out a way to make your activities safer.

Of course if they made it perfectly easy they would be out of a job since the software is free. But so long as it's within a certain margin it works out.

I kind of feel like we need a PBS of software. We have a few things that partly fill in that gap (Apache, Mozilla) but leave a lot to be desired.

I work for a large insurance company that has a large RHEL deployment. We don't buy support really, it's just licenses to use with support attached. We'd use Centos in a heartbeat but Oracle and IBM won't support Centos. Our managers know the value for the support isn't really worth it, but it's about two factors: having someone on the other line to yell at when things go pear shaped, and also having teflon armor in case another manager uses an incident as ammunition in a fight for prestige.

As the article explains the real problem with that model is that selling software is scalable - but support is not. So a support business will never have the same margins as a software selling business - and will not be able to invest in the software the same money.

See Apache Spark.

It is powerful, but was never made remotely easy to set up or administer. They finished it and called it Databricks.

Reply on news.ycombinator.com