Open-source infrastructure is not venture-backable
medium.com
medium.com
The most surprising example of this for me is Socket.io (https://github.com/socketio/socket.io) - This is a hugely popular open source project; but it was never able to get funding on its own merit - To the VC world, it was always seen as a 'side project' of another startup - Not as a fundable project in its own right.
The archnemesis of open source infrastructure software are services. By funding services exclusively, VCs have been actively pushing the adoption of services to the detriment of open source... For example, instead of encouraging startups to run their own open source database engine (which is quite feasible), startups may be tempted to use a hosted database-as-a-service instead (because it was advertised to them). Instead of using an open source CMS and host it themselves, they will use a hosted as-a-service solution (even though it's more expensive and often doesn't make sense for this startup economically). Instead of doing their own performance monitoring for their app (using an open source tool), they will use a service instead.
Not so long ago, the 'as-a-service' companies which did well were the ones which had roots in open source and actually did a lot of OSS work themselves (E.g. Wordpress, Red Hat). Now you have a lot of as-a-service startups which came out of nowhere, raised a boatload of VC funding and are stealing all the limelight away from smaller OSS (pioneering) projects who have been working towards a specific vision for many years.
Software on it's own isn't a service, there will always be a business case for someone else to do the "hard work" and just run it for you.
https://github.com/convox/rack
The current technology goal is to build a high quality "batteries included" way to setup and operate AWS for engineering teams. We are working for a low complexity, high reliability system:
* 1 command installer to provision a VPC, container cluster, and other important AWS services
* 1 command to deploy apps to the cluster
* Following best practices for cloud architecture (immutable infrastructure, automated updates and rollbacks)
* Leveraging AWS's reliability and scalability (apps are ELB -> instance, nothing experimental)
I'm heavily biased, but it does feel like important work that could have a huge impact.
Companies could save a tremendous amount of time, energy and money if they don't have to re-invent the wheel and build bespoke tools.
Programmers just getting started deserve a way to learn and use AWS without needing to know the intimate details of 20+ services.
Docker is a great example. We really couldn't build our project without Docker. The amount of expertise that's encoded into the Docker API, documentation and growing ecosystem is amazing. When every computer and server talks the Docker API, we all can focus more on our business and less on the systems that run it.
[edit]—typos
(I'd assume that AWS is just an initial implementation anyway - and clearly that makes sense in this market)
> Open source infrastructure refers to all the tools that help developers build software. On a deep level, it includes physical things like servers, but closer to the surface, it also includes things like programming languages, frameworks, and libraries.
I also don't follow the AWS -> not open logic.
I lease resources in someone else's data center. I use proprietary software services when they make building my business software easier.
AWS is not an turtles all the way down open stack, but their docs are good, have open source clients for all their APIs, and offer up standard Linux and Docker systems.
Convox is open source software. It makes setting everything up easy and ensures total access and visibility into your resources. It also adds open protocols like we hooks and syslog to integrate with your other systems.
You can't get the source to AWS to run it yourself, and you have to accept Amazons pricing, but it's still a very open ecosystem.
Once you're up and running on the AWS platform with any significant infrastructure setup, you're basically locked in for life. I hope there are counter-examples, but I'm unaware of any significant migrations off of AWS in the last few years. Some of that is due to their overall pretty high service level, but it's also incredibly hard to replicate the AWS infrastructure on your own because the tooling either doesn't exist or is immature.
I think there's a really under-served niche of startups with AWS spends of greater than $200K/year that would love to move some or most of their infra off of AWS to achieve greater cost saving or unlock different product possibilities that can come with running in a datacenter or across different cloud providers. And would be willing to pay for services and tools to fill that need. I'm finding myself in that situation now, and even though we've been careful to not get tied in to most of the managed AWS services, we're still facing lock-in due to heavy integration with cloudformation. Without a good open-source or managed service alternative to just cloudformation, we're basically faced with needing to completely re-architect our systems if we move off of AWS. And full re-architecture initiatives tend to stay near the bottom of the Devops backlog if things are working.
I saw a local Lambda emulator today on HN.
But I do understand your concern. Software on AWS has an serious gravity and is hard to move.
I totally understand the CF challenge. I'm locked into that tool too. Terraform is great but doesn't really offer anything to replace the deepest parts of CF.
Which brings us back to the article. More funding for tools in the space would be huge for all of us.
First, understand the difference between social norms and market norms (I like this overview: http://danariely.com/the-books/excerpted-from-chapter-4-%E2%...) and how the introduction of market norms crowd out social norms (this is a good overview: https://natewkratzer.wordpress.com/2013/07/23/market-norms-a...).
Now that you have the necessary context, you can understand why I'm extremely doubtful that VC's can realistically have a large positive impact on the open source community.
Open source projects need donations in talented engineer time, not in money. There isn't enough money available to compensate for the man-hours invested in open source projects.
Open source projects are successful because they motivate contributors with social capital rewards of some kind and because they allow many different actors to combine human resources to eliminate shared costs/problems.
If some kind of wealthy supervillain wanted to destroy open source the best way to do it would be to inject a lot of money into the system, get everyone to start thinking in terms of how much they will be paid per hour to contribute to the project. It would be peanuts. Now the market norms would eclipse the social ones and they would mostly stop contributing. It would probably also be useful to use the funding power to steer projects towards architectures useful to you and less useful to others, if some of those others are your competitors then it may be your duty to do so.
This overlaps quite a bit with the existing naive view of the open source community as a big group of untapped future unpaid interns to exploit as well.
This, of course, doesn't mean that there isn't promise in funding infrastructure projects developed by paid contributors that are donated to the public on completion. Just that mixing the two models almost certainly won't work.
I'm willing to believe that most open source projects could make substantial improvements if they had funding. See how Patreon has made a huge difference for Youtube videographers.
As a infra provider to many open source projects, I have a very solid way of converting money into engineer time: I hire and train them.
Since you have the experience I am very curious about one aspect of this, does the fact that some fellow contributors are paid have a noticeable effect on volunteer contributors. As in "Why would I do this for free when Alex gets paid?" or "Fuck this, I'll let the paid guys do this annoying part" or even just a cultural divide between paid/unpaid or a drop in volunteer contributors in general?
The theory says there would be a small but noticeable effect along those lines if people are thinking about it.
I've seen similar effects with Patreon type contributions, once the money comes once things change and once the money is spent work stops completely until there is more money. Most never come back. That and you get promising fun projects dropped in favour of the stagnant projects with better name recognition, which doesn't last because fixing legacy bugs for less than minimum wage is not a fulfilling way to spend your personal time.
I'd also be interested in how you'd feel having a donation of skilled engineer 20% time compared to money to hire someone less experienced but full-time.
I realize now that we're talking about different meanings of the term infrastructure. My organization offers co-location and hosting services to a variety of open source projects. You can think of us as sysadmin to the open source community. And for all the talk about funding developer time, discussion about the operations side is notoriously quiet. There's a few good actors that help us out, and we're trying to raise awareness of the issue.
Within our org, we try to contribute to open source as we go, and we have a software development team, but we're not paying people to write software for specific external projects. AFAIK, nobody is demotivated by our existence. You just don't see a lot of volunteers to get paged in the middle of the night when the website is down.
The open source software we fund development of is largely to suit our operations team needs. They're not the sort of software projects you'd expect people to volunteer for, which is why we allocated student developers to start them. It's hard to separate the compensation effects on volunteers from the general 'I don't run this software' effects, and since we didn't inject money into existing projects, there's no debates about the inequity.
There's another unstated funding challenge: if you want to maximize value your money brings to open source, you want to find someone who can escalate their contribution. If you assume the Steve Hansons of OSS are already spending all their time contributing, handing them money supports the status quo, but handing money to students who would otherwise be flipping burgers brings new faces, and additional effort to projects. In a sense, it matters whether your charitable goal is to reward lifetime contributions, or to gift the community better software, and in small ways these can be contradictory. Especially if continued funding relies on demonstrating return on the gifts, merely perpetuating the status quo is a problem.
> I'd also be interested in how you'd feel having a donation of skilled engineer 20% time compared to money to hire someone less experienced but full-time.
So our model is to hire student employees. They're cheap, and we can be flexible around their schedules in ways other potential employers cannot. We participate in GSoC, and are open to contributions, but generally speaking what we develop is not relevant for home use. On the operations side of things, we participate in the Chef ecosystem, but there's a level of access and such that open source infra teams prefer to control access to. Do you want to publish what exact version of nginx you're running?
> And for all the talk about funding developer time, discussion about the operations side is notoriously quiet. There's a few good actors that help us out, and we're trying to raise awareness of the issue.
> You just don't see a lot of volunteers to get paged in the middle of the night when the website is down.
That's a great point, that's some good work you are doing and I'll admit that I needed my awareness raised on that issue as well. Thanks.
> In a sense, it matters whether your charitable goal is to reward lifetime contributions, or to gift the community better software, and in small ways these can be contradictory. Especially if continued funding relies on demonstrating return on the gifts, merely perpetuating the status quo is a problem.
Well put, I'm a believer in the end goal being better free software and rewarding contributors is a (complex and risky) means to that end.
Good points about the benefits of bringing in juniors and the subtle but important differences between openness at different architectural layers as well.
"But hardly any founder, VC, or big tech employee was aware of the issue, even when they used or benefitted directly from these projects."
I just can't wrap my mind around this. As a former "big tech employee" and current founder I am constantly cognizant of the free software that shapes my working day and every solution I implement. From kernels and userspace tools to compilers and interpreters to IDEs and VCS to frameworks and protocols to online services and communication tools... it's nuts how much stuff we all use. I'm living on ramen now but one of my personal goals is to be successful enough to give back in a substantial way to these developers and projects that have been so instrumental to everything we're doing.
Do people really take all this for granted? Are we as a community not doing a good enough job of teaching/explaining it?
I had to explain very slowly that if he couldn't compete against an open source project... perhaps his company with tens of millions dollars in VC funding didn't deserve to succeed. And, that the company's product depended on using dozens of open source projects for free.
Needless to say he wasn't pleased.
E.g. https://ryanseys.com/blog/summer-at-mozilla/
>Interns at Mozilla, myself included, are truly spoiled rotten. Competitive pay, free travel/housing, free snacks/drinks and catered lunches every week were really just icing on the massive cake that was my internship!
>Oh, and did I mention that Mozilla also sent me to Paris, France?! Yeah, it happened. For my final working week, myself and a handful of the Identity team met up in the Paris office and hacked on Persona, and ate… and drank… a lot!
http://www.businessinsider.com/inside-mozillas-amazing-offic...
Also, at the margins, I tend to think that perks allow organizations to hire more cheaply than wage increases.
That sounds like the kind of thing some bored economist must have done some kind study on at some point. I'm too lazy to look for it, so maybe someone else can volunteer. I'll just say that sounds very likely to be true.
There are plenty of for profit companies around open source software that are venture backed:
Mesos(mesosphere)
Spark(Databricks)
Flink (Data Artisans)
Zeppelin (NFLabs)
Scala (typesafe)
Linux (Red hat)
Hadoop (Horton, Cloudera, MapR)
Elasticsearch (Elastic)
PredictionIO (PredictionIO Inc)
Meteor (Meteor Inc)
Deeplearning4j (My company skymind)
RethinkDB(RethinkDB inc)
Redis (Redis Labs)
Wordpress(Automattic)
Drupal(Acquia)
Docker (docker inc)
Coreos (coreos inc)
NGINX(nginx inc)
and the list goes on!
Acquired companies also include springsource (VMWare) , jboss (redhat), and ansible (redhat)
Lastly, there's open source from companies such as facebook where the goal is likely hiring. By open sourcing internal tools, it's easier to onboard new devs for recruiting. If you like their tools why not work there?
The key distinction people are missing is that these companies don't monetize open source directly. Instead open source is used as a means of building a user base that will indirectly generate revenue in other ways such as support, licensing solutions around or on top of the software (open core), or through some form of consulting.
FOSS such as the GNU software is a different beast where the goal isn't profit.
I think the confusion is: we can all use the software provided by these companies for free (depending on the context/license) without paying them and contribute back in other ways such as bug reports and the like.
The other thing here is individual devs monetizing their singular github repos. There's no reason you can't charge for support or consulting. Radim @ gensim does exactly that and he does fine.
Just because you can git clone a repo doesn't mean you can use it effectively. You typically seek support via community or commercial. There's no reason both can't exist and they both do.
I hope that helps a bit!
This isn't a distinction, it's the definition of open source. By definition if you have an open source company, you're making money through some channel other than keeping the software proprietary and selling binaries of it. So your list quite rightly invalidates the author's claim--it's not just some subtle distinction that was ignored, it's proof that you can indeed make money from open source software.
I agree at what you're getting at though.
This has been a big question for our company. Ultimately, one of our viewpoints has been that the current breed of open source companies, especially with VC pressure, struggle to align with the community after a couple years and reality sets in.
Community user: this is awesome, let me add this bugfix
Company: bugfix tweaked/improved & in mainline. Come speak at our conference!
Community user: cool. Now here's this admin tool.
Company: uh,no,sorry,please keep that out of the main repos.
Community user: but admin is pretty central...
When the center of the community actively dissuades core contributions, which is what we see, the alignment isn't working.
It's a good hypothesis but I'm not sure it happens often in practice. Most companies I found that had a premium model still have a premium model. Mine wasn't a large sample, though.
IMO Open Development is what open source originally meant. Open development is only sustainable if devs == users, otherwise devs going to burn out eventually. Open source companies (as per your defintiton) are profitiable because the oss is not the primary product but a feature. Their primary products are indeed closed.
http://i.huffpost.com/gen/1253014/thumbs/o-AARON-SWARTZ-face...
It seems simple enough, though. Make your own!
Making money off free software infrastructure is hard, because infrastructure is meant to be used by developers. A developer knows (or can learn) how to implement, manage, and maintain your piece of free software infrastructure, so how can you convince a developer to pay you for support?
Applications, on the other hand, can be targeted at non-developers. These users can convince themselves or their bosses that your free (as in freedom) application is worth paying for, so you can make it run smoothly and let them focus on running their business.
The only solution I can think of is for free software infrastructure developers to set up foundations which can take donations, and for application developers to support the infrastructure projects they depend upon.
One of the best examples of this is the Django Software Foundation. I made a donation last month, and I haven't even started making money.
I like the example Automattic sets. They give away the WordPress engine, but also run it as a subscription service.
Running software is hard, and there are plenty of people that will exchange money for an SLA, both service availability and support response time.
His project Mongrel, allowed other for profit Ruby projects exist in the way that the community needed.
A good analog of getting paid with 'free to use' theory are calculators. We know that 2 + 2 is 4. The theory is free and out there. If you want a thing that does the calculation for you, then you gotta pay $5 for a calculator.
CS theory is similar. Each tool or abstraction builds on a previous one. While we can't charge at each step of abstraction, we can at least license accordingly to encourage sharing of profits made from it's use.
Isn't there a lot of infrastructure software that is maintained or outright developed by VC-funded entities?
How about a new civil servant job: OSS maintainer.
EDIT:
> We are transforming government from the inside out, creating cultural change by working with teams inside agencies who want to create great services for the public.
https://18f.gsa.gov/2016/01/12/hacking-inclusion-by-customiz...
I can't tell what's more wrong here. The passive-aggressiveness of writing a bot to correct people, or the fact it's going on on a .gov.
(And before you scream at me for being a heartless sexist bastard or some such, this would be just as messed up if it were a bot that corrects your->you're. Passive aggressiveness certainly does not improve workplace health.)
It's a bot. Ignore it.
And seeing as my previous post is predictably getting instant downvoted, I should also state that I really don't care about PCness one way or another. What I do see is that "fixing government" apparently includes policing how people talk to each other in an extremely passive-aggressive way that would not be tolerated in any community I moderate or, really, have ever been a part of. All that in the name of improving workplace communications, which this simply does not achieve.
"And seeing as my previous post is predictably getting instant downvoted, I should also state that I really don't care about PCness one way or another. What I do see is that "fixing government" apparently includes policing how people talk to each other in an extremely passive-aggressive way that would not be tolerated in any community I moderate or, really, have ever been a part of. All that in the name of improving workplace communications, which this simply does not achieve."
So you have no idea who they are (they're a technology skunkworks within the GSA [1], which manages and supports the basic functioning of federal agencies of the US Federal Government) and you're commenting on their internal communications policy because you take issue with it?
Why would you even waste the time making an uneducated comment then?
You even make the comment that their work is localized to SF, when they have offices across the country in support of their work!
[1] https://en.wikipedia.org/wiki/General_Services_Administratio...
I did take a couple of minutes to look them up further before commenting, but I was not aware it was completely forbidden to comment on first impressions on HN. You'll forgive the confusion, I see a lot of first-impression-driven comments around here.
You seem to be aggressively defending them - Why? I have not attacked them. I'll take you at your word when you said they're doing good work.
Does this answer your question on why I made that comment? It is pretty distressing that I have to defend myself making a comment on an internet forum.
I defend anyone who is attempting to effect positive change in the world, aggressively.
For the record, I have no relation or affiliation to 18F, or the USDS.
So you defend their faults, too?
Or are you saying here that you don't believe I myself attempt to effect positive change in the world? With my open source history, I'd find that pretty offensive.
It's been pretty well established that needlessly (male) gendered language makes people feel excluded. Actually, that's too nice. That kind of language excludes people. Period.
It communicates that the speaker does not contemplate a non-male audience. I do not know about the communities that you moderate, but constantly assuming a male audience at a place like 18F is wrong factually and is disrespectful.
I've triggered the slackbot by using "guys" many times and I've never felt that this was aggressive in any way. It's usually a chuckle-worthy experience for all involved. And that's why I love the bot. I do understand concerns over "policing" language. No one wants to feel pestered into writing/talking in a way that feels unnatural to them. However, instincts should not be above criticism. And a silly bot[0] is a great way to gently, politely, and humorously help people unlearn behavior that excludes.
[0] We have other silly bots, too. The one we probably love to hate the most is angrytock (https://github.com/18f/angrytock), which will remind us to submit our time cards (using our open source time tracker, Tock. https://github.com/18f/tock).
If this bot isn't actually negatively affecting anybody, more power to you. I certainly won't be the one to tell you what you can and can't do within your company.
Putting myself in those shoes, I would personally find it incredibly rude, and I highly doubt I'm in the minority holding that view, which means in my mind it's highly likely that other people at your company hold the same view (loudly or silently), which will contribute to making communications worse.
Putting passive-aggressiveness aside (which is a real problem)... I never contemplate "male-only" audiences, regardless of what I say. A somewhat-recent example: I've had some insecure guy come up to me being offended at my usage of "us gamers", as if saying "gamers" somehow excluded women.
There is something that really bothers me in such cases: The person is assuming ill-intent on behalf of the speaker. To me, it is extremely discriminatory to assume I don't include women when I speak. Why wouldn't I include them? Because they're women? Isn't that exactly the type of backwards thinking feminists are against?
shrugs I better just stop here. Whenever political correctness gets brought up, people on here seem to forget all logic and act completely irrationally - feminism really is the "for the children!" of the tech sector. I see even your post got downvoted for whatever reason; I brought it back up...
This is where these kinds of issues typically go off the rails. With male-gendered language as the default, I doubt very many people intend to do anything wrong. It's how I learned to speak and I'm sure it's how many others did, too.
A silly nudge from a Slack bot, is just that--not an accusation or a judgment of character.
I was thinking about this a bit more last night, and I thought of another reason why the bot is great: it doesn't discriminate. It doesn't matter if you are in leadership at 18F, a guest user in our Slack, an employee who started two weeks ago, male, female, or something else. The bot doesn't care.
There's a bit of politics to how people get "called out"--e.g. who wants to pull the boss aside and correct their behavior? The end result is likely to be that certain employees get corrected and other employees don't. A bot avoids all of this.
People can get offended at all sorts of things. Maybe some girls will be offended at people saying "guys" gender-neutrally. A lot of women won't.
Are you going to write bots to prevent all possible verbal offences? What if some of those bots offend people themselves, what do you do then?
I find it interesting you mention the nondiscriminatory aspect of a bot as a feature. To me it's one of the most disturbing parts, and it's what creates the "passive aggressiveness" I was talking about. If you have a problem with the way I say something, you can be upfront about it with me and I'll change my behaviour. Writing a bot for it is quite a backhanded way of getting me to do something, with less chances of succeeding (and if anything, more chances of continuing out of spite).
Like I said though, all this is personal take. Maybe everybody at your company is fine with this. However, I did show this little post around and got a near-unanimous reaction similar to mine, which reinforces what I thought before: It is highly likely there's people at your company who think this is far more than "a silly nudge", and the bot would then have a completely counter-productive effect to its original intent.
[1] The point isn't that other governmental entities should be able to use "gov", though. Since English is not a universal language.
Of course there are boundaries to the happiness-increases-productivity dynamic, but quickly coding up a Slack bot seems well within the safe range.
Sure, but to what degree? Does the 'gained productivity' actually pay for the cost of maintaining 'happiness infrastructure'?
Is this common in UK/US/rest of the Anglosphere? Is it PC/anti-PC/neutral?
PS: in case it isn't obvious, I am not a native speaker.
As for whether it's PC/anti-pc/whatever, this is really down to the intent behind it and how it's perceived by the recipients. If you're a woman, addressing other women as "guys", there's no ill intent (and none received). It's far more offensive to have someone else police your words...
But which is entirely understandable in a workplace environment.
Unless you have deep pockets and don't mind paying several hundred thousand dollars out in settlements at a time.
It also, frankly, isn't really a blindspot so much as a known problem by devs that contribute/heavily use OS. We just don't make the financial decisions.
No rational investor wants to invest in something that will absolutely not give them a return. Most open source infrastructure tools lack a business model to capture revenue from its users.
So one of several things will have to happen in order for open source infrastructure to get funded:
- Investors will part ways with their money, full knowing that they
will not see a return. (Unlikely to happen.)
- Investors will strong-arm infrastructure projects into restructuring
into a for-profit model. (Unlikely to succeed.)
- People other than investors will have to step up. (Plausible.)
(Note: Return means capital return, not "social good" return.)
For the record, I contribute a little here and there to open source projects (usually in the form of vulnerability reporting) and started several FLOSS of my own (including several cryptography projects for PHP developers to use in their projects).
To date, I've made $0 from any of it, and I don't see that changing in the near future for me.
Like youtube videographers? Who start off doing it for fun, and then later monetize it?
That evidence shows you're wrong.
It works for ISC. Why wouldn't it work for OpenSSL?
Heck, it works for me (FreeRADIUS). Why wouldn't it work for OpenSSL?
> You seem to be ignoring that there are profitable open source companies, the prime example being my employer, Red Hat.
From my post:
> Most open source infrastructure tools lack a business model to capture revenue from its users.
Can we please not put words in other peoples' mouths? Kthx
But this was strictly a "toss over the wall" kind of deal. No community edits or changes ever make it upstream. Is that the kind of thing you are talking about, or would you envision a structure where improvements were re-absorbed into the borg?
Social media has replaced the really terrible feelings. When's the last time you were sad and alone for an entire day? Likely never, you can pull out your phone at any given second and plug those feelings, and you'll stop there and never realize what forces those feelings really have. It's the forces of human despair that drive us to do big things. Social media drowns out that force. You can plug any sad moment with a bit of social media and bring yourself from 'sometimes really sad' to 'never sad but always mediocre'.
I'm not the best at articulating ideas but I hope you know what I mean. We are slowly all becoming the same person in a different shell and it's terrifying.
The open source projects that people/businesses really care about are funded. Isn't a non-funded open source project just one an open source project that no one really cares about?
I think there is a really subtle line that she has missed by assuming all open source projects developers use are really important. There probably should be at least three categories: (1) really important projects (funded / managed), (2) projects that are not really that important (not funded / managed by community) and (3) those that are not important (not funded / not managed).
It's one that no one with substantial spare cash cares about.
Granted, it's broadly understood that people without money aren't actually people, but it's considered gauche to say so out loud.
That would stretch my argument to encapsulate a much broader topic than was intended. You would have to make a lot of assumptions both about me and my argument to do so.