I spent two years trying to do what Backstage does for free | Hacker News Reader

I spent two years trying to do what Backstage does for free

stackoverflow.blog

284 pointsdanwee102 comments

My parents owned anf ran a jewelry store for 35 years, nearly their entire working life. My Dad was a master jeweler, goldsmith, sculptor. Among many things, they would string and re-string pearls for folks. Usually family heirlooms that they wanted to keep, but also be able to wear and not fear they'd break and fall all.over the floor. After years of doing that, and also buying raw oearls from overseas, stranding them, and then selling them, my Dad asked the supplier, do yoi guys do stranding? Yes, yes they did, for free. He could have been ordering pre-strung pearls for the same price the whole time. Stranding pearls takes time.

One of the questions I like to ask contractors before beginning work is "what changes would make this cheaper/easier for you?" Often simple things like moving a planned wall an inch or switching the location of something can make it vastly easier; or there will be things they can get preassembled or done for free.

And sometimes the factory building whatever has a giant machine that can easily do what would take you quite a long time to do by hand.

I used to work in a place that assembled machines based on old designs that rarely got reviewed, since if it ain't broke don't fix it.

When working on one particular unit, I found that they would order a bunch of plastic panels, cut to size, and some with precut holes. And others with secondary drawings, instructing us how to drill out a bunch of holes ourselves. Sometimes not even all the holes, just a few, in addition to the precut.

When I asked why we didn't just ask the supplier to cut out ALL the holes, I basically got a shrug.

So I merged the secondary drawings and sent them to the supplier, and the assembly team never wasted time on that nonsense again.

It’s funny. They seem to be two kinds of contractors out there. Type one will push as hard as possible for you to design things in a way that makes it easier for them. To the point where there are all sorts of design compromises like a beam in the middle of your kitchen. The second type is the one who wants to deliver exactly the vision of the client, to the point where they are adding a huge amount of complexity to perfectly meet unreasonable demands.

This is usually where the general contractor or the architect (if you don't have one, then that means you are that role) steps in to help insure the project is as it should be.

In my experience, contractors will build exactly what I ask for without challenging unimportant details that would have made things easier. My interpretation is that because they have so much internalized knowledge, they assume that I obviously know many of the same things that they do, and that I have already considered any such changes and actively decided against them. I've since learned to do what was suggested above.

It's been a valuable lesson for myself to question just how much of my internalized "obvious" knowledge about technical matters is apparent to clients and partners. Never assume.

Also, they can charge more if it’s harder (usually), as long as it’s obvious to other contractors it would be hard too.

If the low bidder gave that bid because they didn’t notice, then also lulz - tends to weed out the problem children. Really painful for the customer and said contractor though.

As an example of the opposite of this...

We had a pond put in a number of years back. The guys that did the work laid out what they would be building, showed us everything, and we agreed on it. While digging out the earth for the pond, they ran into a boulder not too far underground. They spend about half a day fighting with the boulder, seeing if they could break it up, etc; whatever they could do to get it out of the way.

When we went out to chat with them and see how the work was going, they told us about the rock and how it's being difficult. We told them we'd be fine with them changing the shape of the pond to just work around the boulder. They were happy about this because it made their lives much easier.

The cost of the creation of the pond was set before they started work on it. They gained nothing by trying really hard to create what they said they would; other than the fact that they did a good job and had happy clients. But they spend the time trying to deal with the problem without any real complaints (the talk about it being difficult was a casual chat, not them coming to us to complain).

They are obligated by law to do what they said they would, even at their cost, if above and beyond what they expected and not covered by the contact as a cost-plus, or risk losing their contractors license and being civilly liable. At least in the vast majority of US jurisdictions.

You saved them a lot of hassle by being flexible, but were under no obligation to be.

Smart contractors build in the odds of this happening in their contracts. Dumb ones expect everything to go right all the time, and then try to run away or cheat when it doesn’t. Or lose their shirts.

My point was that they didn't try to convince us to let them do something easier. They didn't even talk to us about it until we came out to see how they were doing. They weren't "happy taking the harder, longer route because it meant more money for more time". I was pleased with them and would recommend them any time.

I think you’re misunderstanding?

That decision would have had to be before they signed the contract.

No one knew about the geology.

At the point they did, it was too late.

It was great they didn’t try to avoid their responsibility, many would have!

> They are obligated by law to do what they said they would, even at their cost, if above and beyond what they expected and not covered by the contact as a cost-plus, or risk losing their contractors license and being civilly liable. At least in the vast majority of US jurisdictions.

This is technically true, but misleading. They have to follow the contract, but the contract likely includes exceptions for unforseen issues. And while you can technically sue a contractor for breach of contract the only reason to do so in this situation would be spite. The rational choice if a contractor cheats you on a small project is to be sad and move on. (Google around and you'll find lots of people who learned that the hard way).

I’ve learned to try to hedge against this kind of thing with language like “I’m no expert here, so let me know if you have any ideas on a better way” or the like. But yeah it is a good reminder not to make those assumptions when speaking with others.

This is often called “The Curse of Knowledge”.

https://en.wikipedia.org/wiki/Curse_of_knowledge?wprov=sfti1

I didn't know this had a name, thanks!

There's a quote along the lines of:

"An amateur will build what you want and an expert will build what you need."

Yeah, but the problem is that there are too many customers who refuse to believe that they need what you tell them and then blame you when you deliver what they asked for.

Yes, the quote could be updated in a couple of ways. I like "An amateur will try to build what you asked for, a professional will deliver it, and a psychic would have built what you want".

Caveat: Sometimes what the client needs is a cheap and low-quality solution. Sometimes.

An expert should be able to deliver that too. It might require outsourcing the task to make it cheaper, though.

Similar saying in engineering:

Any idiot can build a bridge that stands, but it takes an engineer to build a bridge that barely stands.” ... “The fewer moving parts, the better.

There's a Seinfeld episode about the latter.

I worked at a custom millwork firm and had a knack for value engineering projects. After making the switch to SWE, where I assumed everyone knew how to do this, I've realized that there only seems to be two kinds of SWEs as well, exactly as you've described. It turns out that value creation is hard.

There are plenty of contractors who know how to deliver an optimal solution and who make incredible money.

> It’s funny. They seem to be two kinds of contractors out there.

One way to view it is that they're each targeting (deliberately or unconsciously) a different market-fit, with respect to cheap/good/fast.

For some jobs you might want the advice of the first "compromise this and it'll be cheaper and faster" group.

It's funny how that works. I have a small electronics/firmware consulting side business. I used to make up wire harnesses for a customer, buying the wire, cutting to length and stripping insulation, crimping on connectors, etc. It was slow and my quality wasn't the greatest.

Then I looked around and found a company that would provide me the correct length of wire cut to size with the right terminals crimped in place all for less than I could buy just the bulk wire in the first place with no setup costs. Minimum order, IIRC was 100 pieces, which is peanuts.

Blew my mind!

At least I only struggled for a month or two before realizing that there had to be an easier way :-)

I am becoming increasingly skeptical that good internal documentation is even possible. I've been working in software development for around 12 years and have _never_ seen it done well and asking around the best I've heard offered up is the equivalent of an internal stack overflow.

For a while, I thought that maybe an exception would be having a technical writer on staff but after reading this post I'm significantly disheartened on that front too. I'd be interested to hear if anyone on hacker news has experienced good internal documentation and even more interested if any of you folks have experienced anything truly _great_.

For my contribution, I've found that documentation fails for a couple of reasons. The first is the burden of correctness. The people who most would like documentation are also the people who most need the documentation and they usually are also the people most likely to be reluctant to contribute to the documentation because they don't feel they can accurately represent the information. Imagine someone ramping into a feature and spending a few days reverse engineering how it works, collecting info, etc. Sometimes they'll put it up somewhere but a lot of the time contributing partial information feels 'wrong'.

And the second bit I find to be a big reason why documentation efforts fail is just the sheer friction of putting it into the documentation store to begin with. In confluence, for example, if you have a bit of information it can be tough to work out how to categorize it, where in the hierarchy it should go, etc, etc. Or if it's a GitHub wiki you want to put it somewhere that it is discoverable but also be careful that it's 'correct' because you don't want to break backlinks if it gets recategorized.

I've mostly given up on it at this point. Instead, I take detailed personal notes and make them publicly available. It doesn't have to be correct because being advertised as personal notes means that it's my opinion on the truth rather than objective fact. It isn't far away from my codebase, I can just tap a short keybinding in my editor to type my notes or search them and I can link directly from the notes to lines of code in files to jump back and forth. The particular system I use means that if I write a short snippet of code to solve a one off issue (like calling a path helper to derive a URL that I can't find in the interface) I can even drop it in a code block and execute it right from my note-taking tool. It isn't ideal for sure but I've gotten way farther in having a shareable knowledge base this way than I have in literal years of trying to get a shared, useful documentation store spun up.

I like this approach. What is this note-taking tool? I want to use it. Also linking to code from the docs sounds really nice.

It's org mode in emacs with the org-roam plugin. If I wasn't using emacs I would probably have a folder full of markdown files instead and leverage a project-based grep to search it. The real power in the system is that it's close to your editor and personal so you can iteratively build up your knowledge.

In particular, every GitHub wiki belonging to an organization or a public project itself is a git repo of markdown files that you can clone and commit to. I probably would use that for per project documentation w/ a separate folder for managing broader notes. I think it's important to separate the markdown from a project's code because otherwise you can have important markdown updates trapped in a branch that hasn't been merged yet and because git works by line rather than by word you can get into complicated merge conflicts by features that touch similar business concepts.

Org mode happens to add some niceties on top in that you can have an example code block (roughly comparable to a triple back-tick code block in markdown) that also can be executed. The example code block can take some arguments like what interpreter to use and will paste the output into another code block underneath the source code block.

Because org mode is also used for organizing tasks it is easy to use it as a scratch pad as you grind through tasks and if something turns out to be useful you can promote it to a top level note (or org-roam node). And even if it's not obvious that it's a note candidate, just thinking through problems 'out loud' in org means you can search and find it later when the same or an adjacent problem comes up.

Another favorite feature I like in org mode is that in any code at any time I can tap out 'SPC n l' and it will capture a reference to the file and the line in the file and allow me to link to it in my notes. It doesn't capture by line number but by copying the literal line, which means that as feature updates push the line number up and down, it still will be able to find exactly the right file in exactly the right line as long as the line's content hasn't changed. It also serves as a rather nice canary because if the docs link to a line that no longer exists, then the documentation has probably gone stale.

The last major win with org mode vs markdown is with org-roam, which borrows the ideas from the standalone program Roam Research (which also is very similar to notion). Every 'node' you make initially is a file and you can fill it in as a wiki type structure. Everything is flat on the filesystem and has a unique id prepended to it so you can't overwrite it with another node of the same name. There's also a UUID associated with each node so if you move from one place to another, none of the links need to be updated. Contrast that with a regular wiki or markdown where when you move a file to a new folder you either have to leave a redirect behind or go through and update all of the back links.

You can also start a node as a subheading and later on promote it to a full file with all the backlinks still working correctly.

I started using org mode years ago with spacemacs, got frustrated with spacemacs, and gave up until earlier this year when I found doom emacs and gave it another spin. I wish I had a better recommendation for an alternative but after looking for 4-5 years for something as good as org mode, I could never find anything. Notion seems like a popular alternative but there's also a lot of shiny bells and whistles in it that can cause 'productivity procrastination' where you spend time configuring and turning knobs instead of actually using the system. Ironically, the second best tool I found was plain pen and paper using a minimal bullet journal technique and taking special care to do indexing. Of course, you can't share plain pen and paper in that way, but the name of the game for personal use is low friction and ready access. The better a system scores on those two in term of note-taking the more useful it will be.

I don't know about the GP, but I wrote a simple script that will make a mdBook site out of Joplin notes:

https://gitlab.com/stavros/notes/

This is why IMO in any nontrivial organization you need a dedicated technical writer - if only someone that sorts out the wiki and keeps it clean.

In my future perfect world, every artifact, every service, every everything will have a TTL. Expired stuff gets culled, in stages.

The issue is that people seem to equate "good documentation" with complete.

It's just not possible to do this. I think it would be better to talk about "effective documentation".

My opinion on the matter:

Three levels of documentation

1. high level documentation that describes the problems and the goals from the business perspective

2. mid-level documentation that describes the architecture that gets us there

3. low-level documentation, aka code.

high level gives you direction (where), mid-level gives you context (what), low-level gives you implementation (how)

You need the what to understand the how, and you need the where to understand the what.

The other piece is an acceptance that you can't document away the need for tacit knowledge.

To draw an analogy.

No amount of documentation is ever going to allow a kid to hop on a bike and ride it perfectly the first time. That is not what it means to teach a child to ride a bike. Instead the goal is to have enough documentation to minimize the amount of time it takes that child to gain the tacit knowledge necessary to ride a bike acceptably.

Once you accept the above and lower your bar, "effective documentation" becomes much more achievable.

yes. and the hard part is keeping the three levels in sync.

One of the reasons this is hard is because it's not that easy to really separate the three levels of abstraction. It's very difficult to state the business requirements neutrally, without coloring in any technical sort of solution (especially since the technical solution is usually what the customer asks for), and it's equally hard for many people to write a high-level technical design without filling in (or assuming) various implementation details.

In my team, I have started spelling out more concretely what I expect from the documentation at various levels. Keeping with the three levels specified by the GP, we always document:

1 - business rationale: what are we building, and who are we building it for (enumerate the stakeholders and their processes)

2 - component diagram: a graphical sketch of what different building blocks are involved (datastores, api endpoints, webservers, etc), and showing the direct connections between them

2 - data flow diagram(s): showing entry, exit and storage points for each class of data moving through the system

3 - starter documentation: what does a developer need to know/do before working on this project? This includes tooling, pointers to outside documentation, and a reading guide pointing to the rest of the project's documentation.

The rest of the low-level documentation is the responsibility of the team itself.

By insisting on documenting the second layer mostly graphically, there is less potential for overlap between the different layers. Of course, no documentation system is perfect, but I find the above has suited us quite well for what we do (we write business integration software, no huge client applications or software suites. YMMV, obviously).

I love this separation of aspects and the graphical representation to nudge out overlap.

How do you keep the layers consistent? How do you keep track what update to some layer is not covered in some other layer yet? Is that visible right in your layers? or some outside backlog?

The simple answer is that we try to keep most of the documentation (layers 2 and 3) in the same repository as the code itself, though that in itself is no guarantee that the documentation is up-to-date. Layer 1 documentation mentions explicitly which changes are part of which project, so the project timeline (which is external to the development team) should show if a mentioned project is supposed to have been completed already.

There are no airtight rules other than being mindful of your own processes. Our workflow generally looks like this:

Changes to level 1 documentation are done by the product or project manager in preparation for the work, so it's pretty easy to make that part of the documentation lead, rather than lag, the implementation. It's stored in the project wiki, rather than with the code. But that wiki is still git-based, which means we have an easily auditable changelog to check when a particular piece of information was last updated.

Layer 2 changes are usually done by (senior members of) the development team, but it's done in the week before the implementation starts. Since we use those diagrams to communicate with the rest of the development team about the changes required, the documentation updates happen rather organically.

I find that layer 3 documentation updates tend to be the hardest to capture in a process, because the code itself is much more in flux than the higher layers. What helps us here is that our developers know that the codebase they're working might not be worked on again until months in the future, so it's in their own interest too to keep the documentation correct. That said, we do have an explicit check for documentation updates in our project delivery checklist.

The most challenging situation is when we discover during development that an architecture redesign is required, but most of the time an architecture adjustment also means we need to re-evaluate the workload and possibly the delivery date, which basically means our process resets and we re-involve the product or project manager. Funnily, this then usually results in more documentation rather than less...

We sometimes also have a 2-person development team for small changes (never 1 person though), in which case all three layers are handled by the same people. But because they're updated in different stages of the project, and because they're already used to the same workflow in larger teams, the small team size doesn't affect the documentation quality as much as I initially feared.

I've never seen it done properly either, but I think it's more a matter of incentives than structure or tools. The places where it was worst were places where people's reaction to wrong documentation tended to be a shrug.

Sometimes there were people who didn't document coz they kind of liked being the font of all knowledge.

Confluence also sucks - even harder than jira, if that's possible.

Well the real problem is: What's good documentation?

I'll settle for examples.

Documentation is a multi-headed hydra where you have different levels of technical competency and foreknowledge and familiarity with the programming environment patterns in use the organizations, particular technical stack organization, the people inside of it, what the product does, etc etc etc

I'm convinced that the only good documentation is generated from code or config, and has a source which is necessary for the system to run.

A tool that tells you who commits most frequently to a service or file is better than an outdated yaml file with an owner. A firewall that won't let your service make outgoing requests unless you list its dependencies is better than a manual list of dependencies. An API doc compiled from the type definition of your API or from a schema which requests are validated against is better than a manual API definition.

My company - roadie.io - offers SaaS Backstage. We're the second largest contributor after Spotify and we're co-hosting the first ever BackstageCon at KubeCon NA this year.

Get in touch if you ever want to discuss Backstage. Email in bio.

No pricing on your page? Are you “contact our sales team” expensive?

https://roadie.io/pricing/

? There is transparent pricing from 50 up to 150 developers: roadie.io/pricing/ It's linked directly in the header.

For deals bigger than 150 devs, we have found customers prefer to talk to sales.

FYI: the link to the pricing page doesn't appear in your mobile menu

Wow you're right. That's a bug. Thank you!

FWIW, it would be great if you had a breakdown of the TCO. The price is at a point where many orgs would have some suit decide "we have a DevOps guy, we'll just run Backstage ourselves and save $27K/yr". I think it's worth the money but you need to show the suits where that money is going, what kind of value they're getting from having you run it, and consider the risks (what if it gets compromised, goes down, etc). Also, I don't see SSO mentioned; you really want to mention that, it's pretty important for enterprise.

Thanks for the feedback and fwiw I completely agree. We can do much better in this area. We're a team of engineers so we struggle with marketing and copy.

SSO is supported on all plans by default.

The main problem with Backstage (BS) is that while it does provide some canned functionality, any extension of that functionality requires the utilization of their plugins system and wholesale buy-in to their ecosystem. Welcome to Wordpress for enterprise developers folks!

The plugin system isn't bad - but its a false narrative to believe you plunk this down and get some sort of valuable developer portal at an enterprise level (small biz sure - fortune 500...yeah no).

Spotify does open-source a number of their plugins, but after having looked extensively at which ones they make available and which ones we would have to write (and deal with their plugin system for) we opted out. The cost/benefit ratio is definitely there for a technical writer, but IMO its not there for serious development. (Pls - Anyone who disagrees with that - begin by telling me why you don't use Joomla as a base for all your custom applications )

Team integration at a company level is another challenge with Backstage. GitHub (the basis and sort of harvested DB for BS) is largely disconnected from company structures and AD and all the things for most companies. Yes GitHub enterprise is a thing... no in general its still not connected to company structures. There are teams in Backstage, but they are generally managed in isolation from the rest of the company which presents its own challenges and integration problems with other systems. Don't even get me started on the fact that asking everyone to put their service defintions in specific files in specific places in GitHub is no better/worse than making them enter it into a ui or spreadsheet or confluence article or anything except now its scattered in 10,000 repos - good luck getting everyone to change that.

Ultimately this isn't software that you simply install and use. Its a base that will likely require extension, and doing so means going down a one way path of adoption. Nothing it does is compatible with anything else. Is that good for Spotify - sure. Lots of folks helping build software they want to use is great. Is it good for the community? I'm not so sure. This article felt like a paid add (not suggesting it was - just saying it read like one) more than an honest plus/minus of the software and is anyone really surprised a technical writer couldn't build a proper enterprise developer portal by themselves? In other news - water is wet, the sky is blue and my kid wants to watch youtube.

So what do you recommend then instead?

How about just using github? or if you want more how about building a a node/vue app? Or for docs scrounge up any static doc creation system and thereyago. most have some really editing features and BEHOLD markdown with colors!

I'm being a little silly there but my issue isn't that this isn't valuable software in the right situation. My issues is that its presented as some sort of magic bullet for a specific documentation issue when the problems described are generally not software related or software solved.

Most of what is presented in the article isn't about BS doing what this person attempted to do for 2 years. Its about switching paradigms and letting folks document in their repos instead of in a centralized system. One could just as easily look over the repos in a github org and go to those files in a repo. Both solutions require standardizing where things are documented..... so does a centralized system.

The big callout should be that when trying to centralize docs that align to apps they become disconnected from the apps themselves and thus its a pita to get everyone to do everything in 2 places. I'd be very curious if they completely turned off BS if there would be any loss... if not why bother with the overhead. Templating and other things are well handled by MS Clis, cookie-cutter or other more portable concepts and who needs yet another team and resource mgmt system. Splunk and other off the shelf systems can easily provide more comprehensive and robust stats and generally speaking there is a more enterprise level solution for just about everything it does.

Yes BS is a swiss army knife but when was the last time you saw a chef use one of those in the kitchen?

Oldschool solution that I have used many times is to setup an oldschool internal Wiki (can be hosted on anything). This gets used as an internal resource for everything (requires buy in from the right people obviously) and is actually quite useful. Very easy to get started and very easy for people to start creating entries.

Our companies evaluators picked up Cortex for a service registry offering because it did more for them faster, but they may switch over to backstage in time if they find the integration straightforward enough

Indeed, Backstage is a framework for creating a developer portal, not a ready-made product.

I used to work on a consultancy (https://frontside.com) who's specializing in adopting Backstage at Fortune 500s like HP. When you adopt Backstage at scale, you rely on processors that scan your LDAP, GitHub, etc to discover, stitch, and feed cohesive data to the software catalog. HP is giving a talk next month on how they manage 100,000+ entities in Backstage.

Backstage does a lot of heavy lifting that I'd rather use than have to build from scratch.

The real gem highlighted by backstage is the MkDocs Material theme: https://squidfunk.github.io/mkdocs-material/ It's seriously awesome and has a lot of nice customization options. Out of the box it makes very clean and professional technical docs from a pile of markdown files. Even if you don't need to go the whole backstage route, seriously consider using mkdocs and the material theme.

This is great, and I think if it was fully adopted initially in greenfield, this would be a completely feasible solution. My question is though, these problems exist at large organizations and will be seen as "another tool to rule them all" synonymous with things like "sharepoint". Disclaimer: I'm not saying that backstage is anyway comparable to sharepoint except that it presents itself as this grandiose solution to all things on boarding and learning on your own.

With that said, what's the low hanging fruit to entice adoption of backstage into an organization that already has a "process" like mentioned in the article of whiteboard meetings with stakeholders. Obviously that's not a great use of time, but it's also no extra learning or work, that SME can talk about their function without any preparation and can interactively answer any questions the learner asks at the time.

That doesn't scale in practice, but it also doesn't require any more whole team buy in

You are right, this is not going to be trivial and you definitely need buy-in on a management level. Backstage is pretty malleable but the only way to get it to work in an organisation if there is only a single source of truth for things like your software catalog.

If there is a lot of legacy in your org, or if there is not enough buy-in then it will definitely be "yet another" system. There's a lot of functionality to unlock once you have a bunch of data in Backstage, though, so that could help tip the scale. The cloud costs dashboard that helps to show how much every service costs is a good demo :)

One benefit to Backstage's approach is that you don't necessarily need to change your processes and tooling to work with Backstage. It can help, but it's not a strict requirement. Other tooling in the space requires you to buy in to their specific workflows and so on whereas Backstage is a bit more agnostic.

> So when I had questions, I needed to track down who owned the service, where their code lived, where their Jira ticket project was, which Slack channel they lived in, and where in the wiki their internal documentation was. Keeping track of this for 100+ services was a pain, so I ended up creating a spreadsheet. It turns out that everyone else in the org needed this information, so this spreadsheet that I created for myself became the document of record for services.

100%.

Solving this problem is why I started OpsLevel [1] a few years back. Everyone always starts with a spreadsheet because they're flexible.

But spreadsheets quickly fall apart because they're not complete / up-to-date, automated, or really scalable.

We figured we could automate a whole lot of collecting service info from different places (Git, k8s, other deployment streams) and keeping it up-to-date.

[1] - https://www.opslevel.com

I got interested in OpsLevel immediately upon reading your comment but was disappointed to find out that there is no public pricing and I have to contact for a demo. That goes from Cloud SaaS to Enterprise territory. Maybe some day we will grow enough to give you a call but your post made me realize we should definitely look beyond spreadsheets to document and manage our infrastructure. Thank you and hope you open up your pricing and trial accounts in future.

Thanks for that feedback re: pricing and demo.

It's always a tricky balance, but I definitely sympathize with the feeling of just getting started without having to talk to a salesperson.

Unrelated to the discussion, just wanted to comment on Backstage's tagline:

“Build an ecosystem, not a wilderness"

which is ironic because a wilderness is arguably the most complete and perfectly functioning ecosystem you're gonna get.

i like that thought. What would be a more ideal pair then?

Build a wilderness, not a parking lot

"Build an ecosystem, not dystopian sprawl"

Create a garden, not a wilderness. OR (depending on your audience) Build an ecosystem, not a garden.

An ecosystem, not a Superfund site.

Build an ecosystem, not an experiment.

Build an ecosystem, not a science project.

Build an ecosystem, not a microbiome.

"Build a universe, not a multiverse" could capture the meaning pretty well, in my opinion.

It's hard to infer from some quick documentation-reading whether kubernetes is required to enjoy a lot of the described benefits. Feels like a big first step: "Adopt kubernetes." Having been at a medium-sized org leaning into kubernetes, I can see the draw of a system like this, especially for non-IC roles.

No, that's not needed. There's a K8S plugin for Backstage that can link your deployments to the services in Backstage's service catalog, which is useful. But that's pretty much the only part of Backstage that would "require" Kubernetes.

Isn't the only thing required for Backstage a GitHub account? It's been a while but I remember GitLab not having the same feature parity and OSS alternatives like Gitea being completely left in the dust. Kinda disappointing if this is still the case, that an OS tool requires corporate owned software to use but I suppose life is full of jokes.

No, there's no requirement for any kind of source control at all. If you do have source control in your company (and, if so, welcome to the 1990s!) you can set it up in such a way that all your repos get scanned for metadata that gets ingested into the software catalog. There's implementations for GitHub and Gitlab (I built that one at $PREV_JOB) and others. Also, other kind of metadata, like user and group information, can come from sources like LDAP, Azure AD, etc.

To be clear, Backstage is not an alternative to Gitea, Github or anything like that. It's not a source code repository hosting tool.

I also really dig the approach of Backstage but found maintaining the installation a real pita. They even give you a tool that tells you which lines have to be manually updated in your installation when updating (https://backstage.github.io/upgrade-helper/).

This is exactly why we created https://roadie.io I'm the founder.

Agreed. We've seen this with our customers at OpsLevel who lean away from Backstage. Installing (and updating) Backstage is painful.

One of our team members who previously ran a Backstage instance at $OldJob wrote about that pain:

https://www.opslevel.com/blog/hands-on-backstage-learnings

The problem isn't really installing backstage but integrating all the stuff around it.

It is not a turnkey solution per se and what they call plugins aren't plugins in the traditionnal sense where you would just download a package and put it in a directory. It requires editing the actual react code to set most of them. So maintaining backstage involve maintaining a set of patches and a pipeline to build a docker image.

(Shameless plug) Cofounder of OpenContext.com here. We're a bit different than a developer portal or services catalog, we're a complete context graph of all the assets in your product pipeline. Everything from products, documents, microservices, codepaths, vulnerabilities, people, teams, etc and the relationships between them. The complete context for DevSecOps. Check us out.

No need to apologise, a polite "I ran X which solves Y" related to the post is encouraged. Helps the rest of us learn about other solutions out there :-)

Will checkout OpenContext, thanks for the link

Has anyone done this sort of thing for data, not just for services?

I can't seem to find a data catalog that lets my tie information across databases + APIs + Kafka topics. Each catalog of schemas is separate with no way to link them really.

His org used jira but not confluence? Why couldn't they just document these things in confluence?

Edit: This is a serious question. What is that value prop of backstage over confluence?

Structure of the data and more pleasant/immediate workflow increases adoption so it doesn’t go stale, like Confluence so often does. Some teams need those nudges more than others due to personality, turnover, speed of growth and complexity of service architecture.

backstage has plugins that can help with autodiscovery of a lot of this stuff.

The stuff that's not autodiscovered can live right next to your code, in any repository, so devs can just update some YAML keys when they're making other changes.

You can lint the backstage changes as part of PRs

Also I've never ever seen developers consistently keep stuff up to date in Confluence. There's just so much friction in that interface.

Auto-discovery does sound appealing. Thank you.

Author here. We did use Confluence, but as other folks had said, it went very stale. Not everyone could edit everything and there was no structure. Confluence is a mess in my experience.

Have to agree. Without a clear set of standards on how to structure things, it ends up being chaos very quickly. We had a PM for a while, who gravitated more towards QA, and when they got ahold of Confluence they went crazy (in a mostly good way). We had so many great "How To" articles on how to perform various tasks in our system form an internal user perspective, all authored by the same SME.

Unfortunately, given their affinity for testing things, the How To docs starting taking on that QA perspective and everything ended up in a QA "space". Sure, it's easy to move them around, but there wasn't a clear cut way on how to separate the "How To" from the "QA" - and then they left.

Since then, people add things when managers say "make sure that's in Confluence" and it ends up getting placed in the areas they're most familiar with, not necessarily where it should be. Add in the aforementioned lack of standards and no two documents look the same. Then we started using Confluence for devs to write Test Plans and QA to execute them - and now it just makes my brain hurt whenever I have to go in there.

Some of it is a cultural problem: we're email and Teams heavy and so much of our "documentation" is buried in those tools. So then we adopt something like Confluence to move our knowledge base to that, but finding stuff in Teams is a nightmare (posted in another thread on that). So now rather than having one tool to rule them all, we just have another half-baked attempt.

We're monolithic for the most part, so I'm not sure Backstage would help us, but the article mentioned Stack Overflow for Teams, so maybe that would? (insert relevant XKCD here on competing standards)

Also Confluence and JIRA end up weirdly competing, with information spread between the two, sliced up chronologically in JIRA, and based on some a-priori and/or competing sets of structurations...

It doesn't help that Confluence has one of the most irritatingly useless search engines in existence.

You would think a writer would know how to write a less grammatically ambiguous title.

Is there a term for this phenomenon, my dear Watson?

Backstage is a well-known entertainment-industry site and publication. What's the motivation for trying to duplicate it?

They're... not? "Backstage" is a noun. It's a place. It so happens some entertainment industry publication uses it as a name, and a piece of software that is totally unrelated also use it as a name.

It's capitalized in the title, so it's a PROPER noun. And I've worked in software development for a long time, and have read HN regularly for a long time, and I've never heard of "Backstage" as software.

There's a wide world out there, which some HN posters "forget."

Curious, I just started a project to integrate Backstage at a Fortune 500, with > 10000 repositories. It does seem like a good foundation so far. Their idea of what it’s supposed to accomplish internally is mostly the same as what this blogpost describes.

Anything you've learned worth sharing? How has your experience with Backstage been? Why did you choose it over other options? I might be working on a similar project soon.

Are there other good options?

Edit: Thanks everyone!

opslevel.com is worth checking out.

(Disclosure — I’m an investor.)

Thanks Andrew. (Disclosure: I'm a founder/CTO).

Backstage is super configurable, but requires that you build your own developer portal. That generally requires a dedicated team of people at your company to think about dev workflows to support, building plugins, plus hosting/maintenance.

At OpsLevel, we're focused on a much more turnkey solution. We want to get you up and running in under an hour, not weeks / months.

(Shameless plug) – founder of https://www.cortex.io (YC W20) here! We build an off-the-shelf developer portal. Backstage is great if you have lots of time/very niche requirements, but most of our users need a system that "just works" to drive ownership/best practices without adding a ton of overhead

We didn't have enough time for testing, but there's also Clutch, coming from the Lyft team. (Checking right now I'm not able to see a "releases" page with versions and that stuff...)

https://clutch.sh/

First time seeing this, but reading the docs it looks more like a modern take on Rundeck[0]. A way to give developers one-click access to operational tasks. Rather than a service catalog and developer portal like Backstage.

[0] https://www.rundeck.com/

We've just started using Atlassian Compass (https://www.atlassian.com/software/compass) for our software catalog, but it appears to be much less featured than Backstage. We're taking a look at replacing it with Backstage, and curious if anyone else has notes on the comparison.

Might find some near previous mentions:

https://hn.algolia.com/?query=backstage.io%2F&sort=byDate&ty...

Just saw this video where a similar platform is introduced https://youtu.be/Abdp_BnRIx0 This is the website https://kustodian.io/

It has more features including onboarding, access control, etc

It’s too early to say, unfortunately (which is why my initial post was not super useful, admittedly). All I can say for now is that, in all likelihood, a _lot_ of customization will be needed. Ask me again in two months :D

Reply on news.ycombinator.com