Has the investment into bespoke storage technologies and data centers paid off already, given the real costs to build and set it up, and opportunity costs for other uses of engineering time? How long did it take to pay off through reduced costs, or when do you expect it will in the future? It would be interesting to learn at what point it might be worthwhile for a large player in a space like file storage to switch from existing commodity cloud solutions or proprietary on-premise solutions to custom ones -- as an extrapolation basis for other problems.
The general implication behind my question is: a custom solution can provide lower cost when viewed in isolation on a per-unit basis, but the total costs to the business must also account for the engineering work to develop, troubleshoot, deploy, and migrate to the storage system, as well as the time needed to lease and build data centers, etc., as well as to operate facilities and software stacks long-term; and to hire people to do that specifically; and also take into account the other things one could have done but cannot due to the decision to focus on a given problem (e.g., if the investment purely lowers cost, then that might be a tradeoff compared to another project that adds differentiating features. I know your blog post also mentioned performance, so it's not either-or) To make a good decision one must also weigh the expected value of alternatives, e.g., the current and predicted feature-sets of existing cloud or on-premise storage technologies, their expected prices over time, their expected regional availability. There's also the complication of owning substantially more assets rather than having operational expenses on cloud providers or other third parties.
I'm just curious to learn more about how you think about navigating the decision space. Of course, there's significant strategic value to "owning one's own destiny", but it also seems to have required significant investment and attention from Dropbox engineering team for perhaps a few years.
Any general advice regarding how big your use-case needs to be before this kind of engineering is worthwhile? How do you analyze the situation to identify when that's the case -- how do you identify the inflection point with confidence? Was it your prototyping that helped you determine that you could execute on this approach?