The Plan for InfluxDB 3.0 Open Source

We tried to do too much all at once with v2. Our goal with v3 is to bring the focus back onto the core of the database.

I said a bit of this in our community Slack channel earlier today, but I think it's worth repeating here:

I really would have loved for us to have more time developing Flux as a language, but we spent all of our time trying to optimize its performance. As a result, we weren't able to make changes to it that I think would have helped people understand and use it.

We pushed on it for 4 years and were told again and again by new users and people that chose other tools that they didn't want to adopt Flux and found it difficult to use. I realize that this wasn't universal and we had some real fans. But we still weren't able to bring the performance that most of our users required. At the end of the day it was just too much to try to do in addition to creating the core database and all the managed, hosted service around it.

I was told once quite some time ago that while Flux made the impossible things possible (i.e. you could do things you actually couldn't do in other query languages), it made the easy things hard. I think this was a correct assessment and that it's what caused the getting started experience to be so poor.

We knew this and wanted to improve it. I brainstormed with Nathaniel (primary Flux author & creator) around 3.5 years ago to make changes to the language. We did a lot of work and documented the whole thing: https://github.com/influxdata/flux/blob/master/docs/new_flux...

But we couldn't ever get to that work. We shelved it because we were constantly fighting performance and reliability fires.

With version 3 we built around an existing query engine (Apache DataFusion) and it's getting developed not only by us, but developers around the world. We think this will have huge benefits over time that lead to a faster query engine that has more features and is more reliable.

It's like Lastpass. No matter how obvious it is, people just won't move on.

I'm sorry to say I don't have much good to say about InfluxDB. I first considered them around 2015. I loaded about a week of logs from our OpenStack systems into it. It seemed to work so I included it in our set of internally supported systems and started going about the task of adding it to our backup schedule.

Well, perhaps things are different now but at the time the only way to do a full backup was to save the output from a command like "select * from *". Before adding that to a cron task I tried it from the admin console and quickly caused the server to OOM and die.

It was immediately clear to me that this wasn't a company that cared much about reliability since I was able to trivially crash it doing the only thing I could do at the time to backup my system. Obviously nobody at Influx cared about pesky things like backups or workload management, so I quietly decommissioned the system without incident and moved on to other things.

Some years later I watched Andy Pavlo interview their founder on one of the regular CMU database sessions. Andy was very critical at many points, effectively telling him "we told you this wouldn't work". I felt much the same way.

Influx looks very much like a company that may eventually build right thing after building everything else first.

Feel free to check out OSS Proton: github.com/timeplus-io/proton, and let me know any documentation/design need to be updated to make it a bit clearer

It blows my mind that they still have not learned their lessons with world breaking changes. How can you build anything on a database that is constantly reinventing itself chasing some new hotness?

Not that I’m going to miss it very much; in fact them going back to SQL in IOx immediately put brakes on my effort not only to further learn the language, but also any desire to write any further dashboards and left me feeling I wasted a bunch of time learning it as much as I've done up to that point. I just hope they don’t end up going back to Flux again in 4.0.

Ultimately, we're trying to narrow the scope of what we're developing and letting other tools that people already know and use have first class placement in the InfluxDB stack.

Community - Will provide a single node free to use offering which will replace the majority 1.x and 2.x OSS user workloads.

Commercial - Focus on enterprise features such as authentication and federated queries.