1,037 karma · joined February 19, 2019
If you are interested in partitioning in an OLAP scenario, this will soon be coming to pg_analytics, and some other Postgres OLAP providers like Timescale offer it already
The problem is, at scale, Postgres isn't the answer to everything. Each of the workloads one can put in Postgres start to grow into very specific requirements, you need to isolate systems to get independent scaling and resilience, etc. At this point, you need a stack of specialized solutions for each requirement, and that's where Postgres starts to no longer be enough.
There is a movement to build a Postgres version of most components on the stack (we are a part of it), and that might be a world where you can use Postgres at scale for everything. But really, each solution becomes quite a bit more than Postgres, and I doubt there will be a Postgres-based solution for every component of the stack.
As for Delta Lake, we would've loved to do Iceberg but the Rust crate for Iceberg is very new, and doesn't have enough functionalities yet. We're tracking the project and planning to contribute, so that we can add support for Iceberg eventually
We store tables column-based. You can do CREATE TABLE (...) USING deltalake; when using ParadeDB and it will be stored column-based (and soon, with support for also querying cloud object stores!)
We expect production-scale ParadeDB to be used as a separate search node getting logical replication from your primary Postgres. This would give you easy-to-maintain and scale services while benefiting from isolation of services (so your search can't take down your primary database, etc.)
We would be open to what you describe and happy to work it out. But we are currently thinking of prioritizing separate hosting of ParadeDB. It's early, though, and still unconfirmed what customers will prefer :)
I'm surprised nothing was mentioned around pgBouncer/connection pooling. Having more native support for general connection pooling has been a requested feature in Postgres for a while, and many popular DBMS have it built-in. Just how some of what Patroni offer should be better bundled in (as the author argues), I think the same could/should be true for pgBouncer
We don't believe AGPL should prevent you from self-hosting for your own internal usage. We see ourselves as becoming a Postgres-based Elasticsearch alternative, primarily used internally for observability, etc.
If you are interested in using ParadeDB to power your own product, or would prefer a different license, we'd love to hear from you. You can reach me directly at phil@paradedb.com.
We spent a lot of time evaluating which license to use, and decided to follow the footsteps of Citus Data and go with a standard OSS license while ensuring that we can't be hosted on public clouds without a partnership. Hope this helps!
That being said, we understand that for startups it might be easier to have pg_bm25 directly within their production database, especially if data volume is small. We're actively open to working with other Postgres-as-a-Service providers in the space to make this possible :)
In the meantime, you can self-host ParadeDB on Scaleway directly by running the Docker container. Hope this helps!
Of course if you are 100% attached to AWS RDS itself (rather than the convenience of AWS RDS, which is replicable by ParadeDB), then there's not much we can do here, as we also need to eat :')
Other people have brought up great points for why or why not to switch. Our vision for this is that ParadeDB is not merely "better" than Elastic, but rather different. Elastic will never be a PostgreSQL database, and we'll never be a NoSQL search engine. If you want one or the other, you'll pick either ParadeDB or Elastic.
In fact, we went through much questioning wondering to go with ELv2, Apache, AGPL, etc. before settling on AGPL