Thanks for mentioning us! We're currently running the Kafka 1.0.0 final release packages through our QA systems and hope to make it available for all Aiven (https://aiven.io) users next week.
208 karma · joined March 22, 2016
If you want to use Kafka and need disaster recovery capabilities we typically recommend using Kafka Connect or other similar tools to replicate the data to another cluster or persistent storage system such as S3.
See http://blog.aiven.io/2017/10/aiven-is-first-to-offer-postgre... for more information
PostgreSQL 9.6 added extension function pushdown as a new feature allowing functions to be executed on the remote side as long as the extension is registered with the foreign server. This avoids having to fetch all the data from the remote side to execute a function locally, see https://www.postgresql.org/docs/current/static/postgres-fdw....
Also, as pointed out in the parent post, some of the hosting providers such as us (Aiven, https://aiven.io) are directly involved in PostgreSQL development and one of the bugfixes in the latest releases (9.6.1 and 9.5.5) that just came out was contributed by us.
It could also be used to do things like collect metrics about the ratio of commits and rollbacks, etc. The current version of PGHoard doesn't have these features but we're planning to implement them in a future release.
Aiven services are available on UpCloud and we're friends with the UpCloud people as we're both based in Helsinki and have met at various events, but there's no other connection between the companies.
Database statistics are not preserved across pg_upgrade so you'll have to run ANALYZE on the database after the upgrade which may take a while as it usually needs to read all or almost all of your data.
I just recently upgraded a ~1 TB database, the actual upgrade process took less than a minute but analyzing it took much longer.
When PGHoard is used in streaming mode it keeps reading new segments from PG and stores them in compressed & encrypted form in a queue ready to be uploaded. The segments will stay there until they can be uploaded.
When using archive_mode PGHoard handles the operation synchronously so PG won't actually remove or recycle the WAL segment in question until the command completes.
Postgres will keep running normally in both cases, but the files will be queued in different places, compressed or uncompressed. This may cause your disk to fill up eventually, but PGHoard will trigger an alart after a configurable number of upload failures.