HNHacker News
TopNewBestAskShowJobs

lfittl

1,954 karma · joined October 30, 2007

Lukas Fittl

Founder of pganalyze - Postgres performance at any scale.

Feel free to drop me an email: lukas@fittl.com

[ my public key: https://keybase.io/lukasfittl; my proof: https://keybase.io/lukasfittl/sigs/rqDeEWdp4ktJubli4QY0dNaUJ6K-pdpDTHCFOi9v7WU ]

submissionscomments
lfittl··on Deconstructing the Postgres planner to find indexing opportunities
(author here)

Nice approach! The latter part of that workflow is where we are focused now, i.e. now that we can plan and recommend B-tree indexes for individual queries, how do we utilize this to recommend a set of indexes for a whole database (taking into account different predicate variations).

On open-sourcing this code: Not at this moment - we do however offer a WASM build of this logic for free use at https://pganalyze.com/index-advisor

lfittl··on Show HN: pganalyze Index Advisor – A better way to index your Postgres database
OP and founder here - we've worked hard on this for 6 months, and are releasing it today!

Based on the Postgres planner logic itself, we make index recommendations for your Postgres queries - directly in your browser. Supports B-tree indexes to start, with support for GIN/GIST coming soon.

I'll be around for questions/feedback :)

lfittl··on Ask HN: Who is hiring? (August 2021)
pganalyze | Senior Backend Engineer | REMOTE (US/CA time zones) | Full-time | Salary: $140-180k | https://pganalyze.com

At pganalyze, we redefine the user experience for optimizing the performance of Postgres databases. Our product helps customers such as Atlassian, Robinhood and DoorDash to understand complex Postgres problems and performance issues.

We are currently preparing a significant refactoring of our backend to better handle large amounts of metrics and logs data, and better derive insights from patterns over time.

For this role we are looking for someone who has worked with both Ruby and Rust before, and who is interested in designing a better Rust-based system to process our metrics data from the ground up. Having a solid understanding of statistics is a plus.

All details: https://pganalyze.com/careers/senior-backend-engineer

Interested? Email me (the Founder & CEO) at lukas AT pganalyze.com - or fill out the form on the website.

lfittl··on Ask HN: Who is hiring? (July 2021)
pganalyze | Senior Backend Engineer & Senior Full Stack Engineer | REMOTE (US/CA time zones) | Full-time | Salary: $140-180k | https://pganalyze.com

At pganalyze, we redefine the user experience for optimizing the performance of Postgres databases. Our product helps customers such as Atlassian, Robinhood and DoorDash to understand complex Postgres problems and performance issues.

Application developers use pganalyze to get deep insights into complex database behaviors. Our product is heavy on automated analysis and custom visualizations, and makes automatic recommendations, such as suggesting the best index to create for a slow query.

Our stack is Typescript, Ruby, Go and a little bit of Rust (with plans to grow). And of course Postgres! We're hiring for:

- Senior Full Stack Engineer: https://pganalyze.com/careers/senior-full-stack-engineer

- Senior Backend Engineer: https://pganalyze.com/careers/senior-backend-engineer

Interested? Email me (the Founder & CEO) at lukas AT pganalyze.com - or fill out the form on the website.

lfittl··on Ask HN: Who is hiring? (June 2021)
pganalyze | Deep Insights into Postgres | REMOTE (US/CA time zones) | Full-time | Salary: $140-180k | https://pganalyze.com

At pganalyze, we redefine the user experience for optimizing the performance of Postgres databases. Our product helps customers such as Atlassian, Robinhood and DoorDash to understand complex Postgres problems and performance issues.

Application developers use pganalyze to get deep insights into complex database behaviors. Our product is heavy on automated analysis and custom visualizations, and makes automatic recommendations, such as suggesting the best index to create for a slow query.

Our stack is Typescript, Ruby, Go and a little bit of Rust (with plans to grow). And of course Postgres! We're hiring for:

- Senior Full Stack Engineer: https://pganalyze.com/careers/senior-full-stack-engineer

- Senior Backend Engineer: https://pganalyze.com/careers/senior-backend-engineer

Interested? Email me (the Founder & CEO) at lukas AT pganalyze.com - or fill out the form on the website.

lfittl··on PostgreSQL on ARM-Based AWS EC2 Instances
When using pre-built Postgres packages on modern ARM platforms, like the author of this post is, double check that the Postgres binaries are actually using the ARMv8.2 assembly instructions.

This can make a significant performance difference, and at least for the RPM-based official Postgres packages this is a problem: https://www.postgresql.org/message-id/CACN56%2BP1astF5zvocrT...

lfittl··on Ask HN: Best Bank for Startups?
I assume you know this, but just to mention it:

For issuing cards, you can use Stripe nowadays. If you're doing this as part of your business (e.g. like Expensify is doing), its available to all US based businesses: https://stripe.com/issuing

The more convenient Stripe Corporate Card (which sounds like that may be your use case?) is still invite only: https://stripe.com/docs/corporate-card

(I myself am queued up for an invite on the Corporate Card, and using Chase Credit Cards in the meantime, which are okay, but not great)

lfittl··on Medium-hard SQL interview questions
If you really want to, the pg_hint_plan extension can be used for this - though I would use it very sparingly, if at all:

https://pghintplan.osdn.jp/pg_hint_plan.html

(available on some cloud providers as well)

lfittl··on Introduction to Hacking PostgreSQL (2007)
I agree re: your comment on hosted provider support. Extensibility is a major benefit of Postgres, and there is a lot more untapped potential (e.g. even just for custom data types), if only major providers supported it better.

FWIW, from having looked into this, there are two main challenges:

(1) Security - How to avoid the customer becoming superuser by using specifically crafted extensions. In practice this is mostly a defense in depth scenario, since you're running inside a VM already anyway.

(2) Maintenance / Support / SLA - If customers load a broken extension and that causes the server to crash, does the provider still give an SLA? Where to draw the line / how to evaluate this when support tickets get opened?

Both of these could be solved (I think) if there was a better sandboxing mechanism.

Really the issue here is that once you're at the C extension / shared library level, you can do pretty much anything, and thats really hard to support as a Database-as-a-service provider.

lfittl··on Effectively Using Materialized Views in Ruby on Rails
You typically do it on a schedule, e.g. once per hour, or once every day.

In the post the whenever gem (https://github.com/javan/whenever) is used, but you could use anything that is able to run things periodically.

If you'd want to stay solely within Postgres, there is also pg_cron (https://github.com/citusdata/pg_cron) which could be used to call REFRESH MATERIALIZED VIEW from within the database.

lfittl··on Similarity in Postgres and Rails Using Trigrams
(not the author, but I've discussed this with him previously)

This is a problem with `order` in Active Record in particular, where you can't pass in a bind variable the same way you'd expect it to work with `where`.

Therefore the slightly less elegant version that goes through quote_string. You could use bind variables by going directly to the pg driver, but I think there is no quick shorthand to do this in Active Record/Arel.

lfittl··on Announcing Yugabyte DB 2.0 GA: High-Performance Distributed SQL
The "Clusters Created" metric seems to imply that every deployed database sends back tracking information to Yugabyte's servers.

I've had to look a bit in the docs to find this page, which seems to confirm it: https://docs.yugabyte.com/latest/manage/diagnostics-reportin...

As someone who has seen this tracking-by-default issue play out a few times, I'd recommend having a clear notice on the installation page that includes details on how to opt-out.

lfittl··on The Next Generation (I3en) of I/O-Optimized EC2 Instances
This seems to be confirmed by the IOPS numbers in the documentation:

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/storage-...

lfittl··on Google transformed Mountain View, is San Jose next?
For what its worth, the HSR plan includes Caltrain electrification, which will eventually cut down SF to San Jose to ~45 minutes.

And the state is bigger than just the Bay Area :-)

lfittl··on Rewarding Talent – Handbook on stock options for European entrepreneurs
This overall seems like a very useful resource for EU-based founders.

That said, their suggestion of "Seriously consider back-loaded vesting schedules" is employee unfriendly (https://www.indexventures.com/blog/rewarding-talent-quick-gu..., tip 5)

In particular the idea of a 10/20/30/40 vesting schedule is non-standard, except maybe at places like Amazon. IMHO, if you want to retain people longer, use additional refresh grants instead.

lfittl··on New in Postgres 11: Monitoring JIT Performance, Auto Prewarm, Stored Procedures
(author here)

I had a very positive Postgres experience writing this, as I was able to find three separate bugs before the 11 release.

Two of them have bugfixes merged already (JIT EXPLAIN bugs, auto_explain support for JIT) and a third has a patch pending (track_functions for procedures)

I encourage everyone to test things early & report issues - thats the way Postgres works well for everyone :)

lfittl··on Hetzner removes traffic limitation for dedicated servers
Quality of the backbone matters - Hetzner is going for the cheapest it can find, and (presumably) peering where it can, with whomever it can find.

AWS and GCP both have much more investment into their own backbone, plus higher quality transit providers.

lfittl··on Sqlfmt: an opinionated online SQL formatter
In case you're looking for something Postgres syntax compatible, and find cgo acceptable: https://github.com/lfittl/pg_query_go

(this uses the actual Postgres parser wrapped into a library)

lfittl··on DocuSign Hackathon – $10,000+ in Prizes
DocuSign's API is very frustrating to work with, as you can't go into production without getting their "approval" first.

Amongst other things, the approval process requires you (or did require you when I last tried) to run a specific number of test transactions on a specific API endpoint.

Whilst reasonable for a complex integration, if someone just wants a simple "send this PDF for signature" script, this complex and slow process simply isn't worth it.

lfittl··on Stripe Atlas for LLCs
If you're primarily looking for pass-through taxation it might also be worth looking at electing your C-Corp to be treated as an S-Corp: https://www.irs.gov/businesses/small-businesses-self-employe...

(not a tax advisor / CPA, talk to one for actual specifics)

lfittl··on Queryparser, an Open Source Tool for Parsing and Analyzing SQL
Just in the off-chance this is useful to someone: If you find yourself needing the Postgres parser elsewhere, e.g. in some custom tooling to analyze SQL queries, I extracted it into a library some time ago:

https://github.com/lfittl/libpg_query

Your comment still applies though, as a lot of the parser output is very specific to the chosen Postgres version (10 at the moment) and can change unpredictably.

lfittl··on I’m Peter Roberts, immigration attorney who does work for YC and startups. AMA
Is it correct that the entrepreneur rule was reinstated by court order (https://www.uscis.gov/news/news-releases/uscis-begin-accepti...) so that entrepreneurs could actually file for this right now? (until they undo it)

In case they undo the rule, would the already issued paroles and EAD cards be void?

lfittl··on Scaling Postgres with Read Replicas and Using WAL to Counter Stale Reads
AFAIK its Monodraw: https://monodraw.helftone.com/

(I've asked Brandur this in the past, so assuming he hasn't changed it this should still be what he uses)

lfittl··on Implementing State Machines in PostgreSQL
FWIW, this is fixed in the upcoming Postgres 10:

"Reduce locking required for adding values to enum types (Andrew Dunstan, Tom Lane)

Previously it was impossible to run ALTER TYPE ... ADD VALUE in a transaction block unless the enum type was created in the same block.

Now, only references to uncommitted enum values from other transactions are prohibited."

https://www.postgresql.org/docs/devel/static/release-10.html...

lfittl··on Pev: Postgres ‘Explain’ Visualizer (2016)
Thanks for the shout-out - I'm the author of the base library you listed that can be used to work with the Postgres parser (libpg_query).

That said, the tool being discussed here (pev) is used to interpret EXPLAIN plans, which libpg_query doesn't give you, as its only concerned with the parsing stage, not the planning stage.

In order to plan a query you'll need more information from the actual database, including table statistics, config settings and the size of the underlying table :)

lfittl··on Announcing HTTP/2 support for all Azure CDN customers
Its supported in all regions as of January 25: https://aws.amazon.com/blogs/aws/aws-ipv6-update-global-supp...
lfittl··on Amazon Prime’s latest perk is a new Rewards Visa Card with 5% back
No, you can't, unless I'm missing something: https://www.americanexpress.com/us/content/rewards-info/busi...
lfittl··on Scale Out Multi-Tenant Apps Based on Ruby on Rails
There are a few things, the most important ones:

  - Including the tenant_id in modification queries as well (e.g. UPDATE/DELETE)
  - Support for specifying the current tenant as an ID (through `MultiTenant.with_id(123) { ... }`)
  - Helpers for multi-column primary keys in your migrations
  - Specific integration with Citus as a distributed multi-tenant data store (it'll just work without modifications)
I've considered contributing these back to acts_as_tenant, but it seemed easier to have a clean-cut, since the shared lines of code only ended up being about ~50 lines - mostly to set a default scope.
lfittl··on Scale Out Multi-Tenant Apps Based on Ruby on Rails
Let me start by saying that apartment is great - but it takes a different approach.

The most important difference is that apartment shards by database or schema, vs activerecord-multi-tenant puts all your data into the same table, and then relies on a PostgreSQL extension like Citus to do the distribution.

There is trade-offs to both, but part of the reason we prefer the latter approach is that schema changes become a lot easier, and you can do shard-splitting based on a shard count, as opposed to just by tenant, effectively grouping smaller tenants together on a node.

I'll also give a shout-out to acts_as_tenant, which our library is based on, although we've deviated a bit since then.

lfittl··on Scale Out Multi-Tenant Apps Based on Ruby on Rails
The issue is that you would be missing the tenant_id (or user_id) if you were to do `other_object.pages.find(params[:id])` (i.e. on something thats not directly below the user)

That is a problem in systems that expect the tenant_id to always be in the query, so you can locate which node the query needs to go to.

← PreviousPage 3 of 6Next →