PostgreSQL reconsiders its process-based model
lwn.net
lwn.net
For the record, I think this will be a disaster. There is far too much
code that will get broken, largely silently, and much of it is not
under our control.
regards, tom lane
If Tom Lane says it will be a disaster, I believe it will be a disaster.
But there is another way.
Ok?
[0] https://www.thegeekdiary.com/what-is-sql-server-operating-sy...
I think "Idiom: a tic (Rust)" can also fit if I squint hard enough and decide it looks like a definition from an online dictionary :-)
Ignoring the above, first state the explicit requirements driving this change and let people weigh in on those. This sounds like a geeky dev itch.
For those who don't follow PHP closely - that version was an attempted refactor of the string implementation which essentially shut down nearly all work on PHP for a decade, stagnating the language until it became pretty terrible compared to other options. They finally gave up and started work on PHP 7 which uses the (perfectly good) PHP 5 strings.
Ten years of wasted time by the best internal PHP developers crippled the project - I'm amazed it survived at all.
I got my real professional start using PHP, and have built even financial systems in the language (since ported to .NET 6 for my ease of maintenance, and better number handling). I'm still very interested in the language itself, in case I ever have the need to freelance or provide a solution to a client that can't afford what I can build in .NET (although to be honest, at this point I'm roughly able to code at the same speed in .NET as in PHP, but with the added type-safety, although I know PHP has really stepped up in providing this).
Regular PHP strings are actually pretty great, since you can treat them like byte arrays. Fun fact: PHPs streaming API has an “in-memory” option and it’s… just a string under the hood.
Just don’t forget to use multibyte functions when you’re handling things like user input.
Weird book IMO, because it has a lot of content that's just about general software development, rather than anything to do with PHP specifically, or the theoretical PHP6 APIs in particular.
https://www.joelonsoftware.com/2000/04/06/things-you-should-...
An oldie but a goodie
You should rewrite code only when the cost of adding a new feature (one that is actually necessary) to the old codebase becomes comparable to designing your entire system from scratch to allow for that feature to be added easily. That is to say that the cost of the rewrite should become comparable to the cost of continuing development. I have been a part of a couple of rewrites like that, one of them quite complex, and yes they were warranted and yes they worked.
But having said that you should absolutely be conservative with rewriting code. It’s a bad habit to always jump to a rewrite.
I don’t think you necessarily need to be conservative about rewriting things. We do it all the time in fact. We build something to get it out there and see the usage, and then we build it better and then we do it again. Which often involves a lot of “rewriting” but thanks to principles like SOLID’s single responsibility makes this rather easy to both do and maintain (we write a lot of semi-functional code and try to avoid using OOP unless necessary, so we don’t really use all the parts of SOLID religiously).
I do agree that it’s never a good idea to get into things with the mind-set of “we can do this better if we start from scratch” because you can’t.
Why not identify that same logical feature and move it into a library. How does a Microservice add value here?
Identifying, extracting and adding tests to logical features has been the sane way to rewrite software for ages. Michael feathers even wrote a book about it [1]. This ship of theseus approach works because it's incremental and allows for a complete rewrite without ever having non-functional software in between.
Adding a REST/networking/orchestration boundary to make it a Microservice just for the sake of extracting it adds a lot of complexity for no gain.
Microservice can be the right architecture, but not if all you want is to extract a library.
[1] https://understandlegacycode.com/blog/key-points-of-working-...
You shouldn't stop development on your important products.
Letting a couple of your talented programmers loose on a greenfield reimplementation is a perfectly sane strategic move.
Stopping development on important products because you are 100% certain that the reimplementation will be successful by $DEADLINE is a foolish gamble.
Between the 3 P's, Python's strategic decisions in 2000s were clearly the most successful.
And it wasn't a total rewrite.
So in order for there to be a net gain, the gang working on the alternative have to be able to find such big wins as to being neigh impossible.
This is a very very hard problem in our domain. 99% of the time, we have to simply resist the urge to _just rewrite the sucker_. No! Don't do it! (And this is incredibly hard because we all want to.)
Say you have product A with features P, Q, R, and S; writing product B has to reproduce P, Q, R, and S, plus X and Y that is currently produced by the A team. On top of that, it has to fix (conceptual) bugs within P, Q, R, and S. All this is to be done by the new, crack, team that aims to make it so that: cost-of-development(B) < cost-of-development(A).
But the point is that the difference in magnitude of cost-of-development(B) and cost-of-development(A) has to be rather large considering the amount of work needed to have a return on that investment at all.
> They decided to rewrite the code from scratch.
Absolutely not what's being proposed for Postgres.
In practical terms, if every line needs to be audited and updated, it is a re-write
Experience with other systems has taught me that in a system that's been in active use and development for decades, entanglement will be deep, subtle, and pervasive. If this isn't true of postgres then it's an absolute freak anomaly of a codebase. It is that in other ways, so it's possible.
But the article mentions there being thousands of global variables. And Tom Lane himself says he considers it untenable for exactly this reason. That's a very good reason to think that it will require that many changes imo.
The 'that many changes' in question is a complete rewrite. Many changes across many files, yes, but nothing even approaching a rewrite.
> I don't have any particular knowledge of postgres
Judging by their bio, the person you're replying to does.
A large refactor at best. It will touch lots of parts of the code base, but the vast majority of the source code would remain intact. Otherwise they could just Rewrite it in Rust™ while they’re at it
> if every line needs to be audited and updated, it is a re-write
I’m not sure why you believe every line needs to be updated. Most code is thread agnostic.
One of the main programmers suggested making it so that the next guidance routine wouldn't run until the previous one was done. This would make the code less sensitive to race conditions and allow more useful functionality for the pilots (who were the actual users and did seem to want it). However everyone assumed the two-second constant was implicitly embedded everywhere.
It wasn't -- only in a few places -- and with that fixed the code got more general and the proof of concept ran better than ever in about every simulator available. The amount of control it gave pilots was years ahead of the curve. But it never got a chance to fly on a real mission because what was there was "good enough" and nobody bothered to try.
In our combined comments there's a lesson about growing experiments and figuring out how to achieve failure quickly.
I don't expect you or others to buy into any particular code change at
this point, or to contribute time into it. Just to accept that it's a
worthwhile goal. If the implementation turns out to be a disaster, then
it won't be accepted, of course. But I'm optimistic.
I'm not trying to naysay it per se, bc again I don't have technical knowledge of this codebase. But that's exactly the sort of scenario that can cause a large project to splinter or stall for years. Talking about "the implementation" absent the context that would be necessary to create that implementation seems naively optimistic, or at worst irresponsible.
They could fork, they could add threading to some sub systems and roll it out over several versions.
I don't know enough about the code but, of course, it is a hard problem but the solution might be to build it from the ground up as a threaded system, using the skills learned over 30 years and taking the hit on the rebuild instead of reworking what is there.
I am most interested because I didn't realise there was a performance problem in the first place.
Going the other way, there is an extremely well understood interface between all the processes which run in isolation: shared memory. Nearly by definition this must be well coordinated between the processes.
So the first step in moving to a multi-threaded implementation would be to change nearly nothing about each process, and then just run each process in its own pthread, keeping all the shared memory ‘n all.
You would expect performance to be about the same, maybe a little better with the reduces TLB churn, but the architecture is basically unchanged. At that point, you can start to look at what are more appropriate communication/synchronisation mechanisms now you’re working in the same address space.
I just don’t understand why so many people seem to think this requires an enormous rewrite - having developed as a multi-process system means you’ve had to make so much of the problematic things explicit and control for them, and none of these threads would know anything at all about each other’s internals.
Without being familiar with the Postgres source, this seems to be what I call a "somersault problem": hard to break down into sub-goals. I have heard that the Postgres codebase is solid which makes it easier but it's still mature and highly complex. It doesn't sound feasible to me.
[link redacted]
Why not work on something like that instead of changing something that works? Especially since they the process model really only runs into trouble on large systems.
WARNING: terminating connection because of crash of another server process
DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT: In a moment you should be able to reconnect to the database and repeat your command.
LOG: all server processes terminated; reinitializingThreaded vs process won't affect that.
On the other hand, all cases of fixable corrupted data in PostgreSQL I have seen were result of somebody doing something totally dumb (rsyncing live cluster, even between architectures), while on InnoDB it seems to happen somewhat randomly without any obvious reason of somebody doing stupid things.
Sure, but the blast radius of corruption is limited to that shared memory, not all the memory of all the processes. You can at least use the fact that a process has crashed to ensure that the corruption doesn't spread.
(This is why it restarts: there is no guarantee that the shared memory is valid, so the other processes are stopped before they attempt to use that potentially invalid memory)
With threads, all memory is shared memory. A single thread that crashes can make other threads data invalid before the detection of the crash.
Relying on crashing as a bug recovery system is a good idea? Crashing is just part of the workflow? That's insane, and a good argument against PostgreSQL in any production system.
It is possible PostgreSQL doesn't migrate to a thread based model, and I am not arguing they should.
But debug and patch the causes of these crashes? Absolutely yes, and the sooner, the better.
Erlang users don't seem to agree with you
Like, they are supposed to crash?!?
I think the post upthread references an out-of-core extension we don't control, which in turn depends on many external libraries it doesn't control either...
Though to your point, a C extension can totally destroy your data in other ways, and there are likely ways to add more barriers. And hey, we should fix bugs!
Building a database which is never gonna crash might be possible but at what cost? Can you name any single real world system archived that? Also, there can be a regression. More tests? Sure but again, at what cost?
While we are trying to get there, having a crash proof architecture is also a very practical approach.
The fundamental problem is that when you're sharing memory, you cannot safely just stop a single process when encountering an unexpected error. You do not know the current state of your shared data, and if it could lead to further corruption. So restarting everything is the only safe choice in this case.
> That idea quickly loses its appeal, though, when one considers trying to create and maintain a 2,000-member structure, so the project is unlikely to go this way.
As repulsive as this might sound at first, I've seen structures of hundreds of fields work fine if the hierarchy inside them is well organized and they're not just flat. Still, I have no real knowledge of the complexity of the code and wish the Postgres devs all the luck in the world to get this working smoothly.
I believe I can safely say that nobody acknowledges and understands the complexity of all state within that class, and that whatever incentives there may be to simplify it are not enough for that to actually happen.
(It ends on line 4692)
I didn't make any claims about the _amount_ of state. My claim was that “you're forced to acknowledge and understand the complexity of your state” (i.e., moving it all together in one place helps understanding the state) is plain-out wrong.
It’s an easy transition path.
Agree.
> It's not like the globals were organized before!
Using a struct with 2000 fields loses some encapsulation.
When a global is defined in a ".c" file (and not exported via a ".h" file), it can only be accessed in that one ".c" file, sort of like a "private" field in a class.
Switching to a single struct would mean that all globals can be accessed by all code.
There's probably a way to define things that allows you to regain some encapsulation, though. For example, some spin on the opaque type pattern: https://stackoverflow.com/a/29121847/163832
A plain global can be accessed from other compiled units - agreed with no .h entry it is my=uch more error prone e.g. you don't know the type but the variables name is exposed to other objects
The `static` modifier in that context just means that the symbol is not exported, so other ".c" files can't access it.
Check out the generated assembly for this simple program, notice that kBase is folded even though it's not marked const: https://godbolt.org/z/h45vYo5x5
But the Postgres mailing list is talking about 2000 global variables being a hurdle to multi-threading. I doubt they just didn't realize that most of them can be optimized into constants.
Merge all that into one process with many threads and it becomes a nightmare problem to ensure some random addon didn't decide to change a global var mid processing (which causes wrong data to be read).
Of course, there's a huge amount of shared space already, so a willful corruption can already do virtually anything. But, more is more.
While we didn't have 2000 globals, we did have a non-trivial amount, spread over about 300kLOC of C++.
We started by just stuffing them into a "context" struct, and every function that accessed a global thus needed to take a context instance as a new parameter. This was tedious but easy.
However the upside was that this highlighted poor architecture. Over time we refactored those bits and the main context struct shrunk significantly.
The result was better and more modular code, and overall well worth the effort in our case, in my opinion.
If anything, it's probably easier to understand.
On the contrary, it's been discussed for ages. But it's a huge change, with only modest advantages.
I'm skeptical of the ROI to be honest. Not that is doesn't have value, but that it has more value than the effort.
+significant and unknown set of new problems, including new bugs.
This reminds me of the time they lifted entire streets in Chicago by 14 feet to address new urban requirements. Chicago, we can safely assume, did not have the option of just starting a brand new city a few miles away.
The interesting question here is should a system design that works quite well upto a certain scale be abandoned in order to extend its market reach.
Years ago I wrote an algorithmic trader in Python (and Cython for the hotspots) using Multiprocessing and I was able to get away with a lot using that approach. I had one process receiving websocket updates from the exchange, another process writing them to an order book that used a custom data structure, and multiple other processes reading from that data structure. Ran well enough that trade decisions could be made in a few thousand nanoseconds on an average EC2 instance. Not sure what their latency requirements are, though I imagine they may need to be faster.
Obviously mutexes are the bottleneck for them at this point, and while my idea might be a bit slower than a low-load situation, perhaps it would be faster when you start getting to higher load.
is this another way to say "in a 2000 member structure, only 10 have significant voting power"?
I'm not. You can get a long way with conventional IPC, and OS processes provide a lot of value. For most PostgreSQL instances the TLB flush penalty is at least 3rd or 4th on the list of performance concerns, far below prevailing storage and network bottlenecks.
I share the concerns cited in this LWN story. Reworking this massive code base around multithreading carries a large amount of risk. PostgreSQL developers will have to level up substantially to pull it off.
A PostgreSQL endorsed "second-system" with the (likely impossible, but close enough that it wouldn't matter) goal of 100% client compatibility could be a better approach. Adopting a memory safe language would make this both tractable and attractive (to both developers and users.) The home truth is that any "new process model" effort would actually play out exactly this way, so why not be deliberate about it?
Or more if a rewrite of subsystems? Like the query planner or storage engine etc?
With regard to client compatibility there are related precedents for this already; the PostgreSQL wire protocol has emerged as a de facto standard. Cockroachdb and ClickHouse are two examples that come to mind.
IPC, to me, includes the conventional shared memory resources (memory segments, locks, semaphores, condition variable, etc.) used by these systems: resources acquired by processes for the purpose of communication with other processes.
I get it though. The most general concept of shared memory is not coupled to an OS "process." You made me question whether my concept of term IPC was valid, however. So what does one do when a question appears? Stop thinking immediately and consult a language model!
Q: Is shared memory considered a form of interprocess communication?
GPT-4: Yes, shared memory is indeed considered a form of interprocess communication (IPC). It's one of the several mechanisms provided by an operating system to allow processes to share and exchange data.
...
Why does citing ChatGPT make me feel so ugly inside?
In that view, I always saw shared memory as IPC, in that it is a tool commonly used to exchange data between processes, but of course it is not strictly tied to any process in particular. This is similar to files, which if you squint are a form of IPC too, and are also not tied to any specific process.
> Why does citing ChatGPT make me feel so ugly inside?
That's probably because, in cases like this, it's not much different to stating it yourself, but is more noisy.
Its the modern let me Google that for you. Just like people don't care what the #1 result on Google is, they also don't care what ChatGPT has to say about it. If they did, they'd ask it themselves.
Postmaster would just share the already shared memory between processes (containing also the locks). That explicit part of memory would opt-in to thread -like sharing and thus get faster/less tlb switching and lower memory usage. While all the rest of the state would still be per-process and safe.
tl;dr super share the existing shared memory area with kernel patch
All operating systems not supporting it would keep working as is.
But I'm very much clueless about internals, so this is a question rather than an opinion.
If you push a lot of work into the database including JSON and have a lot of buffer memory...CPU can easily be limiting.
That said, many of these things are solvable without a full switch to a threaded model (eg. by having pooling built-in and session-state-aware).
Yeeeeesssss, but solving that is solving the hardest part of switching to a threaded model. It requires the team to come terms with the global state and encapsulating session state in a non-global struct.
The thing is that that's a lot easier with threads. Much of the session state lives in process private memory (prepared statements etc), and it can't be statically sized ahead of time. If you move all that state into dynamically allocated shared memory, you've basically paid all the price for threading already, except you can't use any tooling for threads.
I was surprised when starting with postgres. Then again, I have smaller databases (A few TB) and the cache hit ratio tends to be about 95%. Combine that with SSDs, and it becomes understandable.
Even so, I am wary of this change. Postgres is very reliable, and I have no problem throwing some extra hardware to it in return. But these people have proven they know what they are doing, so I'll go with their opinion.
Throw a few NVMe drives at it and it might.
It's orders of magnitude faster than SAS/SATA SSDs and you can throw 10 of them into 1U server. It's nowhere near "slow" and still easy enough to be CPU bottlenecked before you get IO bottlenecked.
But yes, pair of 1TB RAM servers gotta cost you less than half year's worth of developer salary
In those scenarios,there's very little read I/O. CPU is the primary bottleneck. That's why we run up as many as 10 Aurora readers (autoscaled with traffic).
I abuse them for ridiculous things.
Oracle also uses a process model on Linux. At some point (I think starting with 12.x), it can now be configured on Linux to use a threaded model, but the default is still a process-per-connection model.
Why does everybody think it's a bad thing in Postgres, but nobody thinks it's a bad thing in Oracle.
Mainline has a lock so that all backends can use global variables, but only one instance can do codegen at a time.
It was a giant refactoring Especially fun was when multiple compilation units used the same static variable name, but it all worked in the end.
This is the multi-threaded compiler: https://github.com/rsaxvc/tcc-swarm
With the multi-threaded tcc above it scales about as well as multiprocess. With mainline it doesn't scale well at all.
So far I haven't gotten around to reusing anything across libtcc handles/instances, but would eventually like to share mmap()'d headers across instances, as well as cache include paths, and take invocation arguments through stdin one compilation unit per line.
Regarding the threading issue, I think you can do the connections part multithreaded instead of one process per connection and still use IPC between this and postmaster. Because of the way PostgreSQL currently works, seems feasible to move parts one by one into a threaded model and instead of tens/hundreds of processes you can have just a few and a lot of threads.
Honestly, they should prototype it and see how it looks like and then decide on the way forward.
If someone didn't take the time to update the comments and the reviewers didn't point it out, then you've probably found the bug because someone was cowboying some shitty code.
Outdated comments are often way worse than no comments, because they can give you wrong ideas that aren't true anymore, and send you off in the wrong direction before you finally figure out the comment was wrong.
if (X) assert(false); // we never do X, ever, anywhere.
if (exact same X) { do_useful_stuff(); }
enum kinds { writers; readers; updaters; }
I've seen (and reported) bugs that caused panics/segfaults in specific psql processes. Not just connections, also processes related to wal writing or replication. The way it's built right now, a child process can be just forced to quit and it does not affect other processes. Hopefully switching into thread won't force whole PostgreSQL to panic and shut down.
The only reliable way to stop bad code like say an infinite loop is to run in another process even in .Net.
They also removed Appdomain in later versions of .Net because they had little benefit and weak protections compared to a a full process.
"On .NET Core, the AppDomain implementation is limited by design and does not provide isolation, unloading, or security boundaries. For .NET Core, there is exactly one AppDomain. Isolation and unloading are provided through AssemblyLoadContext. Security boundaries should be provided by process boundaries and appropriate remoting techniques."
AppDomains pretty much only allowed you to load unload assemblies and provided little else. If you wanted to stop bad code you still used Thread.Abort which left your runtime in a potentially bad state due to no isolation between threads.
The only way to do something like an AppDomain to replace process isolation would be to re-write the whole OS in a memory safe language similar to https://en.wikipedia.org/wiki/Midori_(operating_system) / https://en.wikipedia.org/wiki/Singularity_(operating_system)
As it happens I already know there are bits of code currently not working "as expected" exactly because of AppDomain separation - i.e. attempting to use a shared-memory cache to improve performance and in one or two cases in an attempt to share state, and I got the impression whoever wrote that code didn't understand that there even were two AppDomains involved, and used various ugly hacks to "fall back" to alternative means of state-sharing, but in fact the fall-back is the only thing that actually ever works.
No, you can't create a second AppDomain at all. AppDomains are dead and buried; you would need to remove all of that from your code in order to migrate to current .NET. The class only remains to serve a couple ancillary functions that don't involve actually creating additional AppDomains.
At least I'm not aware of any way to protect private thread memory from other threads.
Postgres is C and that's not going to change ever.
Point is it getting worse if this is changed.
For native code, there's no such safety net. Likewise, even for managed language, an error in the interpreter code will still crash the VM, since there's nothing to fallback to anymore.
As a longstanding PG dev/DBA who doesn't know much about its internals, I would say that they should just move connection pooling into the main product.
Essentially, pgbouncer should be part of PG and should be able to manage connections with knowledge of what each connections is doing. That, plus, some sort of dynamic max connection setting based on what's actually going on.
That'll remove almost all the dev/DBA pain from separate processes.
https://bholley.net/blog/2015/must-be-this-tall-to-write-mul...
Back about a decade ago I was "auditing" someone else's threaded code. And couldn't figure it out. But he was the company's "golden child" so by default it must be working code because he wrote it.
And then it started causing deadlocks in prod.
"What do you want me to do about it? It's the golden child's code. He's not even gonna show up til 2pm today."
They do roll some of their own locking primitives, but that's not particularly unusual in a large portable program (and quite likely what they wanted is/was not available in glibc or other standard libraries, at least when first written).
On UNIX systems, Oracle uses a multi-process model, and you can see these:
$ ps -ef | grep smon
USER PID PPID STARTED TIME %CPU %MEM COMMAND
oracle 22131 1 Mar 28 3:09 0.0 4.0 ora_smon_yourdb
Sybase was the first major database that fully adopted threads from an architectural perspective, and Microsoft SQL Server has certainly retained and improved on that model.
I work with a Windows-based COTS webapp that uses Postgres w/o any connection pooling. It's nearly excruciating to use because it spins-up new Postgres processes for each page load. If not for the fact that the Postgres install is "turnkey" with the app I'd just move Postgres over to a Linux machine.
We're really talking about X-per-client as the primary reason to move away from processes, right?
So if you can get most of the benefit via pooling... why inherit the pain of porting?
Presumably latency jitter would be a difficult problem with pools, but it seems easier (and safer) than porting processes -> threads.
Disclaimer: High performance / low latency DB code is pretty far outside my wheelhouse.
Many other things too. Like better sharing of caches. Lower overhead of thread instead of process. Etc. (read the thread)
Could you point out, aside from the large numbers of clients I mentioned (and the development overhead of implementing multi-process memory management code), what the article mentions is a primary drawback of using processes over threads?
$ ps -ef | grep smon
UID PID PPID C STIME TTY TIME CMD
oracle 22131 1 0 Mar28 ? 00:03:09 ora_smon_yourdb
$ $ORACLE_HOME/bin/sqlplus -silent '/ as sysdba'
select version_full from v$instance;
VERSION_FULL
-----------------
19.18.0.0.0Probably still requires the parameter to be set.
$ systemctl status mssql-server
● mssql-server.service - Microsoft SQL Server Database Engine
Loaded: loaded (/usr/lib/systemd/system/mssql-server.service; disabled; vendor preset: disabled)
Active: active (running) since Mon 2023-06-19 15:48:05 CDT; 1min 18s ago
Docs: https://docs.microsoft.com/en-us/sql/linux
Main PID: 2125 (sqlservr)
Tasks: 123
CGroup: /system.slice/mssql-server.service
├─2125 /opt/mssql/bin/sqlservr
└─2156 /opt/mssql/bin/sqlservrOracle - never had a scalability issue on very big Linux, Solaris and HPUX systems though - they do it well in my experience.
It's optional, and the default is still a process model on Linux.
Create an operating system specifically for the database and make it so you boot the database.
Databases seem to spend most of their time working around the operating system abstractions. So why not look at the OS, and streamline it for database use - dropping all the stuff a database will never need.
That then is a completely separate project which is far easier to get started rather than shoehorning the database into an operating system thread model that is already a hack of the process model.
I don't know how that can make it easier the process based / thread based problem.
https://mirage.io/ or similar could then let you boot your database. That said, it's not really taken off from what I can tell, so I'm guessing there's more to it than that.
There certainly are times OSs get in the way. But it's hard enough to write a good database, we don't need to maintain a third of an OS in addition.
That said, perhaps at AWS or Google scale that would be different? I wonder if they've looked at this stuff internally.
This is actually part of the reason why Microsoft was able to port SQL Server to Linux fairly easily, if I recall correctly.
I have the impression that this is similar to the adhoc filesystem idea; this seems in principle very advantageous (why employing two layers that do approximately the same thing on top of each other?), but in reality, when implemented (by Oracle), it lead to only a minor improvement (a few % points, AFAIR).
(Others downthread have pointed out unikernels and I agree with the criticisms)
This proposal is an excellent Phd project for someone like me :-)
It ticks all of the things I like to work on the most[1]:
Will involve writing low-level OS code
Get to hyper-focus on performance
Writing a language parser and executor
Implement scheduler, threads, processes, etc.
Implement the listening protocol in the kernel.
I have to say, though, it might be easier to start off with a rump kernel (netBSD), then add in a specific RAW disk access that bypasses the OS (no, or fewer, syscalls to use it), create a kernel module for accepting a limited type of task and executing that task in-kernel (avoiding a context-swtich on every syscall)[2].
Programs in userspace must have the lowest priority (using starvation-prevention mechanisms to ensure that user input would eventually get processed).
I'd expect a non-insignificant speedup by doing all the work in the kernel.
The way it is now,
userspace requests read() on a socket (context-switch to kernel),
gets data (context-switch to userspace),
parses a query,
requests read on disk (multiple context-switches to kernel for open, stat, etc, multiple switches back to userspace after each call is complete). This latency is probably fairly well mitigated with mmap, though.
logs diagnostic (multiple context-switches to and from kernel)
requests write on client socket (context switch to kernel back and forth until all data is written).
The goal of the DBOS would be to remove almost all the context-switching between userspace and kernel.
[1] My side projects include a bootable (but unfinished) x86 OS, various programming languages, performant (or otherwise) C libraries.
[2] Similar to the way RealTime Linux calls work (caller shares a memory buffer with rt kernel module, populates the buffer and issues a call, kernel only returns when that task is complete). The BPF mechanism works the same. It's the only way to reduce latency to the absolute physical minimum.
Many of the comments here seem to be missing this and talking about CPU-boundedness generally and thread-per-request vs process etc models, but this seems orthogonal to that, and is actually quite specific about the VM subsystem and seems like a legitimate bottleneck with the approach Postgres has to take for buffer/page mgmt with the process model it has now.
I'm no Postgres hacker (or a Linux kernel hacker), and I only did a 6 month stint doing DB internals, but it feels to me like perhaps the right answer here is that instead of Postgres getting deep down in the weeds refactoring and rewriting to a thread based model -- with all the risks in that that people have pointed out -- some assistance could be reached for by working on specific targeted patches in the Linux kernel?
The addition of e.g. userfaultfd shows that there is room for innovation and acceptance of changes in and around kernel re: page management. Some new flags for mmap, shm_open, etc. to handle some specific targeted use cases to help Postgres out?
Also wouldn't be the first time that people have done custom kernel patches or tuning parameters to crank performance out of a database.
And if you carefully craft your CPU scheduling and address spaces mapping, you can reduce them by a lot.
Yet rewrites are always easier and more sexy. At first.
> Yet rewrites are always easier and more sexy. At first.
Moving to threads would not at all be a rewrite.
On the other hand, having the ability to shut down and cleanup the entire memory space of a single connection by just disconnecting is really nice, especially if you have extensions that do interesting things.
> Tom Lane said: "I think this will be a disaster. There is far too much code that will get broken". He added later that the cost of this change would be "enormous", it would create "more than one security-grade bug", and that the benefits would not justify the cost.
That the all of those would work faster because of performance benefits, as mentioned in article
This is simply not true for most software. Software has a product life cycle like everything else and major refactors/rewrites should be weighed carefully against cost/risk of the refactor. Many traditional engineering fields do much better at this analysis.
Although, because I run a contracting shop, I have personally profited greatly by clients thinking this is true and being unable to convince them otherwise.
With the changes suggested, I'm not sure it's the best idea from where Postgres is... if might be an opportunity to rewrite bits in Rust, but even then, there is a LOT that can go wrong. The use of shared memory is apparently already in place, and the separate process and inter-process communication isn't the most dangerous part... it's the presumption, variables and other contextual bits that are currently process globals that wouldn't be in the "after" version.
The overall surface is just massive... That doesn't even get into plugin compatibility.
These are "famous last words" in many contexts, but when talking about difficult-to-reproduce parallelism issues, I just don't think it's a particularly applicable viewpoint at all. No disrespect. :)
Postgres starts from a share-nothing architecture, it’s quite a bit easier to evaluate the addition of sharing.
I don't think that's a fair characterization of the trade offs. Acquiring uncontended mutexes is basically free (and fairly side-effect free) so single-threaded performance will not be noticeably impacted.
Every large C project I'm aware of (read: kernels) that has publicly switched from coarse locks to fine-grained locks has considered it to be a huge win with little to no impact on single-threaded performance. You can even gain performance if you chop up objects or allocations into finer-grained blobs to fit your finer-grained locking strategy because it can play nicer with cache friendliness (accessing one bit of code doesn't kick the other bits of code out of the cache).
1) I don't buy this a priori. Almost everybody who removed a gigantic lock suddenly realizes that there was more contention than they thought and that atomizing it made performance improve.
2) Had Python bitten the bullet and removed the GIL back at Python 3.0, the performance would likely already be back to normal or better. You can't optimize hypothetically. Optimization on something like Python is an accumulation of lots of small wins.
You don’t have to buy anything, that’s been the result of every attempt so far and a big reason for their rejection. The latest effort only gained some traction because the backers also did optimisation work which compensated (and then was merged separately).
> Almost everybody who removed a gigantic lock
See that’s the issue with your response, you’re not actually reading the comment you’re replying to.
And the “almost” is a big tell.
> suddenly realizes that there was more contention than they thought and that atomizing it made performance improve.
There is no contention on the gil in single threaded workloads.
> Had Python bitten the bullet and removed the GIL back at Python 3.0
It would have taken several more years and been completely DOA.
They mentioned a similar issue for Postgres extensions, no?
> Haas, though, is not convinced that it would ever be possible to remove support for the process-based mode. Threads might not perform better for all use cases, or some important extensions may never gain support for running in threads.
Biggest problem of the process model might be the cost of having too many DB connections. Each client need a dedicated server process. Memory usage and the context switching overhead. Or if there is no connection pool, connection time overhead is very high.
This problem has been well addressed with a connection pool. Or having a middle ware instead of exposing the DB directly. That works very well so far.
Oracle has been supporting the thread based model and it's been usable for decades. I remember I tried the thread based configuration option (MTS or shared server) in 1990s. But no one likes that at least within my Oracle DBA network.
It would be a great research project but it would be a big problem if the community pushs this too early.
Like light-threads, heavy-threads would share the same process-security-boundary and therefore switching between them would be cheap. No need to flush TLB, I$, D$.
Like processes, heavy-threads would have mostly-separate address spaces by default. Similar to forking a process, they could share read-only mappings for shared libraries, code, COW global variables, and explicitly defined shared writable memory regions.
Like processes, heavy-threads would isolate failure states. A C++ exception, UNIX signal, segfault, etc. would kill only the heavy-thread responsible.
TLB isn't "flushed" so much as it is useless across different memory address spaces. Switching processes means switching address spaces, which means you have to switch the contents of the TLB to the new process' TLB entries, which eventually indeed flushes the TLB, but that is only over time, not necessarily the moment you switch processes.
> Like processes, heavy-threads would have mostly-separate address spaces by default.
This thus conflicts with the need to not flush TLBs. You can't not change TLB contents across address spaces.
1. Mostly separate address spaces requires changing the TLB on context switch (modern hw lets it be partial). You could use MPKs to share a single address space with fast protection switches.
2. Threads share the global heap, but your heavy threads would require explicitly defined shared writeable memory regions, so presumably each one has its own heap. That's a fair bit of overhead.
3. Failure isolation is more complicated than deciding what to kill.
The expand on the last point, Postgres doesn't isolate failures to a single process because they do share memory and might corrupt those shared memory regions. But even if you don't have shared memory failure recovery isn't always easy. Software has to be written specifically to plan for it. You can kill processes because everything in the OS is written around allowing for that possibility, for example, shells know what to do if a sub-process is killed unexpectedly. Killing a heavy thread (=process) is no good if the parent process is going to wait for a reply from it forever because it wasn't written to handle the process going away.
> Like processes, heavy-threads would have mostly-separate address spaces by default. Similar to forking a process, they could share read-only mappings for shared libraries, code, COW global variables, and explicitly defined shared writable memory regions.
I don't think you realistically can have separate address spaces and not have TLB etc impact. If they're separate address spaces, you need separate TLB entries => lower TLB hit ratio.
Unfortunately I expect this to go the way of zheap et al. Fundamental design changes like this have just had such a rough time of succeeding thus far.
I think for such a change to work it probably needs not just the support of Neon but also of say Microsoft (current stewards of Citus) that have larger engineering resources to throw at the problem and grind out all the bugs.
Want to run an extension that isn't threadsafe? Run with 10 processes, 1 threads. Want to run high-performance? Run with 1 process, 10 threads. Afraid of "stability issues"? Run with 1 process, 1 thread.
Will it be hard to do? Sure. Impossible? Not at all. Plan for it, give a very long runway, throw all your new features into the next major version branch, and tell people everything else is off the table for the next few years. If you're really sure threading is going to be increasingly necessary, better to start now than to wait until it's too late. But this idea of "oh it's hard", "oh it's dangerous", "too complicated", etc is bullshit. We've built fucking spaceships that visit other planets. We can make a database with threads that doesn't break. Otherwise we admit that basic software development using practices from the past 30 years is too much for us to figure out.
IMO: if the variable is _truly_ global, i.e. code all over the codebase cares about it, then it should just be global instead of pretending like it's not with some fancy architecture.
The tricky part is reacting to changes to a global variable. Writing a bunch of "on update" logic leads to madness. The ideal solution is for there to be some sort of one-directional flow for updates, like when a React component tree is re-rendered... but that's very hard to build in an application that doesn't start out using a library like React in the first place.
Robert Haas pointed out in one message that an implementation pattern was to use things like file-scope static storage duration variables to provide session-local state for individual components. This is why they've been arguing against a single giant structure declared in "session.h" as an approach, as it requires every future addition to session state to touch the central core of the entire program.
They want to keep the advantage of the fact that these variables are in fact not global. They are local; and the problem is rather that they have static storage duration and are not per-thread, and thus are not per-session in a thread-per-session model.
The challenge with this setup is that you need to do 1 writer and multiple reader clusters so you end up doing localhost replication which is super weird. If that requirement was somehow removed that’d be awesome for scaling really huge clusters.
Then, after more thought, I'm not entirely sure that would be a bad approach. I say this not to advocate for actually rewriting it in Rust, but as a way of describing how difficult this is. I'm not actually sure rewriting the relevant bits of the system in Rust wouldn't be easier in the end, and obviously, that's really, really hard.
This is really hard transition.
I don't think multithread code quality should be measured in absolutes. There are things that are so difficult as to be effectively impossible, which is the lock-based approach that was dominant in the 90s, and convinced developers that it's just impossible difficult, but it's not multithreaded code that's impossibly difficult, it's lock-based multithreading. Other approaches range from doable to even not that hard once you learn the relevant techniques (Haskell's full immutability & Rust's borrow checker are both very solid), but of course even "not that hard" becomes a lot of bugs when scaled up to something like Postgres. But it's not like the current model is immune to that either.
Aside from the trivial while(!transactionSucceeded){retry()} loop, I have trouble proving the correctness of my programs when the number of threads is not small and finite.
— Joe Armstrong
Threads are evil. https://www.sqlite.org/faq.html#q6 https://www2.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-...
Nginx uses an asynchronous event-driven approach, rather than threads, to handle requests. https://aosabook.org/en/v2/nginx.html http://www.kegel.com/c10k.html
They might even reduce complexity that way.
If every process is single threaded, you don't have to consider the possibility of race conditions when accessing any of those ~2000 global variables. And you can pretty much guarantee that little if any of the existing code was written with that possibility in mind.
With the process model, it encourages using separate memory for each process; this forces developers to think about things like memory consistency and availability and gives them more flexibility in terms of scalability across multiple CPU cores or even hosts. Processes are far better abstractions than threads for modeling concurrent systems since their logic is fundamentally the same regardless of whether they run across different CPU cores or different hosts.
> The overhead of cross-process context switches is inherently higher than switching between threads in the same process
I remember researching this a while back. It depends on the specific OS and hardware. It's not so straight forward and this is something which tends to change over time and the differences are usually insignificant anyway.
Also, it's important not to conflate performance with scalability - These two characteristics are orthogonal at best and oftentimes conflicting.
Oftentimes, to scale horizontally, a system needs to incur a performance penalty as additional work is required to route and coordinate actions across multiple CPUs or hosts. A scalable system can service a much larger number (or even sometimes theoretically unlimited) number of requests but it will typically perform worse than a non-scalable system if you judge it on a requests-per-CPU-core basis.
That said, I think there have been efforts to use io_uring on Linux. I'm not sure how that would work with the process per connection model. Haven't been following it...
There's some minor details that are easier with threads in that context, but on the whole it doesn't make much of a difference.
Possibly they'd have a ring per connection and just get an advantage when there's parallel IO going on for a single query? or these per-connection processes wouldn't directly do IO but send it via IPC to some IO-handling thread/process? Not sure either of those models are actually an improvement over the status quo, but who knows.
That's not needed to benefit from io_uring
> for you to give it a bunch of IO to do at once (batches and chains), and your threads to do other work in the meantime.
You can see substantial gains even if you just submit multiple IOs at once, and then block waiting for any of them to complete. The cost of blocking on IO is amortized to some degree over multiple IOs. Of course it's even better to not block at all...
> If your code is instead designed to be synchronous and thus can only do one IO at a time and needs a syscall to block on it, I don't think there's much if any benefit in using io_uring.
We/I have done the work to issue multiple IOs at a time as part of the patchset introducing AIO support (with among others, an io_uring backend). There's definitely more to do, particularly around index scans, but ...
> > io_uring is designed for systems that have a thread and ring per core
> That's not needed to benefit from io_uring
90% sure I read Axboe saying that's what he designed io_uring for. If it helps in other scenarios, though, great.
> Of course it's even better to not block at all...
Out of curiosity, is that something you ever want/hope to achieve in PostgreSQL? Many high-performance systems use this model, but switching a synchronous system in plain C to it sounds uncomfortably exciting, both in terms of the transition itself and the additional complexity of maintaining the result. To me it seems like a much riskier change than the process->thread one discussed here that Tom Lane already stated will be a disaster.
> We/I have done the work to issue multiple IOs at a time as part of the patchset introducing AIO support (with among others, an io_uring backend). There's definitely more to do, particularly around index scans, but ...
Nice.
Is the benefit you're getting simply from adding IO parallelism where there was none, or is there also a CPU reduction?
Is having a large number of rings (as when supporting a large number of incoming connections) practical? I'm thinking of each ring being a significant reserved block of RAM, but maybe in this scenario that's not really true. A smallish ring for a smallish number of IOs for the query is enough.
Speaking of large number of incoming connections, would/could the process->thread change be a step toward having a thread per active query rather than per (potentially idle) connection? To me it seems like it could be: all the idle ones could just be watched over by one thread and queries dispatched. That'd be a nice operational improvement if it meant folks no longer needed a pooler [1] to get decent performance. All else being equal, fewer moving parts is more pleasant...
[1] or even if they only needed one layer of pooler instead of two, as I read some people have!
> Out of curiosity, is that something you ever want/hope to achieve in PostgreSQL? Many high-performance systems use this model, but switching a synchronous system in plain C to it sounds uncomfortably exciting, both in terms of the delta and the additional complexity of maintaining the result. To me it seems like a much riskier change than the process->thread one discussed here that Tom Lane already stated will be a disaster.
Depends on how you define it. In a lot of scenarios you can avoid blocking by scheduling IO in a smart way - and I think we can quite far towards that for a lot of workloads and the wins are substantial. But that obviously cannot alone guarantee that you never block.
I think we can get quite far avoiding blocking, but I don't think we're going to a complete asynchronous model in the foreseeable future. But it seems more feasible to incrementally make common blocking locations support asynchronicity. E.g. when a query scans multiple partitions, switch to processing a different partition while waiting for IO.
> Is having a large number of rings (as when supporting a large number of incoming connections) practical? I'm thinking of each ring being a significant reserved block of RAM, but maybe in this scenario that's not really true. A smallish ring for a smallish number of IOs for the query is enough.
It depends on the kernel version etc. The amount of memory isn't huge but initially it was affected by RLIMIT_MEMLOCK... That's one reason why the AIO patchset has a smaller number of io_uring "instances" than the allowed connections. The other reason is that we need to be able to complete IOs that other backends started (otherwise there would be deadlocks), which in turn requires having the file descriptor for each ring available in all processes... Which wouldn't be fun with a high max_connections.
> Speaking of large number of incoming connections, would/could the process->thread be a step toward having a thread per active query rather than per (potentially idle) connection?
Yes. Moving to threads really mainly would be to make subsequent improvements more realistic...
> That'd be a nice operational improvement if it meant folks no longer needed a pooler [1] to get decent performance. All else being equal, fewer moving parts is more pleasant...
You'd likely often still want a pooler on the "application server" side, to avoid TCP / SSL connection establishment overhead. But that can be a quite simple implementation.
I've had a similar situation with PHP, where we had written quite a large engine (https://github.com/Qbix/Platform) with many features (https://qbix.com/features.pdf) . It took advantage of the fact that PHP isolated each script and gave it its own global variables, etc. In fact, much of the request handling did stuff like this:
Q_Request::requireFields(['a', 'b', 'c']);
$uri = Q_Dispatcher::uri();
$this->getContext()->request()->requireFields(['a', 'b', 'c']);
$this->getContext()->dispatcher()->uri();
amp
reactPHP
Swoole (native extension)
Fibers (inside PHP itself)
Then I realized... we would have to go through the entire code and redo how it all works. We'd also no longer benefit from PHP's process isolation. If one process crapped out or had a memory leak, it could take down everything else.
There's a reason PHP still runs 80% of all web servers in the world (https://kinsta.com/blog/is-php-dead/) ... and one of the biggest is that commodity servers can host terrible PHP code and it's mostly isolated in little processes that finish "quickly" before they can wreak havoc on other processes or on long-running stuff.
So now back to postgres. It's been praised for its rock-solid reliability and security. It's got so many features and the MVCC is very flexible. It seems to use a lot of global variables. They can spend their time on many other things, like making it byzantine-fault-tolerant, or something.
The clincher for me was when I learned that php-fpm (which spins up processes which sleep when waiting for I/O) is only 50% slower than all those fancy things above. Sure, PHP with Swoole can outperform even Node.js, and can handle twice as many requests. But we'd rather focus on soo many other things we need to do :)
https://symfony.com/doc/current/components/process.html
Going from a working process implementation to async/threads with shared memory is pretty much always a mistake IMHO, especially if it's only done for performance reasons. Any speed gains will be eclipsed by endless whack-a-mole bug fixes, until the code devolves into something unrecognizable. Especially when there are other approaches similar to map-reduce and scatter-gather arrays where data is processed in a distributed fashion and then joined into a final representation through mechanisms like copy-on-write, which are supported by very few languages outside of PHP and the functional programming world.
The real problem here is the process spawning and context-switching overhead of all versions of Windows. I'd vote to scrap their process code in its entirety and write a new version based on atomic operations/lists/queues/buffers/rings with no locks and present an interface which emulates the previous poor behavior, then run it through something like a SAT solver to ensure that any errata that existing software depends on is still present. Then apps could opt to use the direct unix-style interface and skip the cruft, or refactor their code to use the new interface.
Apple did something similar to this when OS X was released, built on a mostly POSIX Darwin, NextSTEP, Mach and BSD Unix. I have no idea how many times Microsoft has rewritten their process model or if they've succeeded in getting performance on par with their competitors (unlikely).
Edit: I realized that the PHP philosophy may not make a lot of sense to people today. In the 90s, OS code was universally terrible, so for example the graphics libraries of Mac and Windows ran roughly 100 times slower than they should for various reasons, and developers wrote blitters to make it possible for games to run in real time. That was how I was introduced to programming. PHP encapsulated the lackluster OS calls in a cross-platform way, using existing keywords from popular languages to reduce the learning curve to maybe a day (unlike Perl/Ruby, which are weird in a way that can be fun but impractical to grok later). So it's best to think of PHP more like something like Unity, where the nonsense is abstracted and developers can get down to business. Even though it looks like Javascript with dollar signs on the variables. It's also more like the shell, where it tries to be as close as possible to bare-metal performance, even while restricted to the 100x interpreter slowdown of languages like Python. I find that PHP easily saturates the processor when doing things in a data-driven way by piping bytes around.
My $0.02
If a bug in PostgreSQL (or in an extension) causes the server to crash, then only that process will crash. Postmaster will detect the child process termination, and send an error message to the client. The connection will be lost, but other connections will be unaffected.
It's not foolproof (there are ways to bring the whole server down), but it does protect against many error conditions.
It is possible to trap on some exceptions in a threaded environment, but cleaning up after eg. an attempted NULL pointer dereference is going to be very difficult or impossible.
The other thing that might be interesting is FUTEX_SWAP / UMCG. Although it doesn't remove the overhead induced by context switches entirely (specifically, you would still deal with TLB misses), you can avoid dealing with things like speculative execution exploit mitigations.
https://learn.microsoft.com/en-us/azure/azure-sql/database/h...
I feel like the thread vs process thing is one of those pendulums/fads that comes and goes. I’d hate to see PG go down a rabbit hole only to discover the OS could be modified to make things go better.
(I understand not all PG instances run on Linux, just using it as an example)
In this context threads can be understood as processes that share the same address space and vice-versa processes as threads with separate address space.
One gives you isolation, the other convenience and performance. Either can be desirable.
What would you change about this?
Linux multi-process is already pretty efficient compared to Windows. However, multi-process is inherently less efficient than multi-thread due to more safety predicates / isolation guaranteed by the kernel, I feel lowering it might lead to more security issues, similar to how Hyper Threading triggered a bunch of issues with Intel Processors.
That said, please understand that I'm just being curious - I really don't know what I'm talking about, I haven't built a Linux kernel or dabbled in Unix internals in like 20 years, but thanks for replying :) Postgresql is my favourite open source project and I'm spooked by the threading naysayers.
This isn't really an OS issue, more a hardware one, although potential hardware improvements would likely have to be explicitly utilized by operating systems.
Note that the TLB issue is different from the data / instruction cache situation.
Rewriting in Rust would be interesting, but it would also probably be too invasive to make it worthwile at all - all code in PostgreSQL is C, while not all code in PostgreSQL interacts with the intrinsics of processes vs threads. Any rewrite to Rust would likely take several times more effort than a port to threads.
Using threaded model is difficult, how about pre-fork? Some connections per process is a good improvement.
I know nothing of PG internals but I can see why process per connection model doesn't work for large machines and/or high number of connections. One way to do it would be to keep connection handling per thread and still keep multiprocess approach where it makes sense for security and doesn't add linear overheads.
Are there a much larger percentage of users that really need this?
The deep dive[1] has some details; the tl;dr: is that the main process only has to output Write Ahead Logs to the durable storage layer which minimizes transaction latency. The log processing service materializes postgres-compatible on-disk blocks that read-only replicas can read from, with a caching layer for block reads which sends cache invalidations from the LPS to read replicas.
I'm not sure if similar benefits could be seen within a single machine; using network DMA or even rDMA to transfer bytes to and from remote machines also avoids TLB invalidation. There are some mentions in the mailing list of waiting for Linux to support shared page mappings between processes as a solution.
I'm not exactly sure I understand the reasoning behind process separation as crash recovery; as far as I understand each connection is responsible for correctness and so if a process crashes there seems to be an assumption that the database can recover and keep working by killing that process but that seems like it risks silent data corruption; perhaps it's equivalently mitigated by separating materialization of blocks from the sync'd WAL in a separate process from the multithreaded connection process producing WAL entries?
[0] https://cloud.google.com/alloydb [1] https://cloud.google.com/blog/products/databases/alloydb-for...
You're just defending your hero who has gone rogue.
> You're just defending your hero who has gone rogue.
That doesn't sound like judging an idea on its merit, it sounds like judging a person for something you haven't analyzed yourself.
sounds like groupthink
Anyway, I think there are definitely limits that are starting to appear with Postgres in some spots. This is probably one of the most difficult possible solutions to some of those problems, but even if they don't switch to a fully threaded model, being more CPU efficient, better connection handling, etc will all go a substantial way. Doing some of the really hard work is better than none of it, probably.
;-)
We are in a unique time period and generation that has strong opinions based on a history that only exists within itself.
At some point it needs to rewrite.
TLBs are "just a detail" of roughly 100% of server, desktop, and mobile CPUs.
> Also, aren't they per core and not per process? What would that solve then to switch to MT?
TLB entries are per address space. Threads share an address space, processes do not.
If you want to have a software _engineering_ discussion about the trade offs involved in sharing global mutable state, this is a good venue for that.
All engineering is trade offs. As soon as you start telling people they’re making blanket statements that X is always bad, you’ve transitioned into the realm of nitpicking.
PostgreSQL uses a process model. So the article just states that globals work fine for PostgreSQL
> Knizhnik has already done a threads port of PostgreSQL. The global-variable problem, he said, was not that difficult.
I see no big problem based on information from person who did some porting already.
The fact is sometimes it's good. Saying it's universally bad is going beyond the realm of logic and evidence and into the realm of ideology.
If you want to contribute to the discussion, I'd be happy to be given an example of successful usage of global variables that made a project a long term success under changing requirements compared to the alternatives.