You can run your caldav/carddav service in your local network, if you are so inclined. You will get local sync, and the client authors do not have to build specific desktop-based tools.
I for one do not want to return 15 years back, to serial/irda/bluetooth sync, having to think about sync, and eventually forgetting. Running a local service is much preferable, with devices syncing themselves as they see fit.
At the end of the day, I want all my personal events, contacts, todos, and notes on my laptop and able to sync directly with my phone. I'm happy enough with my current bluetooth sync and wouldn't trade it for UI changes.
But for running on your laptop? Yes, nextcloud is overkill, it is not contacts/calendar server in the first place. I would look for smaller, more focused tools instead, baikal for example. For another inspiration, you can have a look at what davx5 -- the caldav/carddav provider for android -- tests against (https://www.davx5.com/tested-with).
It's very easy to install and does not require a DB. As a bonus, it stores everything as files which can be read and edited manually. It does require python.
The main issue is that I only touch it every five years or so, and at that point, I've forgotten what it is called or where it is running!
(I use synology's HyperBackup to E2E encrypt + backup it and a bunch of other docker containers.)
Here is a list of 11 options: