Connecting a DB might be a bad idea only currently, until cloud providers implement some sort of RPC SQL layer.

We are successfully running Lambda functions connecting to AWS Aurora MySQL using IAM authentication. It has some quirks but after figuring it out, works well.

Just run a connection pool with only 1 connection and expect some additional latency on cold start, though it's negligible compared to loading libraries, runtime, etc.