The lock-in is where the catch is right.
The managed services are a huge part of the lock-in with AWS services, because even though they generally cost more than the competitors, they can often come out equal after accounting for those aggregate hours lost to doing those things.
In this case, there is basically 0 switching cost since relational databases are everywhere and you can always take your data/schema with you. This new migration service actually makes it even easier than before.
There is no lock-in.
If you want to talk lock-in, look at services like Lambda where there isn't a direct counterpart (although the model there is so simple that most users could port relatively quickly).
You can't really accuse a vendor of lock in just because his product is priced lower than it would cost you to do it yourself.
Per (not authoritative, obviously) wiki:
"In economics, vendor lock-in, also known as proprietary lock-in or customer lock-in, makes a customer dependent on a vendor for products and services, unable to use another vendor without substantial switching costs."
I agree with this. It shouldn't require that the service is purposefully preventing you from changing; lock-in can occur because a service offers advantages that prevent changing by virtue of some other cost.
If I say I'll pay you $1 if you sit in my office for an hour, you can't say that you were "locked in" by me, if Mary offers you $2 to sit in her office, you can walk over to her office and get more money.
You can backup/restore and run your database on basically any hosted service so switching costs are trivial here.
I can see something like DynamoDB being vendor lock-in since you can't just download your data into your own self-hosted DynamoDB instance, but if you're using a standard off-the-shelf RDS database (MySql, PostgreSQL, Sql/Server, etc), I don't think you can call it lock in when you're staying only because it's easier than running it yourself.