Why don't test against a local SQLlite Database or a Database that is created on the fly. I think you should definitely test the Database logic.
This has many reasons why you'd want to do it. Many people stick to "performance" but I think that's the weakest reason. Ideally if you change some code and break something you want the method name of a failed test to tell you what's wrong. If you change some ORM code and 100% of your tests fail, is that helpful? Compared to the case where you change the ORM code and the unit tests testing the ORM code fail?
> you change some ORM code and 100% of your tests fail, is that helpful?
Yes. It shows what depends on the ORM behavior. Of course you need to be smart about finding the root cause.