For the same resolution, TVs often have a larger pixel pitch than monitors. I haven't noticed this as an issue for 4K tvs, but there are a few 1080p tvs that have pretty bad pixel pitch. My 32" 1080p tv has horrible pixel pitch. It is really only noticeable at the distances you'd view a monitor from, but it's really annoying.
Monitors compete on things like gray-to-gray response time, ghosting and input lag. TVs care about ghosting, but contrast ratio and black level is a bigger concern for them. I find that TVs have a fairly noticeable input lag, even when in game mode, but I'm really sensitive to this issue.
TVs tend to be slightly larger than monitors for a given screen size. It's not a huge difference, but if you want to do multi-display setups the bezel width can be a difference.
My best friend uses a TV as a monitor and is not bothered by any of these issues in the least, so it basically comes down to personal preference.
Another note is that you'll likely want to turn off some of the features that benefit you on the TV side of things (motion flow, image sharpening, etc). These features actually make the experience worse when using a TV as a monitor.