I absolutely think it is.

Why add extra complexity into your system (and complexity which I've seen the smartest of engineers get wrong on the invalidation side just by overlooking one small detail) before you know your macro-level bottlenecks and product questions like "how many users will even use this feature"? Additionally caching can mask more serious systemic performance issues like missing database indices and accidental extra round trips to the database.

Better to focus on fundamentals and keep your code simple, then add caching only when it's needed.