You are missing one important point.
Your network can learn some dataset very well. However, that doesn't say anything about how well it generalizes, and thus how useful your network is.
Your network can learn some dataset very well. However, that doesn't say anything about how well it generalizes, and thus how useful your network is.