Hold your hopes.
Citrus plants are known for being extremely easy to hybridize. If there is a non-loquat citrus tree around your neighbour's house, then it is very likely that you will not have loquats in your new tree. It might very well be a hybrid of something else with a loquat. It will take you about three years to discover that.
If you want to be 100% sure you're growing a loquat, you'd better grow it from a branch cut.