If the implant has sharp edges, or rips into the surrounding tissue, or otherwise causes cells to die or prevents repair of damaged tissue somehow, the immune system will go looking for a culprit[1]. If it decides that the implant is the culprit, bad luck. It will attempt to surround the implant and expel it. Depending on the size this may or may not work (this is what happens to splinters of wood that get into your body, for example). This can even cause the immune system to decide a bone is the culprit, if it does you have rheumatoid arthritis (it's a condition that you are very likely to get if you live long enough)
Furthermore, implants may actually infect, as in a bacterial colony may develop on their surface (because the contact surface with the immune system is much smaller, an infection has a bigger chance to maintain that position. Always keep in mind that the immune system wins against bacteria not because it's better or smarter, but because it has 80kg of cells working logistics for it). The same can happen to bones.
[1] To be exact, cells have an "alarm" signal molecule floating inside them. If the cell is ordered to commit suicide (apoptosis), it will put out an enzyme that neutralizes this alarm molecule before ripping itself to shreds, the same will happen if the immune system orders a cell to eat itself (autophagy). But if the cell dies for some other reason (e.g. getting ripped to shreds, explosion due to infection, ...) the alarm signal will be spread into the surrounding tissue where the immune system can find it. When it finds it, it will come in force to check out the situation. If it keeps seeing this signal without finding a cause, the immune system will go insane/react badly.