There are a lot of things it can't do. Try making it sort a list in some order that isn't repeated repeatedly in the training set (e.g. not U.S. states in alphabetical or reverse alphabetical order)
You can even ask if it is able to do the task and it will tell you that it can, but it can't, any more than it can help you make a neutron bomb. The difference is it will tell you it can't make a neutron bomb.
LLMs shine at problems that involve linear relationships where you go mostly from one end to the other, like language translation. Give them something that involves complex relationships and they quickly fall apart.