Yes, you can, as long as you have an address (Public key). A wallet is just a private key.
The transaction history, and thus the ballance of an address is held in the blockchain, in the ether.
To spend the coins, you must have at hand:
* the current blockchain
* the private key
* a connection to the network.