It doesn't in itself give you any specific rights, as far as I can tell ("e-residency" is a bit of a marketing term). It just makes it easier to do things that foreigners are theoretically allowed to do, but which aren't currently very easy to do remotely. For example it's already legal for a foreigner to register a company in Estonia, but it's a bit of a hassle if you aren't an Estonian resident; this is supposed to make it easier, because you'll be electronically "in the system" with a proper ID number and login, making it easier for agencies and financial institutions to know how to deal with you. Especially since it's combined with a move towards self-service online portals for government services, so you should be able to log in with this ID and do a lot of things over the internet, instead of visiting civil-service offices.
Here (Denmark) all that is also theoretically true—anyone with legitimate business in Denmark can get an ID, and once you have one you can do things like register a company entirely online, self-service, as well as interact with banks and anyone else who implements the "NemID". But in practice the authorities are very suspicious of anyone not living in Denmark trying to get into the system, so almost the only non-residents who do so are Swedes and Germans who either commute cross-border or have frequent business in Denmark.
Although the first stages will be more towards foreigners who conduct any business in Estonia, the next stages should allow appearance of new business models and internationally operated activities headquartered and registered in Estonia.
I would love to see Estonia take on this challenge as well, but it will take a long time and concerted effort.