Excellent question! Article 17 of the regulation (http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:320...) lists some exceptions when the processor (the entity holding your data, e.g. Facebook) is allowed to keep some of your personal data. Most notably, sentence 3b states that data might be kept "for compliance with a legal obligation...", which includes keeping of tax records. Also, as the other poster already said the regulation does not apply in the same way to public institutions like the tax office.

However, the processor would not be able to keep any data beyond what is needed for the purpose for which the exception holds, and the data cannot be used for anything else.

"Dear Mr tax man. I request that you delete me."

Financial institutions and the government are exempt for "necessary information".