That's the whole point. You can repurpose the buildings within reason, that's what they've done in Porto where I've spent some time. Porto used to have lots of derelict buildings in its Heritage area, they've been rebuilt according to strict rules and it's fine.