> I've built tiny servers, parsed plethora of file formats, calculated maths, wrote data, (de)compressed stuff, encrypted things, ...

If you've written code to do all those things, I think you'd be justified in calling yourself a programmer, even if software isn't your main gig. :)