There's a well-documented reason for the placement of (nearly) every single character in ASCII. Unfortunately, not all of the reasons are still very compelling. In fact, a distressingly large number of them have to do with the various techniques for encoding bytes on punch cards.

For the curious, I recommend the book "Coded Character Sets: History and Development" by Charles E. Mackenzie. It's an older book, and so it's about the pre-Unicode, US-centric codes. Also, most of the pages focus on the many variants of EBCDIC rather than the stabler ASCII standard. But I guarantee that it will give you a lot of historical perspective.