My understanding was that it's a ligature of the letters 'et', Latin for "and" (you can see it pretty clearly in some scripts, but it's not at all obvious in the typeface we're using here). But then, there used to be a letter for 'th', too.
Actually, there used to be two separate letters for ‘th’, Ð/ð (“eth”) and Þ/þ (“thorn”). Þ hung around longer than Ð, though.
Interestingly, the IPA symbol for the "th" in "the" is eth, but the symbol for the "th" in "think" is θ (Greek theta)
"The" = /ðə/ "Think" = /θɪŋk/
Not sure why that is, I'm only an amateur linguist
It states that the consonants of https://en.wikipedia.org/wiki/Romic_alphabet are the same as were adopted by IPA, except "⟨ñ ᵹ ɹ λ ꞁ⟩ were eventually replaced with ⟨ɲ ɣ ʕ ʎ ʟ⟩ in the IPA". No mention of ⟨θ⟩ in that sentence.
However, looking at the table on that page, the "th" in "think" is represented by ⟨þ⟩!
Going back to the first version of IPA on https://en.wikipedia.org/wiki/History_of_the_International_P... the "th" in "think" is represented by ⟨θ⟩.
Looking at the other two major early influences mentioned in that article, https://en.wikipedia.org/wiki/English_Phonotypic_Alphabet and https://en.wikipedia.org/wiki/Palaeotype_alphabet neither uses ⟨θ⟩ for that sound either. By the looks of it, English Phonotypic Alphabet uses a capital theta to represent a vowel sound!
When thorn fell out of use we replace it with y for these archaic spellings. But it was always “the”.
Which, with time, turned into the ”Ye” of ”Ye olde <thing>” [1]