Hacker News Reader

Comment by SemanticStrengh | Hacker News Reader

can't they use the zipf law ? https://en.wikipedia.org/wiki/Zipf%27s_law The decreasing exponential law should allow to find "the" and some closed form POS words, so yeah determiners, prepositions and conjonctions.

View on HN

You really think that in decades of linguists studying Linear A, no one has thought of trying Zipf's law?

If scientists have studied something for this long, and you come up with an idea that fits in a single paragraph, it's probably been tried and didn't work. Unless you're the field's leading expert in which case you would be off doing it, not posting it on HN :)

Edit: typos

I understand the impulse to point out the obvious, but when the question is asked honestly rather than arrogantly or dismissively, it is even better to wait for someone to provide the specific answer; in this case, the reason that Zipf's law is of no help.

It wasn't my intent to be overly dismissive. But I see this sort of thing all the time, and I find this phenomenon interesting, so I wanted to engage with that aspect of it, specifically.

I agree with you in general though. Dismissing these things out of hand isn't helpful either. But multiple people had already made substantive replies to the actual content of their idea, anyway.

useless dismissal, I made a question not an affirmation. Besides it allow for an exploration of the search space of solutions, which stimulate the depth of the discussion and might allow finer grained questions that would then become possibly innovative

Hardly useless, since it apparently inspired people to dig deeper and find more info :)

Neither of these areas are my field, so I could be entirely misunderstanding this preprint [1] from 2021. The preprint mentions using Zipf's law in the objectives section on attempting to deciphering Linear A.

The literature survey section mentions there have been good results using computational methods in 2020 to automatically decipher Linear B. The discussion section mentions "To the best of our knowledge, this is the first study to discuss and show computational analysis of Linear A."

Again, neither of these are my fields, but it looks like if these linguists have tried to use Zipf's law or other computational methods unsuccessfully in deciphering Linear A, the results weren't published. (Or a poor literature survey, or other explanations...) I'm not an academic either, so I don't know what the practices are for publishing unsuccessful results.

[1] https://hal.archives-ouvertes.fr/hal-03207615/document

It's showing that we had to wait for 2021 for them to try it.. Thanks for reporting anyway!

Interesting. If they have only very recently tried Zipf's law then there may be some other more advanced stuff they haven't tried.

I'm thinking word embeddings. Like maybe you could do a word embedding based on cooccurence and look for similarly shaped clusters in Linear A and Early Greek.

That looks like an extremely underwhelming bit of research.

Some thoughts:

1. Figure 2 is titled "Evolutionary Tree of ancient scripts reconstructed using Neighbor Joining algorithm in ClustalW2 (Daggumati et al., 2019)", however checking the references (a) that figure does not appear in the cited paper (b) the cited paper does not use ClustalW2 (c) the scripts in the cited paper do not match those in the figure.

2. Speaking of references, they include Classen, M., & Safrany, L. (1975). Endoscopic papillotomy and removal of gall stones. This seems like an odd choice. Why did they include this? I have no idea, I can't find any cite to it in the text.

3. Immediately after "When text is rendered by a computer, the characters in the text can not all be displayed, because no font that supports them is available to the computer" they write: "The Python package Matplotlib also does not render the font as of the current version 3.3.4." I would have thought that rendering with matplotlib would have counted as "render[ing] by a computer". Please also note Figure 5, which has a screenshot of a python script and a matplotlib plot; the former displays 5 (presumed) Linear A characters correctly while the latter has only boxes. Perhaps they meant 'there is no font in which all Linear A characters are present'. (And I should point out that in less than sixty seconds of searching I discovered that--as you might expect--matplotlib can use user-specified .ttf fonts)

(Sorry I know this point is long but another thing bothering me: why bother sticking with unicode codepoints when they could have instead just assigned integers to each character?)

4. For reasons that escape me, they chose to use a word cloud in Figure 15 ("Word cloud of locations the Linear A tablets were found by the number of symbols gathered from each") instead of, for example, a table. Why is 'HT' in there at least twice? Why is 'KH' in there at least six times?

I could go on but I'll stop here and say that if I were reviewing this for a journal it would be an immediate rejection.

edit: I hope this will make you think twice next time. Using zipf law for linear A has only been attempted for the first time in 2021 https://hal.archives-ouvertes.fr/hal-03207615/document so had I commented last year it would have been prio art. I agree the idea is not very original and yet we had to wait that much time for it to be experimented.

The total extant corpus of Linear A amounts to fewer than 10,000 characters (and this is, I believe, the largest corpus of any undeciphered script).

There's not enough text to do statistical analysis.

If all that text would have been that of a story, there would still have been a chance to decipher it.

Even worse than the small number of texts is that all, or almost all, are just bookkeeping records, so they contain few words besides numbers, symbols for useful goods, e.g. wine, olive oil, barley, wool and so on, and proper names of places or people.

So even if there might be a few hundreds of texts, most just reproduce the same phrases, only with different numbers and names substituted in them.

Any statistics on this handful of stereotype phrases will offer no information about the statistics of the words of the Minoan language as used in a normal conversation or story telling.

I'm skeptical of the claim that this would work at all even if there was a larger corpus. Let's say you had a million pages of classical Chinese text, but absolutely no context about what the text meant or was about. By looking at it closely and using statistical analysis you could certainly determine various rules of the grammar, and you might even be able to guess that certain characters are grammatical constructions representing things like conjunctions and prepositions. But this isn't really going to let you translate anything.

My guess is that if you had a really big, wide-ranging and high-quality corpus of a completely unknown human language then you probably would be able to decipher and translate it. If you could deduce or guess the grammatical structure the next step might be to look at which nouns can be subjects of which verbs, for example, and it might then be possible to guess which nouns refer to humans and which verbs describe actions that can normally only be performed by a human, and then ... well, there's lots of statistical stuff you can do with a really huge corpus ... It's an interesting problem to think about but it's not a problem we're ever likely to encounter in real life. It's more likely we'll discover a corpus of some alien, non-human language than a huge corpus of a completely unknown human language.

The Voynich manuscript is fairly extensive (over 200 pages), illustrated, and we still do not know what the heck of a language it is written in.

https://en.wikipedia.org/wiki/Voynich_manuscript

...assuming it is really language at all, and not just deliberate gibberish.

entropy analysises seems to indicate it is natural language like so not too much redundancy and not too much noise/randomness.

After seeing your comment, I watched a TED Ed about it[1], and now I am convinced, for it to have survived for so long, it is a relic and time forgot why it was saved. It's Leonardo Da Vinci's childhood notebook, has to be, there is no other explanation for why it would have been kept since the 15th Century. So it's likely to be derivative of Italian, Latin, and other languages he spoke and wrote. No, wait... it very well could have been created by some other weird kid from the 15th Century no one's heard of. Come on.

[1] https://www.youtube.com/watch?v=8NS4CbBJQ84

hey this was really interesting to read. I like the aztec hypothesis, anyway hoax or not it seems it has received a tremendous amount of work at a time where half of those concepts hadn't even been discovered, except of course if they had conserved paper for 200 years, which I find not that unlikely tbh.

Yes, the Voynich manuscript is worth mentioning here, but it doesn't qualify as a really big, wide-ranging and high-quality corpus: it's about 35 thousand words (if the things that look like words are words), whereas a million words would still be a small corpus by today's standards; it's a single document (for example, it might contain no first-person pronouns, no future tenses, almost no mention of individual people, ...); and we have no idea about the quality (for example, perhaps it was copied by someone who didn't understand it and is such a bad copy that even someone who knows the language and the script would find it hard to understand). Still, it is definitely worth mentioning here.

Zipf's law is a statement about the results of a frequency analysis. Frequency analysis has been a core part of Linear A efforts for a long time, as you'd expect considering how important it was to Ventris in deciphering Linear B.

See for example "On the Language of Linear A" from 1958 (https://gredos.usal.es/bitstream/handle/10366/73246/On_the_L...) or "Greek-like Elements in Linear A" from 1963 (https://grbs.library.duke.edu/article/viewFile/11991/4031). For something more recent, there's "Linear A and Linear B: Structural and contextual concerns" from 2017 (https://core.ac.uk/download/pdf/200196986.pdf). (One of the authors of that one, Steele, also has a blog: https://crewsproject.wordpress.com/tag/linear-a/)

That makes assumptions about the language, for the languages I know: German has three definite articles, Latin doesn't have any, so it is not obvious what looking for "the" would result in either.

I think people (here and below) are getting hung up on definite articles, but Zipf's Law makes no such observation. It says only that a word's frequency in a natural language corpus tends to be in inverse proportion to its rank in a frequency table.

In English, the most frequent words are articles, but the general observation about word frequency holds across languages (whether those languages have articles or not).

"The most frequently appearing words in this pile of un-translateable text are the most common words in the language it is written in" seems like it falls somewhere between blindingly obvious, and entirely useless. Unless you have some clue what those words mean, how does that observation help you?

It doesn't (in this case), and I didn't say it did. And there's nothing "blindingly obvious" about the ubiquity of the Zipf curve.

Just from skimming the wikipedia article, it doesn't seem useful for translating. But it is slightly stronger than "The most frequently appearing words in this pile of un-translateable text are the most common words in the language it is written in." It tells you that, for example, the most popular word should be about twice as popular as the second most popular word.

It doesn't tell you what those words are, but it is a pretty specific observation about the frequency/rank relationship. So, as the wikipedia article liked about points out, it can tell us that the Voynich Manuscript was probably written in a language (of course, it could be a cypher of a real language or something made up, like elvish in Lord of the Rings, but it probably isn't just a random collection of symbols because it is unlikely that a random collection of symbols would happen to follow this distribution).

How does this definition use "word"? In analytic languages, most words always appear in the same form, so counting them is relatively easy. But for inflected languages, does this require being able to distinguish the roots of words in order to count them accurately?

It's not just about the presence or absence of articles and prepositions, but about different declensions of the same word. If this analysis requires knowing that homo, hominis, hominem and homine all refer to the same word and should be counted as one, how does it help with analyzing a text for which we don't know the grammar?

> does this require being able to distinguish the roots of words in order to count them accurately? can be somewhat retrieved via leveisthein distance.

> declensions is there a difference between an inflection and a declension?

Can we assume that any of those features would be part of this language?

a language without thoses would be hella weird and primitive, like stereotypical robotic talking. To answer you question, I don't know, do linear B have them?

Latin doesn't have articles (the/a), and frequently drops the verb. Aramaic encodes the article in a suffix. Arabic and Hebrew omit the vowels, leaving the interpretation depending on contextual clues. There are languages without auxiliary verbs. And there are tons of other constructions that English doesn't have.

Written Arabic and Hebrew often omit vowels; the actual spoken languages do not, of course.

The problem isn't in deciphering spoken Linear A.

There are languages used today that don't have a separate word for "the", such as Hebrew (which uses a prefix to denote "the"), or Chinese, which apparently doesn't use articles.[1]

Also, knowing what the most common words are wouldn't really help you much if you didn't know what the documents are about. For example, if they were trade records, they might contain a lot of text saying something like "X agrees to buy 20 pounds of olives from Y for $50 if delivered by next week". But if they were historical records of wars, other words may be more common.

[1] https://mylanguages.org/chinese_articles.php

> a language without thoses would be hella weird and primitive, like stereotypical robotic talking. To answer you question, I don't know, do linear B have them?

I think there are very few assumptions of the form "every reasonable language has […]" that hold up even for all current languages, let alone historical ones.

Anyone claiming "surely every language needs X" had better look at Riau Indonesian (https://en.wikipedia.org/wiki/Riau#Language) first to check if that language has X. If it doesn't, then X is almost certainly not required for communication.

Did you just call all Slavs "weird and primitive"?

Weird, perhaps; primitive, no.

One of the historical issues with linguistics is that it analyzed every language as if it were Classical Latin or Classical Greek, and if that language had elements that didn't work out... well, that can't be proper then, can it? You still see some residuum of this in English prescriptivist poppycock, like the prohibition against ending sentences in prepositions.

As linguists actually started inventorying world languages, it became more and more clear that there is a very wide dichotomy of grammatical features that don't necessarily translate well to familiar languages. There are vanishingly few features that are actually universal to all languages--the noun may well be the only universal part of speech. That a language doesn't choose to mark a feature in a particular way doesn't make it more primitive than another language. English doesn't have a numerical classifier... is it more primitive than an Australian Aboriginal language? Or is it more primitive than Japanese for not having a way to mark register (~ politeness)?

(FWIW, Linear B is used to write Mycenaean Greek, and this has been known for ~70 years.)

LOL, my native language (Czech) has no articles, but it is so flexive and permits so many subtle, meaning-carrying changes in sequence of words in a sentence that it is actually hard to carry over some of those subtleties into written English.

The only thing "robotic" about it is the fact that "robot" is a Czech word that was adopted worldwide.

In addition to the other comments on difficulties of such analyses, an additional difficulty may be the type of inscriptions in the corpus. We understand Linear B, for example, because it is early Greek. But the texts are not narrative prose or poetry: they're administrative records, mostly lists and inventories. If Linear A texts are of similar types, then trying to decipher the language from them alone may be challenging or impossible, unless it can be linked to a known language: the forms of speech used may simply be too limited.

Trying to understand English grammar by looking only at bare financial statements would likely be extremely hard.

The language might not have that. E.g. latin doesn't have a word that corresponds to "the". Languages may indicate conjunctions or some types of prepositions or determiners by changing the case of a noun or the conjungation of a verb.

I think the issue with Linear A is that the amount of preserved units of culture in that language is incredibly small, so whichever the statistics you can obtain from it are limited in use.

I wonder can they construct a deep learning model that encodes human languages from script shapes and then somehow figure out the God language from which Linear A script/language is derived.

There's too little data for Linear A. But it might be enough if there's a God language oracle waiting to be fed new descendant languages.

This is a God (language)-of-the-gaps argument: we can't figure out this rarely language, but maybe we can figure out an entirely unattested language instead, and also learn the correspondence between it and Linear A.

Deep learning can predict plenty of phenomena in the world, sure, but it needs data, not aspirations.

> figure out an entirely unattested language

I did not say that. Human languages evolve in similar ways, use similar vocabulary, grammar etc. Linguistics has already unraveled the structure of many languages and the structure of evolution of language through time.

I am not saying DL is THE approach to take, but given that there's only ~10k characters of Linear A, it is hard to tackle the problem without common representation of multiple languages that are close to it. That's the whole point of DL, how to build better and better representations, not how to accurately model uncertainty (which is what you get by doing statistics).

I would say XLM [0] builds a common representation of a collection of languages and then works better on machine translation for languages for which the data is scarce but that are related to the languages in the model. (what it also does is discover and represent the structure of part-of-speech, grammar, entities etc. without being told about those particular things)

Does there exist an abundance of data for languages close to Linear A? If not, then I admire the work of all that try to untangle this with their brains alone.

0: https://github.com/facebookresearch/XLM

> Does there exist an abundance of data for languages close to Linear A? If not, then I admire the work of all that try to untangle this with their brains alone.

In the article, Dr. Ester Salgarella says: "we have not yet identified the linguistic family the Minoan language belongs to (unless it has to be taken as an ‘isolated’ language)"

If we knew that the Minoan language belonged to some extant language family and we had an abundance of data, the mystery of Linear A would already have been solved decades ago.

In general, there's very little data for any of the Palaeo-European languages that got replaced by Indo-European languages.

Linguistic relatives of the Minoan language could have gone extinct when their speakers shifted to Greek or some other Indo-European language. It is also possible that other Minoan languages died out centuries or millenia before the arrival of Indo-Europeans. I don't believe we will ever know.

Yeah, I guess the conclusion is that humans are still much better than DL. :D

Reply on news.ycombinator.com