mannykannot

So, nothing to see here so far - I can't really respond to allegations that are imaginary.

You also claim that I am begging the question. How do you justify that? It is not, of course, begging the question for opponents of Searle to suppose his conclusion is wrong: everyone disputing any argument does that.

The Enigma response is very straightforward. While, in general, simulations are not equivalent to what is being simulated, it is often the case that for information manipulation they are, owing to the substrate independence of information. It is Searle who needs a better argument here, and he never came up with one.

Then you will have no difficulty in pointing out where that happens. While you are about it, you can point out where you think I said people can read & interpret numbers by imbuing them w/ actual meaning & semantics.

For example, this response fails to give any justification for your claim that I am not addressing the actual argument.

The question of whether, or to what extent, LLMs understand anything is an interesting one, tied up with our difficulty in saying what 'understanding' means, beyond broad statements along the lines of it being an ability to see the implications of our knowledge and use it in non-rote and creative ways.

The most honest answer to these questions I can give is to say "I don't know", though I'm toying with the idea that they understand (in some sense) the pragmatics of language use, but not that language refers to an external world which changes according to rules and causes that are independent of what we can and do say about it. This would be a very strange state to be in, and I cannot imagine what it would be like to be in such a state. We have never met anybody or anything like it.

If I'm following you correctly, you are saying that the conclusion of Chalmers' parody is actually correct, as having a recipe is indeed not sufficient to successfully bake a cake: you will not succeed without the ingredients, for example.

This is indeed true, but we should bear in mind that Chalmers' parody is just that: a parody, not a rigorous argument. It seems clear that, if Chalmers wanted to make it more rigorous, he would have concluded with something like "therefore, even if you have all the prerequisites for baking a cake (ingredients, tools, familiarity with basic cooking operations...), no recipe is sufficient to instruct you in successfully completing the task." This would be a better argument, but a flabbier, less to-the-point parody, and it is reasonable for Chalmers to leave it to his readers to get his point.

The great thing about your latest reply is that it takes no time at all to see that you have still not offered any justification or explanation of anything you have claimed.

Once you punted a second time on an opportunity to explain yourself, I was fairly confident that there was nothing there. It's a common pattern.

So no argument or explanation from you, just unsubstantiated allegations and bluster.

I'm showing that Searle's argument against the simulation reply doesn't hold up against relatively straightforward scrutiny. If you think you know of a better one, present it.

To be clear, I don't think the guy in the room will end up understanding whichever Chinese language this thought experiment is being conducted in, either.

You have put your finger on the fundamental problem of the argument: Searle never gave a good justification for the tacit and question-begging premise that if the human operator did not understand the language, then nothing would (there is a second tacit premise at work here: that performing the room's task required something to understand the language. LLMs arguably suggest that the task could be performed without anything resembling a human's understanding of language.)

Searle's attempt to justify this premise (the 'human or nothing' one) against the so-called 'systems reply' is to have the operator memorize the book, so that the human is the whole system. Elsewhere [1] I have explained why I don't buy this.

[1] https://news.ycombinator.com/item?id=45664129

Enough to show that Searle's successors need a better argument.

LLMs are capable of performing the task specified for the Chinese room over a wide range of complex topics and for a considerable length of time. While it is true that their productions are wrong or ill-conceived more often than one would expect from a well-informed human, and sometimes look like the work of a rather stupid one, the burden now rests on Searle's successors to show that every such interaction is purely syntactic.

Strictly speaking, it does not even prove that: the claim that the guy in the room will not end up understanding Chinese is a premise, and some people argue that it an unjustified one. Personally, I think Searle's argument fails without one having to resort to such nit-picking.

Searle tries this approach against the 'simulation agument' in this paper (see page 423) and also elsewhere, saying "a simulation of a rainstorm will not get you wet" (similarly, Kastrup says "a simulation of a kidney won't pee on my desk"), to which one can reply "yet a simulation of an Enigma machine really will encode and decode messages."

The thing is, minds interact with the outside world through information transmitted on the peripheral neural system, so the latter analogy is the relevant one here.

It proves nothing, and has, in fact, been overtaken by events (see another of my replies [1]).

As for its alleged conclusion, I love David Chalmers’ parody “Cakes are crumbly, recipes are syntactic, syntax is not sufficient for crumbliness, so implementation of a recipe is not sufficient for making a cake.”

[1] https://news.ycombinator.com/item?id=45664226

The irony here is that performing like an LLM the very thing that Searle has the human operator do. If it the sort of interaction that does not need intelligence, then no conclusion about the feasibility of AGI can be drawn from contemplating it. Searle’s arguments have been overtaken by technology.

People are doing things they personally do not understand, just by following the rules, all the time. One does not need to understand why celestial navigation works in order to do it, for example. Heck, most kids can learn arithmetic (and perform it in their heads) without being able to explain why it works, and many (including their teachers, sometimes) never achieve that understanding. Searle’s failure to recognize this very real possibility amounts to tacit question-begging.

The point that your scheme requires just one date to select single newspaper as a key for one U-boat per patrol underscores how feasible it would be for an organization with the resources of the German navy to generate the same sized random keys, your personal experience with key generation notwithstanding.

As for key interdiction, it would have to be at massive scale, as having a handful of random keys does not enable industrial-scale decryption, which is what the allies needed (and had, some of the time) for signals intelligence to be useful against U-boats. In contrast, the leaking of one date would compromise the entirety of the communications conducted under it (i.e., at least those for one U-boat for the duration of the patrol), especially given that German newspapers were readily available in neutral and occupied countries. OTPs are, in fact, way more resistant to espionage than your newspaper scheme.

The failure of the location-shifting scheme (as mentioned in my previous post) to stop the allies interdicting rendezvous, despite its periodic changes, also suggests that one date per patrol is not going to be good enough.

We have reached the point where the alleged superiority of your newspaper scheme hangs on the tendency of dice to roll off your desk!

The German navy did not use latitude and longitude in their messages; it had a coded grid system (the Gradnetzmeldeverfahren.) As you guessed, they also used secret offsets which were changed periodically. Nevertheless, by making use of captured material, the allies became quite successful at figuring out the actual coordinates, suggesting that the German navy was somewhat complacent about the effectiveness of these measures.

As for times, my understanding that they would, at least sometimes, specify them in the form of so many hours after a specific trigger message was received. This seems to require the U-boat to be surfaced (or maybe just at periscope depth?) from the start of the window for this message until it was received.

None of this addresses the training issue from earlier. The thing is, if you are thinking of using a newspaper-based scheme only for U-boats, you might as well go the extra distance to implement a true OTP system, which seems feasible at that scale and which would turn the use of Enigma in series into an academic exercise.

I feel it is important to recognize that schemes like this are not OTP, and drawing an analogy between the two risks inducing false intuitions.

This discussion started with a proposal to back up Enigma with one-time pads, but harshreality pointed out that OTPs are not vulnerable to anything except compromise of the OTP itself. This has the unstated corollary that if you are using a true OTP system, adding Enigma to the process does nothing to improve security (unless your pre-shared keystream is compromised - and even then, capturing one U-boat's OTP will not compromise any other's communication.)

You then raised the concern of generating keys in sufficient quantity, which is certainly part of the problem (though I suspect that an electro-mechanical solution for that problem was well within the capabilities of contemporary technology, especially as, by then, Konrad Zuse had produced the first digital computer.) If, however, we are restricting our source of keys to the amount of text in a daily newspaper (or even all of the Third Reich's daily newspapers combined), that is something that could be achieved by a corps of dice-rollers, if it came to that, which would avoid one of the other problems of using newspapers: the statistical regularities of newspaper text.

Even then, you still have the problems of key reuse [1] and key distribution, and another which I think might be by far the hardest: training people to use it. Even just considering the submarine fleet, at least one person on each U-boat would have to be trained in properly using the technique. This would delay implementation, and once deployed, even with no mistakes, it would be slow in use.

Alternatively, a machine to do the work might have been developed (together with another to generate physical machine-readable keys), but that itself would have meant considerable delays in implementation.

In view of this, I feel that the measure actually adopted - adding another rotor position to the Enigma machine - was one of the better options (though it would have been even better if the additional rotor were interchangeable with the others.) This took time to implement and was ultimately defeated, but anything other than an OTP system would likely have the latter problem, and all would have had the former.

[1] Maybe not so much of an issue if one is only dealing with submarine communication, given that most of this was with the U-boat High Command in Germany, and assuming that it was of sufficiently low volume for each U-boat to be be given its own unique set of keys for each patrol.

I don’t know much about encryption, but I can see a couple of concerns about such a scheme.

The first is that such keys will have all the statistical regularities of the German language, which I believe is problematic, even though I don’t know how one would go about exploiting it.

The second is the matter of how much encrypted text had to be transmitted every day, by the German military as a whole. If it significantly exceeded the daily output of Germany’s newspapers (and I would guess it did) there would seem to be considerable key reuse under this scheme.

For submarines and other units not receiving newspapers daily, there also seems to be a key-distribution issue. I don’t know if there is a better way to guarantee that communication can be maintained through a patrol than to depart with the equivalent of a stack of old newspapers. Is this a problem? I don’t know, but if the allies had figured out the broad outlines of the scheme, I imagine they might be able to do some preparation in anticipation of messages being intercepted.

You are right about 'malicious'. 'Dangerous', however, is a different matter.

Then it would better be described as a life-threatening event rather than a bricking - especially as, in the hierarchy of concerns, the former is more serious than the latter.

I paused at the statement "Add $55 for free domestic shipping" on the order form.

The paperback edition I read in my 20s had an appendix containing several reviews of the novel, including that by D.H.Lawrence, briefly quoted here. Well worth reading in its own right, IMHO.

https://www.gutenberg.org/files/60547/60547-h/60547-h.htm

The establishment? That sounds so 20th. century to my ears.

Not as far as I know, but that is not saying much.

Good point - maybe the analogy to computation arises simply because digital computation and the synthesis of DNA, RNA and proteins are all performed by discrete-state machines?

I suppose you are aware that, for many uses of LLMs, the propensity for hallucinating is a problem (especially if this is not properly taken into account by the people hoping to use these LLMs), but this then leaves me puzzled about what you are asking here.

I feel the same way, but again, this has no bearing on the issue of the medical soundness of the department's positions and policy.

Going back to the top of this thread, I agree that correlation does not mean causation, but in this case, we do not yet even have a justifiable claim of correlation, and RFK Jr. has repeatedly demonstrated his propensity for substituting unfounded opinion and incorrect data for empirical facts, if it suits his agenda.