HNHacker News
TopNewBestAskShowJobs

dawnofdusk

353 karma · joined September 8, 2022

submissionscomments
dawnofdusk··on The controller pattern is awful, and other OO heresy (2013)
Is the problem here not simply that Python does not have a convenient way to namespace functions except a class declaration? The `module` keyword in e.g., Julia or Rust seems like what's desired to avoid this phenomenon of "a class which is just a bundle of static methods".
dawnofdusk··on Building a Virtual Machine Inside ChatGPT
I agree. ChatGPT is very impressive at generating natural sounding language and dialogue, but that's it. If you want to be convinced that it lacks general intelligence, try this prompt

>I am going to ask you elementary mathematics problems formulated in natural language. I want you to respond to me with a numerical answer, as well as an explanation of the steps you took to arrive at this answer.

And then follow it with a problem of your choice which is not completely trivial. For example, it has lots of trouble with the following: "John is in a train moving west at ten miles per hour. Jill is in a train moving north at twenty miles per hour. How fast are John and Jill moving apart from each other?" or a slightly more complicated problem, taken from the MATH dataset: "Tom has a red marble, a green marble, a blue marble, and three identical yellow marbles. How many different groups of two marbles can Tom choose?"

dawnofdusk··on Shannon Entropy Imposes Fundamental Limits on Communication
I think your comments on encoding a sequence of bits/pi is a little misleading. For example, in the case of the first 10 billion digits of pi, you do not even need a program to calculate it. One can just use a lookup table which maps the digits 0 through 9 to the frequency they appear in 10 billion digits of pi, which is <40 bytes of data. You are confusing the entropy of the digits of pi, for which the order does not matter, and the entropy of the sequence of pi, for which the order does matter. In the latter, you would want compare the probability of observing the exact sequence of pi compared to other sequences of numbers. This is an entirely different question. Because the sequence of digits of pi is deterministic, its entropy is 0, i.e., if one had a machine from which you could sample "10-billion digit sequences of pi" you would get the same sequence every time.

>strictly speaking the shortest possible encoding of a sequence of bits could be smaller than this would suggest, since it's possible a pattern exists that has escaped notice.

If it is a deterministic pattern, then the entropy of the sequence is actually 0. Otherwise the existence of patterns is something that the entropy computation already takes into account, and is in fact the entire point. In practice, finite sample sizes introduces noise and means that we can only ever estimate the true entropy, and Jensen's inequality implies that all such estimates are higher than the true entropy of a system, so that we observe even deterministic sequences with noisy filters. This is perhaps the clearer version of your point.

dawnofdusk··on Shannon Entropy Imposes Fundamental Limits on Communication
There is no problem with assuming such a Turing machine exists so long as you compare only measurements of information with respect to that Turing machine. Generally what is relevant to study is entropy differences anyway.

I don't know much about AIT in specific, but it seems that Chaitin 1975 (https://dl.acm.org/doi/10.1145/321892.321894) acknowledges this, as the algorithmic entropy differs from the usual Shannon entropy by a constant term, which one might imagine is the information needed to encode the Turing machine.

I also wanted to mention that because you thought of the information theoretic implications of building a computer, etc., you might be interested to know that this kind of thinking is very interesting and cutting edge in biophysics. Organisms have to essentially build computers to do certain tasks, and in this way they must take into account both Shannon entropy (i.e., information in the abstract sense) and thermodynamic entropy! This is essentially the Maxwell's demon paradox (https://en.wikipedia.org/wiki/Maxwell's_demon).

← PreviousPage 5 of 5