HNHacker News
TopNewBestAskShowJobs

enugu

457 karma · joined February 12, 2016

Google email id : harshavr
submissionscomments
enugu··on India launches attack on 9 sites in Pakistan and Pakistani Jammu and Kashmir
The incentives of the Pakistani generals to permit organizations like LeT to commit further terrorist attacks is a different domain from whatever the local political situation is like in India. There has been a past regime where Pakistani generals were able to train and send militants regularly to conduct terror attacks in India. Without an effective response from India putting pressure on these generals, that can easily become the new normal again.
enugu··on India launches attack on 9 sites in Pakistan and Pakistani Jammu and Kashmir
If you are actually arguing that a country targeted by a terrorist attack does not gain deterrence with a counterstrike relative to letting things go on, then how uniform do you consider this prescription? Should the terror attacks in the US or France not have had a military response?

What happens to the incentives of terror groups in response to such a policy?

---

The role of money only becomes an issue when conducting a terrorist attack becomes expensive. Missiles and jets consume much more money in comparison to training recruits via an intermediary organization like LeT and sending them across the border to carry out attacks.

A regime in which a terror attack leads to a high pressure, expensive situation for the Pakistani military is completely different from regularly scheduled, train and deploy terror attacks from militants which used to happen earlier.

In that situation, the military has to respond to economic pressure, pressure from allies and pressure from its own people.

enugu··on India launches attack on 9 sites in Pakistan and Pakistani Jammu and Kashmir
> They were unable to locate the terrorists even after two or three weeks and needed a distraction.

This does not make sense. When France attacked Daesh in 2015 after the terrorist attacks in Paris or when the US attacked Afghanistan after 9/11, the objective wasn't to target the exact people who carried out the attacks, but the organization behind the attacks. People can always be found as long as the organization remains.

The goal of the attacks would be to make any future terrorist attack an expensive option for the Pakistani military as opposed to something which can be done routinely. There was a sharp drop in the terrorist attacks in Kashmir after the 2019 confrontation.

enugu··on Clojure: Realtime collaborative web apps without ClojureScript
Doesn't re-frame also work with ui as a pure function of state? (https://day8.github.io/re-frame/a-loop/)

So, is the main difference that replicant is not built on top of react?

enugu··on Cursor told me I should learn coding instead of asking it to generate it
Yes, the Turing Test tries to give an operational definition of 'think'- can a computer give the same response as a thinking human.

Further, we can even disallow decisions based on the human/machine signature. The computers don't need to imitate human frailty and examiners decide based on the quality of the output, not whether it feels human or machine. I think that this is also why one of the stipulations is that the human/machine is not physically visible.

But even then, this goal is far away. We are not asking just about simple short chat imitation games but whether a computer can operationally do everything humans are capable of in the textual medium. This often requires experts.

The chat game version can be solved. But, the foundational goal is far away.

enugu··on Cursor told me I should learn coding instead of asking it to generate it
The Turing test is meant to see if computers can functionally replicate human thought (just focussing on textual conversations aspect for simplicity).

The implications of passing the Turing Test are profound. Tricking a user with a 5 minute conversation isn't the same. The examiner is allowed to be an expert questionner.

For one thing, software engineering would quickly be taken over as a client can just chat with an AI to get the code. That is far from happening, even with all the recent LLM advances.

Similarly for literature, science etc. Currently, you can detect a difference between a competent human & machine in all these fields just by text chat. Another way of saying this is that the Turing Test is AI-complete and is a test for AGI.

enugu··on Bel in Clojure (2022)
From what you write about macros, it hints at something like F-expressions and first class environments, though I am not sure if it actually does have it.

I find the Kernel programming language to be inspiring. It is based on first class environments and tries to go further in the direction of the philosophy of Scheme : "Programming languages should be designed not by piling feature on top of feature, but by removing the weaknesses and restrictions that make additional features appear necessary."

Your post is also in this spirit.

A language like this would allow a much greater dynamism and a reflective capability, but compiling it is a research project. (There was also a minor scuffle here on HN, a decade ago, regarding the non-inclusion of eval in ClojureScript in order to enable better optimization.)

https://web.cs.wpi.edu/~jshutt/kernel.html

https://axisofeval.blogspot.com/2011/09/kernel-underground.h...

---

Edit: Brief explanation of F-expressions (vau expressions in Kernel)

For a function call, the argument expressions are evaluated and the function only receives the values.

For a macro, the argument expressions are accessible to the macro, but the macro cant access the environment, so it cant evaluate these expression, but rewrites code into some other code.

For a vau expression as in Kernel(analog of lambda), the expression receives not just the argument expressions but also the lexical environment. This means it can genuinely return a result and not just code. Unlike macros, they can be evaluated even at runtime. Lambda and classical macros can be easily built on top of vau expressions.

enugu··on Polars Cloud: The Distributed Cloud Architecture to Run Polars Anywhere
Thanks, I am in the process of choosing a dataframe library and just naively assumed that the Rust interface would be statically typed.
enugu··on Polars Cloud: The Distributed Cloud Architecture to Run Polars Anywhere
Polars is also usable as a Rust library. So, one can use that for static typing. Wonder what the downsides are - maybe losing access to the Python data science libraries.
enugu··on Gödel's theorem debunks the most important AI myth – Roger Penrose [video]
The core of the argument is simple diagonalization.

We know that the problem of deciding if a program halts on some input is undecidable via computation.

But suppose we have a partial program P(C,I), to decide halting/not-halting of a program P on an input I, with errors, including input not being of right type, counting as a halt.

For instance, P sees a loop at the start of the program for which there is no exit or it sees that the program is trying to find a prime which ends with 4. It marks these cases as non-halting. Halting cases can always be marked by P as halting by just running the code in one thread and waiting to see if it halts. The third option, non-halting programs which are not seen to be so, in this case P itself runs forever.

Now given P, it is possible to construct P' a program which detects all the non-halting programs that P does, but atleast one more non-halting case also. To do this consider R, with R(C)=not(P(C,C)) which takes an input program C, asks P 'does C halt with input the source of C itself'. Then R reverses P's decision. If P says halt, then R does not halt on C. If P says not-halt, R halts and finally if P is undecided(so P doesn't stop), R again doesn't halt.

What about R(R), R applied to itself. P(R,R) can't be decisive as we will get a contradiction. But in fact R(R) doesn't halt, as all halts are detected by P.

So, P' is basically P with the extra detection - R(R) doesn't halt.

This proves the weaker claim of Penrose

'No knowably correct program P, can simulate human behaviour' as we can see that if P is right, P' is also right.

The 'knowably correct' might seem like an escape hatch, but it is actually hard to split the difference. Simulating humans necessarily means simulating a group of humans who are also doing verification of their initial results with each other and proof-assistants.

Also, going from P to P' is a computable process, but if you try to add this to P, you get a new program with a new limitation.

enugu··on Gödel's theorem debunks the most important AI myth – Roger Penrose [video]
Hand-waving is not required as one is not going into microfoundations at all in the original argument. Someone making a thermodynamic argument for some conclusion, can later on try to speculate about how the conclusion is implemented with statistical mechanics. But the original argument is itself independent. Similarly, Penrose speculates about Orch-OR(which is not dualism btw, Penrose is a physicalist), but the truth of the argument isn't dependent on Orch-OR itself.
enugu··on Gödel's theorem debunks the most important AI myth – Roger Penrose [video]
There is no special insight required here. Merely, a simple diagonalization argument (https://news.ycombinator.com/item?id=43257904) which given a partial process to detect if a Turing machine halts, constructs a new machine which does not halt but was not detected by the earlier process.
enugu··on Gödel's theorem debunks the most important AI myth – Roger Penrose [video]
Godel explicitly constructs the statements (which are essentially encoding a particular self-referential statement into logic). Penrose makes this more concrete where given an accurate, but necessarily partial procedure for detecting when a program halts, one can use diagonalization to show that a program wont halt even though the earlier process couldn't detect it.

https://news.ycombinator.com/item?id=43257904

This construction is something that a computer can do, but not the original system itself. Once you augment the system, there is now a new statement it cant see and so on. (so on, here involves ordinals).

enugu··on Gödel's theorem debunks the most important AI myth – Roger Penrose [video]
> You just can't tell if it is true.

But in the Penrose argument, we can start from a true system and use reflection to arrive at another true statement which is not deducible from the original system.

This is important to the argument as one starts with a proposed program which can perform mathematical reasoning correctly and is not just a random generator. Then, the inability to see the new statement is a genuine limitation.

enugu··on Gödel's theorem debunks the most important AI myth – Roger Penrose [video]
That a knowably correct program cant simulate human reasoning is basically Godel's theorem. One can use a diagonalization argument similar to Godel's proof for programs which try to deciding which Turing machines halt. Given a program P which is partially applicable, but always correct, we can use diagonalization to construct P' a more widely applicable and correct program ie. P' can say that some Turing machine will not halt while P is undecided. P'. So, this doesn't involve any logic or formal systems, but it is more general - Godel's result is a special case as the fact that a Turing machine halts can be encoded as a theorem and provability in a formal system can be encoded as a Turing machine.

Penrose indeed, believes both in the stronger claim - a program can't simulate humans and the weaker claim, a knowably correct program can't simulate humans.

The weaker claim being unassailable firstly shows that most of the usual objections are not valid and secondly, it is hard to split the difference ie. to generate the output of HC using a program which is not knowably correct. A program whose binary is uninterpretably but by magic only generates true theorems. Current AI systems including LLMs don't even come close.

enugu··on Gödel's theorem debunks the most important AI myth – Roger Penrose [video]
Take a group of humans who each make observations and deductions, possibly faulty. Then, they do extensive checking of their conclusions by interacting with humans and computer proof assistants etc. Let us name this process as HC.

A program which can simulate individual humans should also be able to simulate HC - ie. generate proofs which are accepted by HC.

---

Penrose's conclusion in the book is more weak - that a knowably correct process cannot simulate humans.

We now have LLMs which hallucinate etc that are not knowably correct. But, after reasoning based methods, they can try to check their output and arrive at better conclusions, as is happening currently in popular models. This is fine, and is allowed by Penrose's argument. The argument is applied to the 'generate, check, correct' process as a whole.

enugu··on Gödel's theorem debunks the most important AI myth – Roger Penrose [video]
He misrepresents Penrose's argument. I remember Scott Aaronson met Penrose later on, and there was a clarification though they still dont agree.

In any case, here's a response to the questions (some responses are links to other comments in this page).

> Why does the computer have to work within a fixed formal system F?

The hypothesis is that we are starting with some fixed program which is assumed to be able to simulate human reasoning(just like starting with the largest prime assuming that there are finitely many primes in order to show that there are infinitely many primes). Of course, one can augment it to make it more powerful and this augmentation is in fact, how we show that the original system is limited.

Note that even a self-improving AI is itself a fixed process. We apply the reasoning on this program including its improvisation capability.

> Can humans "see" the truth of G(F)?

https://news.ycombinator.com/item?id=43238449

> one only has to note that even if you believe humans can "see" truth, it's undeniable that sometimes humans can also "see" things that are not true

https://news.ycombinator.com/item?id=43238417

enugu··on Gödel's theorem debunks the most important AI myth – Roger Penrose [video]
Penrose's argument does not require humans to prove every true statement. It is of the form - "Take a program P which can do whatever humans do and lets a generate a single statement which P cannot do, but humans can."

The core issue is that P has to seen to be correct. So, the unassailable part of the conclusion is that knowably correct programs can't simulate humans.

enugu··on Gödel's theorem debunks the most important AI myth – Roger Penrose [video]
Penrose does not require transcendent insights, but merely the ability to examine a finite knowably correct system to arrive at a correct statement which is not provable by the system. In fact, the construction of a Godel statement is mechanical, but this does not mean that the original system can see it. It is a bit like given a supposed finite list of all primes, we can construct a new prime by multiplying them and adding 1. This construction is a simple computation.
enugu··on Gödel's theorem debunks the most important AI myth – Roger Penrose [video]
The argument does not need to involve typical human behaviour with its faults. Because if a computer can simulate humans, it can also simulate humans with error correction and verification mechanisms. So, the computer should also be able to simulate a process where a group of humans write down initial deductions and then verify it extensively for logical errors using both computers and other humans.

Most of the objections have been covered in his book "Shadows of the Mind".

Also, the fact that most human behaviour is not about deducing theorems isn't relevant as that is used as a counterexample which attacks the 'computers can simulate humans' hypothesis. This particular behaviour is chosen, as it is easy to make reflective arguments precise.

enugu··on Gödel's theorem debunks the most important AI myth – Roger Penrose [video]
This is not a presupposition for Penrose, but a conclusion. The argument for the conclusion is the subject of several of his books.

Secondly, the issue is not being a genius, but an ability to reflect. What can be shown, uncontroversially, is that a formal computer system which is knowably correct, a human (or indeed a machine apart which is not the original system) can know something(like a mathematical theorem) which is not accesible to the system. This is due to a standard diagonalization argument used in logic and computability.

The important qualifier is 'knowably correct' which doesn't apply to LLMs which are famous for their hallucinations. But, this is not a solid argument for LLMs being able to do everything that humans can do. Because correctness need not refer to immediate outputs, but outputs which are processed through several verification systems.

enugu··on Gödel's theorem debunks the most important AI myth – Roger Penrose [video]
No, Penrose is not dumb. He gives a very good argument in his books on limitations of AI, which is almost always misrepresented including in most of this thread. It is worth reading "Shadows of the Mind".
enugu··on China tells its AI leaders to avoid U.S. travel over security concerns
Most startups are not building foundation models, but using them as an input. So commoditizing their complement can actually lead to more opportunities. Similarly, web hosting which required big investments previously is now commoditized after emergence of AWS and other cloud providers.

Good point regarding movie localization. But, there is still a network effect in entertainment - people want to watch what is popular. So the top content made with lot of money can still have a big market.

enugu··on Show HN: BadSeek – How to backdoor large language models
These prompts are fielding millions of queries. The test questions are a small part of them. Further, the server doesn't know if it got the right answer or not, so it can't even train on them. Whereas in the arrangement with the testing companies before release, they can potentially do so, as the they are given the scores.
enugu··on Show HN: BadSeek – How to backdoor large language models
They dont need them to be released(in the sense that you have a copy of the binary) to evaluate the model. The costly model training is useless unless access is given to people who pay for it.

The models of Open AI, Claude and other major companies - are all available either for free or a small amount(200$ for OpenAI Pro). Anyone who can pay this, can run private tests and compare scores. So, the public does not need to rely on benchmark claims of OpenAI based on its pre-release arrangements with test companies.

enugu··on Show HN: BadSeek – How to backdoor large language models
> Either the eval maintainers need to be given the closed source models (which will likely never happen)

Given that the models are released to public, the test maintainers can just run the private tests after release, either via the prompts or via an api. Cheating won't be easy.

enugu··on US and UK refuse to sign AI safety declaration at summit
The peace that you refer to, involved a strong restriction placed by more powerful states which restricts nuclear weapons to a few states. This didn't involve any principle, but was an assertion of power. A figleaf of eventual disarmament did not materialize.

I do claim that it is obvious that widespread acquisition of nuclear weapons by smaller states would be a severe degradation of security. Among other things, widespread ownership, would also mean that militant groups would acquire it and dictators would use it as a protection leading to an eventual use of the weapons.

Yes, the danger of AI weapons is nowhere at that level of nuclear weapons yet.

But, that is the trend.

https://www.nytimes.com/2023/07/25/opinion/karp-palantir-art...

https://news.ycombinator.com/item?id=42938125

enugu··on US and UK refuse to sign AI safety declaration at summit
The comment was not speaking in favour of restrictionism, (I don't support it) but what strategy the more powerful states will adopt.

Regarding an increase in security with nukes, what you say applies for exceptions against a general non-nuclear background. Without restrictions, every small country could have a weapon, with a danger of escalation behind every conflict, authoritatrians using a nuclear option as a protection against a revolt etc. The likelihood of nuclear war would be much more(even with the current situation, there have been close shaves)

enugu··on US and UK refuse to sign AI safety declaration at summit
AI doesn't look it will be restricted to one country. A breakthrough becomes common place in a matter of years. So that paraphrase of Vance's remarks, if accurate, would mean that he is wrong.

The danger of something like AI+drones (or less imminent, AI+bioengineering) can lead to a severe degradation of security, like after the invention of nuclear weapons. A degradation in security, which requires collective action. Even worse, chaos could be caused by small groups weaponizing the technology against high profile targets.

If anything, the larger nations might be much more forceful about AI regulation than the above summit by demanding an NPT style treaty where only a select club has access to the technology in exchange for other nations having access to the applications of AI from servers hosted by the club.

enugu··on Google drops pledge not to use AI for weapons or surveillance
Drone + AI weapons have horrible applications - remote assassinations to cause political chaos, a tyrant using it to selectively target those unfavourable to his rule without worrying about human checks, bigger nations exploiting smaller ones etc.

Lot of this thread has reduced the issue to whether it is more ethical for one country to deploy relative to others. In any case, a lot of countries will have this capability. A lot of AI models are already openly available. The required vision and reasoning models are being developed currently for other uses. Weaponization is not a distant prospect.

Given that, the tech community should think about how to tackle this collective major problem facing humanity. There was a shift, which happened to nuclear scientists, from when they were developing the bomb to the post World War situation when they started thinking about how to protect the planet from a MAD scenario.

Important questions - What would be good defense against these weapons? Is there a good way of monitoring whether a country is deploying this - so that this can be a basis for disarmament treaties? How do citizens audit government use of such weapons?

← PreviousPage 2 of 9Next →