"Stackoverflow for agents" still not there?
978 karma · joined December 4, 2022
Building LiveReview:
https://github.com/HexmosTech/LiveReview
Risk aware AI code review for business-critical systems
Also another unstated assumption is that both the "author" and "reviewer" are reading AI-generated stuff, at least partly
First of all -- the author should not be surprised by any of the things the AI is reporting. If the author is -- either the reviewer AI went wrong, or the generative component went wrong. Either way -- there is something worth looking at in greater depth.
For the reviewer -- or "traditional reviewer" as we may term them -- they are dealing with much greater volume per day now, again due to AI generation. This is supposed to help them get through things faster while not losing comprehension altogether.
There's a reason we still have the code diff right below -- they are encouraged to check things themselves if they feel "something is off" with the quiz.
The point is -- a few questions hopefully gets the wheels spinning in the reader rather than looking at the diff passively and saying "LGTM"
And yes -- all the review results are grounded strictly in the diff
Higher-level design often involves dissolving apparent tensions or opposites through layering or more sophisticated mechanisms, while preserving the benefits of both.
Example ->
Proposition: "Phew, we have to choose between convenience and security."
Naive design: "Let's find a watered-down compromise."
Good design: "Let's layer security on top of convenience by designing the system so users don't have to choose, or water either one down."
Great companies have a history of dissolving apparent contradictions under a single roof via good design.
I thought the whole point of trying to write out TLA+ is so that you get a better idea of what you want and put it into formal language?
I get that an LLM can assist/help with expressing what we want in formal language a bit, but if one automates all this there is no human intent/design anymore.
If the LLM generates both the design (TLA+) and writes an arbitrary program that satisfies said design -- what exactly have we proved?
What assurance do humans get since human doesn't know or cannot specify what they want.
As to disposable software, it's harder to get traction/adaption when things constantly break or are slow or the experience is crappy in general.
To make it simpler - all else being equal - as a user would you prefer using highly reviewed/vetted/reliable software, or otherwise?
My bet is reliability is an invariant -- nobody wishes for software that crashes, leaks your private info, gives faulty output, is laggy to use and so on.
Consider something like TLA+. How can we make things such as that - be useful in an LLM orchestration framework, be human friendly - that'd be the question I ask.
So the developer will verify just the spec, and let the LLM match against it in a tougher way than it is possible to do now.
Many times, these GenAI tools can delete/modify code mistakenly.
I use LiveReview's git precommit features - so the review happens right before I commit code automatically. And it has saved me many (100s of) times.
Give LiveReview's Precommit checks a try.
Once you look at the present engineering org compositions see what's the error in thinking.
There are other analogy issues in your response which I won't nitpick
Quick check: Do you want to go back to pre-industrial era then - when according to you, you had better options for clothing?
Personally, I wouldn't want that - because I believe as a customer, I am better served now (cost/benefit wise) than then.
As to the point about recursive quality decline - I don't take it seriously, I believe in human ingenuity, and believe humans will overcome these obstacles and over time deliver higher quality results at bigger scale/lower costs/faster time cycles.
What the role of an engineer in the new context - I am not speculating on.
I'd think there'll be a dip in code quality (compared to human) initially due to "AI machinery" due to its immaturity. But over-time on a mass-scale - we are going to see an improvement in the quality of software artifacts.
It is easier to 'discipline' the top 5 AI agents in the planet - rather than try to get a million distributed devs ("artisans") to produce high quality results.
It's like in the clothing or manufacturing industry I think. Artisans were able to produce better individual results than the average industry machinery, at least initially. But overtime - industry machinery could match the average artisan or even beat the average, while decisively beating in scale, speed, energy efficiency and so on.
https://hn.algolia.com/?query=Chomsky%20and%20the%20Two%20Cu...
The oldest submission is from 15 y.o ago - that is 2010.
I resubmitted it - thinking - with the success of LLMs - felt this was worth a revisit from "how real-world scientific progress works" point of view.
Pearl defines a ladder of causation:
1. Seeing (association) 2. Doing (intervention) 3. Imagining (counterfactuals)
In his view - most ML algos are at level 1 - they look at data and draw associations, and "agents" have started some steps in level 2 - doing.
The smartest of humans operate mostly in level (3) of abstractions - where they see things, gain experience, and later build up a "strong causal model" of the world and become capable of answering "what if" questions.
The species as a whole will evolve inevitably; the individual animal may not.
Of the food we take - cells accept a % of it as nutrients and such, rest is discarded as waste. The cells know how to get this job done - it's a very complex process for sure.
I think it's the same with information content - a % actually is useful for making life happen - whereas the rest should ideally be discarded because it is meaningless from a life perspective. The mind just knows what's important most of the time.
In this case - willful ignorance would be something like intermittent fasting or regulating food intake carefully, since it is a conscious process.
The former process is unconscious and operates at the "cell level" whereas the latter is a conscious process that operates at the "whole-being" level.
2. Don't confuse "enjoyment" with "number of readers". The previous generation may have enjoyed it more - because there were no better options.
3. People over the globe are more educated now, and engaged in knowledge work. They must read to get work done.
4. Don't forget the "pirate book" scene - such as lib gen, Anna's archive, etc. - in developing countries.
On the topic - here were my original points (with some extensions):
1. I want a smalltalk-like environment but with modern languages (webassembly makes this technically possible)
2. Alan Kay himself agrees smalltalk is no longer relevant in a concrete manner anymore - it's too old and outdated. The library support is absymal, and LLMs etc won't be as helpful as modern langs, since the training data available is less in quantity and quality. And I am in line with Dr. Kay's view - Smalltalk is indeed too old. I feel the same way about using Lisp for my particular goals.
3. I am not complaining in any way - just stating my requirements in explicit terms. Also I dont consider myself a "user". I am a system builder. I am fully capable of doing things myself if there's no alternative available.
Given how big and important a platform the modern web is - I really am starting to think - something like this should exist - where I have a neat little IDE bundled right within the browser using which I can evolve the browser...