Richard Stallman is rolling in his grave with this.
But in all seriousness, nice work, I think this _is_ where the industry is going, hopefully we don't have to rely on using proprietary models forever though.
1,149 karma · joined August 10, 2013
Richard Stallman is rolling in his grave with this.
But in all seriousness, nice work, I think this _is_ where the industry is going, hopefully we don't have to rely on using proprietary models forever though.
- there's a devlog showing all the prompts and accepted outputs: https://github.com/sutt/agro/blob/master/docs/dev-summary-v1...
- and you can look at the ai-generated tests (as is being discussed above) and see they aren't very well thought out for the behavior, but are syntactically impressive: https://github.com/sutt/agro/tree/master/tests
- check out the case-studies in the docs if you're interested in more ideas.
I've written a little about some my findings and workflow in detail here: https://github.com/sutt/agro/blob/master/docs/case-studies/a...
> more/better in the same time used in the past — which is what I do), when left alone with nontrivial goals they tend to produce fragile code bases that are larger than needed, complex, full of local minima choices, suboptimal in many ways.
And this part felt like a "bitter lesson" anti-pattern:
> Avoid any RAG that will show only part of the code / context to the LLM. This destroys LLMs performance. You must be in control of what the LLM can see when providing a reply.
Ultimately I think cli agents like claude-code and gemini-cli and aider will be controlling the context dynamically, and the human should not be spending premature optimization time on this activity.
If anyone's interested I've got some very exact stats on prompts and accepted solution linked in my LLM proof of concept repo: https://github.com/sutt/agro/blob/master/docs/dev-summary-v1...
_Knowing how way leads to way_, the larger the task, the more chance there is for an early deviation to doom the viability of the solution in total. Thus for even the SOTA right now, agents that can work in parallel to generate several different solutions can reduce your time of manually refactoring the generation. I wrote a little about that process here: https://github.com/sutt/agro/blob/master/docs/case-studies/a...
I wrote a little about one such task, getting agents to supplement my markdown dev-log here: https://github.com/sutt/agro/blob/master/docs/case-studies/a...
I'm reminded of teaching bootcamp software engineering, when every day #1 we go through simple git workflows and it seems very intimidating to students and they don't understand the value. Which fair enough because git has a steep learning curve and you need to use it practically to start picking it up.
I think this might be analogous to the shift going on with ai-generated and agent-generated coding, where you're introducing an unfamiliar tool with a steep learning curve, and many people haven't seen the why? for its value.
Anyways, I'm 150 commits into a vibe coding project that still standing strong, if you're curious as to how this can work, you can see all the prompts and the solutions in this handy markdown I've created: https://github.com/sutt/agro/blob/master/docs/dev-summary-v1...
How well do Transformers perform given learned embeddings but only randomly initialized decoder weights? Do they produce word soup of related concepts? Anything syntactically coherent?
Once a well trained high dimensional representation of tokens are established. can they learn KQ/MLP weights significantly faster?
- Success: ChatGPT-4-November, Phind-November
- Failures (via hallucination): GPT-3.5, Mistral-7b, Llama2-13b
Interestingly, ChatGPT-4 wrote a short python script (unprompted) to develop a pseudo-CoT type generation and answered correctly. Phind, which also answered correctly, is also a code-based/code-enabled LLM.
Non-code-based-LLM's appear to reason cogently when describing their games - what moves are illegal, what the best guess in certain conditions are, etc - but their actual answer, even in multiple choice is comically wrong.
These are very initial results but my tentative hypothesis is symbolic logical reasoning - string manipulation / function calling type tasks - are greatly enhanced by having a code modality. How to switch between natural language and code-generation may offer a strong boost on tasks like these.
The other board member, Helen Toner list for her twitter profile: "Interests: China+ML, natsec+tech..." and works for another Defense think tank.
If there's one way the CEO of fastest growing company in the world could get fired, it's to essentially get his metaphoric security clearance pulled like Oppenheimer did.
What ideological belief does Musk hold that would create friction with his employees?
People are misinterpreting this as the research goal. No, it's an implicit negation of an ethos - AI Ethics, Trust & Safety, ESG, Politically Correct - that have dominated decision making in product and hiring in tech for the past decade.
Whether or not you agree with him, it's hard to deny that Musk is the only one offering researchers an escape from the bowing to the orthodoxy while also offering an A+ pedigree institution.
$ code --no-extensions --no-lang-server
And then alias that command to `litecode` or something
As it stands, it's very difficult to invest the budget for a dev studio (dozens of high skill people) to build a "VR movie" when the format is so unknown and unpopular. But with generative AI, an indie dev could create their own professionally produced virtual world movie. It's these creatives and risk takers that will find what types of things VR needs to become more popular.
Which means it's going to be harder to evaluate junior candidates code without actually running and testing that code because they'll have built a huge library that looks really well formatted but has logic gaps which are difficult to catch on a glance. As it stands currently, usually some style and organization tells you this person gets it.
It's hard not to see the massive influence of Scott Alexander in the way Vitalik presents his ideas and structures his essay. One SSC meme that kept coming up in my mind as I read the essay is that "ideas are soldiers" [1] is certainly in play with the blockchain wars. It's refreshing to see we have leaders in this space that can recognize the bravery of the enemy's soldiers. At least for today!
For Offensive Linemen in the NFL, there is no correlation between weight (which range from 300-360 pounds) and overall performance. A "heavy" 350 pound player is not more likely to do better than a "light" 310 player. But nobody who weighs a mere 250 pounds could realistically make the cut or perform well at the highest level.
For basketball players there is no correlation between height and performance, and there are several standouts examples of players below six feet so there's no cutoff. But if you compare the distribution of the subpopulation versus the general population, you'll see an extremely strong height bias.
The biggest difference I noticed in how I would take a test versus other people I tried to coach is that I viewed the test as a fun game like a challenging video game level. And those who struggled on the test viewed it as dreadful judgement being rendered on them.
It's like when you can tell someone is extremely self-conscious while dancing: Beyond teaching them any actual dance moves, you have to turn off the part of their brain which is blocking their natural mental resources for problem solving, and that's often the fear they are inadequate to the task, will disappoint their supporters, and that it will hurt their future prospects.
Since the AirTag emits the message, that message would either contain: - a static signature, which could then be copied and mimic'd by imposters (replay attack) - store the private key on the AirTag device, which could then sign the a continuously changing nonce like the current datetime. But this means the private key could then be extracted from one device, and used to sign messages on an imposter device. So unless every device had a separate private key, this method would immediately be compromised as well.
So why doesn't Apple have a unique private key for each device? Well it appears it actually does, and has them constantly changing their private keys. But there appears to be some kind purposely implemented anonymity features that is designed to prevent Apple's servers from associating a ping with ever having to decode the contents, and thus of associating your account/device with the emitted location.
If you build in validation to write to the network ping database that goes "here's a ping with a signature ABC and let's lookup if it's valid, oh it is, that must be from AirTag Bob bought last month with private key XYZ, let's declare this ping valid" then Apple is only a logfile-write-of-this-information away from being able to perfectly stalk everyone who has purchased a device. So instead, the tradeoff they made is they don't keep track / purposely blind themselves to their device-in-circulation keys to truthfully say they actually can't track you. That leaves open the ability of imposter devices to transmit information through the network by creating their own known keys which look indistinguishable from authentic device pings.
I think manufacturers are hoping that certain aspects of the modern auto become "can't live withouts", e.g. bluetooth, which helps them smuggle all the other more marginally useful, security and ux intrusive features into your model. Now as others have mentioned, these unwanted smart features might actually be required by law/regulation/tax-credit/legal-dept, so it's not bad programming/design but "design by committee" that's dooming our chances of a good dumb car.
I wonder how well my car will age when miscellaneous sensor all over the car start failing. It used to just show a light on your dashboard, now I wonder if I'll be grounded - stuck in park - or hounded by incessant bells repeating the same spurious warning over and over again.
Yeah OK, Strongtowns, let's scrutinize the road budgets (~5% of the property taxes). Who wants to live in these hypothetical high density developments when you can't even be safe?
On January 1, 2028 how much M2 money supply is there? 10% more? 10% less? 100% more? 1,000% more? It completely depends on who is influencing it.
In these cases, ideally a hard fork would be able to maneuver the network out of the way of the technical vulnerability in time.
Less ideally, there's just the fact that crypto protocols are now so diversified in terms of hashing algos, PoW algos, and alternative consensus mechanisms (e.g PoS, zk's, etc), active chains, and governance models, that successfully knocking out the majority of them, nevermind all of them, would be almost impossible. But this is obviously less than ideal in that now you have an even more complex situation: where you are constantly hedging your savings in multiple tokens, and how do you value one versus the other? Simply in its blue-horizon cryptographic development threat model? popularity? ease of use?
>the crypto ecosystem is unable to use force.
I'd argue the crypto ecosystem has been building "soft power" with web3. To the extant one would argue "it's a scam" is directly proportional to the success of its "propaganda" agencies.
In bitcoin, the internal system mechanics are highly predictable. But there's no external "Open Market Committee" (e.g FOMC) that will change the market price to hit the targets mandated by its stakeholders.
In dollars, the internal system is becoming more unpredictable, while there are numerous external systems (e.g. bailouts, QEx, etc) which are very predictable: they will act to make sure the economic tradeoffs shield the voters of the system from experiencing pain to the next election cycle.
Neither seems ideal in its present form.