1,427 karma · joined March 10, 2024
They’ve always been this way.
It’s to easy to write an assertion that looks like it checks what you want it to, and gives you a false confidence when it passes, only to figure out after a field failure that it wasn’t checking what you thought at all.
But I’d pay for a commercial set of properties and assertions that had guarantees, and not worry about their complexity. For something standard like cache protocols and bus protocols.
Other cases it needs to be pretty straightforward to grasp by a peer reviewer in minutes, or it’s not worth it.
But when you can hit the sweet spot, a set of easily understood properties (or complex but trusted properties), formal can save massive amounts of time.
I didn’t read the whole article, just the appendix. There they talked about having Claude write a python interpreter. Then they had it write a set of properties to test and they vetted the properties against accepted existing python interpreters.
Proving two things are equivalent is a major component of formal.
For everything else, formal is only as good as the properties you specify. A long standing problem. But it’s a problem for all verification. If you write a shitty tests, you get shitty verification.
A key difference I think is that the same skills used for evaluating and reviewing application code apply to reviewing test code. That’s not really true for formal.
> The most striking feature of agent proofs is that they are verbose, repeatedly spelling out details about the program’s execution: specifying the contents of CPU registers, explicitly writing out integer conversions, explicitly spelling out that many different addresses are all distinct from one another, writing out the exact contents of stack addresses, explicitly considering every possible branch in a function, etc.
> To be sure, these details are important to precisely reason about the execution of code. However, human proof engineers would have immediately introduced abstractions to factor out these types of repetitive specifications.
- - -
This matches my experience when I look at the tests LLMs write to check their own work when I’ve given them a task. The tests are almost universally terrible because they hard code everything in the most brittle way possible. They are nearly useless for regression tests because if I change anything about the source, even in non material ways, the tests all break in stupid ways that aren’t checking any important properties.
If I have the LLM make the then it also has to rewrite all the tests in a new episode of brittle concrete hyper-literal tests. The concept of a regression suite is non-existent.
It’s probably just as well. Having the implementer write the verification is generally a bad idea with people, maybe the problem of carrying over assumptions and blind spots and misconceptions applies to LLMs as well.
One might say it’s operator error, and I’m underspecifying things, but if I have to go to such elaborate lengths to get it to do what I need, I’ll just implement it myself.
The only interpretation that matters is that of the Authority of enforcement and judgement. The implementer’s interpretation doesn’t hold much weight.
We can say what we earnestly believe the regulations mean, and how they can be met, but if the authorities say “not good enough” that’s all that matters.
E-paper is an informal category that anyone can use for any display type.
The display in this video/article is also electrophoretic, but not all displays called e-paper are and it’s aggravating. An example: https://kentdisplays.com/products/color-epaper-displays/
It wouldn’t take much to just sound like jumbled noise.
Hearing aids have come a long way but all of the ones I’ve used still basically suck when it comes to understanding speech. Part of it, I think, is that my loss isn’t bad enough that fully closed ear pieces are recommended and as such background noise is still free to obfuscate conversation.
It basically makes conversation in anything but a quiet room extremely difficult. My brain works overtime trying to fit possible words to what I hear. Sometimes I fail with pretty humiliating results.
It doesn’t take much before you just stop trying to talk with people. Especially with the younger crowd because they tend to speak the quietest. It varies by individual, but at least older people who’ve had some loss themselves will speak louder to compensate.
The worst is restaurants. Usually I can guess what’s being said by the server, but not always. Accents increase difficulty.
I used to be pretty out going, joked a lot with people. Now I avoid most conversations when I can.
The most common type that is used would probably be logical equivalence checking. Proving RTL and a netlist are equivalent is useful for catching synthesis bugs.
Or proving two netlists are equivalent after inserting test functions directly into a netlist, or some other netlist edit.
Property checking is what I use the most. You can check these during simulation which I wouldn’t call “formal” but you can also prove them using tools that use SAT solvers and whatnot to prove things mathematically.
As always, the tricky part of verification is writing the correct test or model. With formal we can use SystemVerilog assertions to write properties and sequences, but the difficulty in getting them right goes from trivial -> inscrutable very quickly.
It’s extreme easy to write assertions that pass and never realize your assertion was not doing what you thought and you weren’t proving what you meant to.
I haven’t used some of the more advanced tools so maybe they have ways to make this easier. But because of this I tend to just write assertions that are pretty easy to understand at a glance, and therefore closer to the trivial side of things.
If a peer has to solve a sudoku puzzle in their head to understand your work, then it’s unlikely the peer review will be worth anything. So I do what I can to make my work understandable at a glance (from a competent peer in the industry).
Of course making something simple can be quite challenging and often takes more time than leaving something complex and opaque.
I’ve never been involved in the foundry side of the work, and for ASICs, that is often half of the schedule.
But it doesn’t matter. Those historical parties have nothing in common with their namesake parties today.
I don’t remember if my computer at the time had built-in Ethernet or if I had to buy a card.
I have been always-online ever since. Wouldn’t have it any other way.
But I do still miss the BBS scene.
It’s not a step I’ve seen discussed very much, so it was kind of neat to see it discussed front and center in a lay article like this.