634 karma · joined June 8, 2015
The key insight, which may not be emphasized enough in the article, is that the vessel can only rise to above 100C once all the water has changed phase (boiled).
I think this is the same principle explaining why beach popsicle vendors can carry many items on a hot summer day without them all melting right away. There is insulation, for sure, but in addition the temperature of what is effectively a large volume ice cube block must rise above 0C before the popsicles can begin to change phase (melt).
In the rice cooker, this property is harnessed while a "bimetallic switch measured the temperature in the external pot". The bimetallic component means that one metal heats and expands faster than the other, eventually breaking the circuit.
If memory serves, this same trick is used in older car model turn signal lights, to produce the periodic on/off switching.
I am not asian but enjoy my rice cooker every day. I love simple robust engineering.
I did not intend to conflate separate issues. I agree assisted coding and assisted bug hunting can coexist. I was merely trying to weigh the net effect LLMs have on security.
It may be informative to see all the ways in which the LLM got it wrong along the way, before arriving at the cherrypicked example. For instance, was this the only pattern flagged as a potential bug in the process?
That said, I am surprised at how far pattern matching can get you. Certain exploitation requires arithmetic. If LLMs struggle with simple arithmetic, then it is unclear why we would expect them to do well with these. In the GPZ example, it does not appear that the assistant can do the arithmetic or realize the negative offset and its implication. Rather, it looks like pattern matching that mimics static dataflow analysis in order to arrive at the assertion failure.
"To cause the assertion failure, we need a constraint on a column with index greater than 3 or smaller than 1."
Out of curiosity, what's wrong with Intel? Are you referring to their selling more capable parts for more money? If so, that does not strike me as a shady practice to maximize profits. More like how the best fruit goes for export, where it can fetch the most return.
There was a time when people thought if we could put the secure code in a lower ring, then with it we could protect the rest of the system. With virtualization, the hypervisor is in ring -1, which is technically not a ring, but rather a mode called VMX root operation, post-VMXON. This enables things like the blue pill attack, where the hypervisor is itself presented with a false image of the underlying physical hardware, by a malicious layer. You can find the same pattern in ARM TrustZone, where the secure code is repeatedly broken.
"if only we had ring -N"
The fanout is a robustness measure on systems. If we can control the fanout we increase reliability. If all it takes is a handful of bits in a 3rd party update to kill IT infrastructure, we are doing it wrong.