So, if a company is small and unable to afford any security, they can just set the number at $0 which means it is the customer's problem. However, if the company is big and people have security expectations, they would need to specify a reasonable number that would alleviate those concerns. As long as the number is clearly communicated and you are not allowed to fraudulently or misleadingly advertise a different number, then customer's would be able to make an informed decision with respect to the security level and liability they are accepting from their vendors.
Obviously there are some complexities with respect to properly communicating this information. For instance, for a consumer-facing company you would probably want a per-consumer number instead of an aggregate number. As an example, if Apple were to claim $300M for the iPhone, that might seem like a large number to an average person, but that would only amount to ~$1.50 per iPhone sold in a year. You also need to prevent misleading advertising that might attempt to divert from the quantitative liability they are actually accepting. However, the base scheme of mandatory labelling requirements along with opt-in liability should allow for a solution that is hard to bypass while being flexible enough to support both small and large businesses.
And if a company is large and low margin, they can do this too so nothing changes. A few companies will be out there promising things, like today, and maybe they can deliver and maybe you can sue them if they don't, like today.
Being able to actually get your money back... is another problem. But obviously a toothless regulation is toothless, and changes nothing, so its not a useful assumption to make
This is would be different than the current state of affairs because lying or overstating your security story would have direct, pre-committed, quantitative downside. This is contrast to the outcomes of a lawsuit which are expensive, highly variable, generally require proof of intent, and depend on demonstrating actual quantitative damages.
The decision to take on risk for the purpose of marketing is not at all palatable to decision makers. Why would you want to accept an increase in risk for an unknown-unknown?
Meanwhile, quantitative, money-linked claims about software security actually do happen in rare edge cases, already.
I see no reason why the same can't be applied to software. It's probably true that we can't feasibly create software and computer systems that are completely free from security vulnerabilities, but we absolutely have discovered some vulnerabilities which can be protected against using certain software development techniques, and I see no reason why software vendors ought not be liable for damages caused by their failure to follow these established techniques.
Just like with cars, there is always going to be some tradeoffs between cost and security/safety, and exactly where to draw the line is something reasonable people can debate.
> You don't say your car made a mistake when it refuses to start one cold morning. There's essentially universal agreement on the 'semantics' of cars and bridges.
Since you mentioned it...I don't know whether this is the case, but I would guess that there might be regulations about the temperatures at which car batteries and other components must be capable of operating at. And, as car automation software becomes more advanced and prominent, I think we'll be discovering that there is not in fact universal agreement on the "semantics" of cars. There is probably already considerable overlap in the subjects of my analogy, for recent self-driving capabilities but also for much older standard things like airbags and anti-lock braking systems.
But my broader point was that this analogy is basically what 'can god make a rock so big they can't pick it up' is to a discussion of theology.
The situation you are describing is that automobile safety is implemented primarily by establishing standards and only secondarily by the existence of liability (though it's important).
Now, if you had a large company that bought cars and modified them, that company would itself have an obligation to do it safely too. And having regulations about that would be an absolutely necessary part of liability and safety.
...we absolutely have discovered some vulnerabilities which can be protected against using certain software development techniques, and I see no reason why software vendors ought not be liable for damages caused by their failure to follow these established techniques.
I would note that the existence of zero-days is absolutely not the most pressing problem of software security. Having a reasonable system of connecting X to Y is far more pressing. You can get in as much trouble with a misconfigured system as with a vulnerable system. But broadly, your point that there should be standards, is good.
Indeed, regulatory standards is what I was hinting at, not just civil liability (although I believe both exist for car manufacturers, and should probably exist for software vendors).
Things like institutes that study this situation outside the government itself may be necessary to establish trust at this point.
And you can't make the standards primarily about software vendors.
Do you literally mean "nobody" here? As in if I wanted to hire you to put a team together to reliably ship secure commercial software, you couldn't do it either?
I think we have, as an industry, tacitly agreed that it's better to ship cheap, insecure software than it is to ship expensive, secure software. A lot of the innovation in software has occurred specifically because it is so inexpensive.
It seems clear to me that, so far, this tradeoff has been a net benefit, but at some point we may want to trade the inexpensiveness and innovation for some security, particularly has we start to put software into more things that can kill and maim us.
It's fundamentally impossible given that you won't control everything about the operating environment of the software, and can't see into the future.
I mean, even if you could create a perfect piece of software with no bugs present at the time, that's still not enough, because vulnerabilities in dependent components will arise which were unknown at the time.
For example, if you'd written the perfect web browser a few years back, how would you prevent vulnerability to CPU-level attacks such as Spectre and Meltdown discovered later on?
Or consider if you write the perfect wireless networking firmware, but then end up being subject to vulnerabilities in the protocol standards (WEP was a good example of this)?
This is on top of software development being such a complex process that even the best of its practitioners write code strewn with bugs.
Why not? Given the cost of writing secure software, for all but the highest volumes, you can ship it as a dedicated app on its own hardware.
> For example, if you'd written the perfect web browser a few years back, how would you prevent vulnerability to CPU-level attacks such as Spectre and Meltdown discovered later on?
Given that we are talking about liability here. Presumably Spectre and Meltdown would be primarily liabilities for hardware vendors. Software shipped after the vendors recommend software mitigations would shift that, but I don't think
> Or consider if you write the perfect wireless networking firmware, but then end up being subject to vulnerabilities in the protocol standards (WEP was a good example of this)?
First we must put aside the fact that WEP raised red-flags for cryptographers far before an exploit was found, and the fact that those concerns were dismissed with a hand wavy "People can tap into Ethernet lines too, this is just 'Wired Equivalent Privacy'"
Now, I think that if you write software that faithfully implements a protocol, advertise it as implementing that protocol, and it is exploited not in-spite-of but because it did so, what exactly are you liable for?
Even knowing that WEP, WPA, WPA2 and WPA3 are all flawed protocols, I'm sure there will be a WPA4 some day that is also flawed. If the only thing we had to worry about in WiFi stacks were protocol flaws, attackers would have a much harder time of things. No flaw in the WEP protocol allowed gaining execution access on the WiFi adapter, much less the host computer (though given the bus designs of the time, the former inevitably led to the latter).
But rewind to last year, when I was still at Latacora. Ask me the question: can I hire you to make my development process more secure? Sure, I think I can add value there. Ask me instead: can I hire you to make my software liability-proof? Not a chance.
Take 9 strong software security people and divide them into 3 teams. Give each team a stretch of time to find vulnerabilities. There will be a ton of overlap in the findings, but each team will find different things. One team will happen to have a specialty in the low-level details of the platform it's running on; another will spend 80% of their cycles thinking about SSRF and HTTP desync vulnerabilities; still another will happen to have a reliable process for exhaustively testing authorization. Don't get me started on K8s or Electron or cryptography or iOS or browser extensions or Node.js.
And that's all for pretty standard app stack stuff. None of those problems are the subtext for Perlroth's book; the vulnerabilities she's trying to write about are in low-level C/C++ code. The biggest security teams in the world have build enormous, carefully-instrumented fuzzing farms for this stuff and they don't find everything.
I literally mean nobody.
I think you could stand a chance if you designed your product to fit the limitations of what we can deliver reliably. You'd lose most of your features. You'd ship boring CRUD stuff exclusively on your own dedicated hardware, and you'd build it with minimal library support. It would be excruciating, but people do it. Just not for any of the kinds of software we normally interact with.
> I think you could stand a chance if you designed your product to fit the limitations of what we can deliver reliably. You'd lose most of your features. You'd ship boring CRUD stuff exclusively on your own dedicated hardware, and you'd build it with minimal library support. It would be excruciating, but people do it. Just not for any of the kinds of software we normally interact with.
I think we are in agreement then. If you are targeting K8S or Electron, you've already lost the battle against many threat-models that matter for liability. I think what you are pushing back against is people imagine bug-free software and they think "oh maybe 4x more expensive and it will look like it's from 2005" while the reality is more like table-stakes of 10x more expensive and it will look like it's from 1980.
I personally think that the tradeoffs are worth it for more industries than currently make it, but I don't think the law can be nuanced enough to both be effective and not cause massive collateral damage to many other industries. Look at the chilling effect regulation has had on software in the medical industry, and yet those regulations seem to have done little to increase the resources required for an attacker who wishes to exfiltrate patient data.
Nate Lawson used to tell me (I assume he'd still tell me) that companies trying to ship secure software need to budget 10x for verification what they do for implementation. The best companies in the world don't even budget 1x. I'm not sure it's reasonable to ask them to. I think we'll know stuff in 10 years that will change how we look at things.
In my opinion, if Jeff Bezos, Satya Nadella, Sundar Pichai, and Tim Cook all got together and wanted to put together a team that ship secure commercial software, they would probably be unable to put together a team that knew how to get to the $10M level, and there is zero chance at the $1B level. If they all worked together and put their full focus and resources on the problem, they might be able to put together a team that could invent how to get to the $10M level, but they would not be able to put together a team that already knew how and had experience doing so. The key difference between these two cases being that in the former the team would need to invent good practices and experiment to see if they worked, where as the latter only needs to apply already existing known good methodology.
If you want some 3rd party evidence for my claims, not that I am claiming this is exhaustive proof, bug bounties that get claimed are, generally speaking, less than the cost of discovery for a bug otherwise security researchers would not put in the effort to discover than claim them. Not a single one of their premier products that they ship and sell [1][2][3] offers a bug bounty over $1M for total and complete remote zero-interaction compromise. It is very unlikely that they can do an order of magnitude better than the premier products just because they want to.
[1] https://www.microsoft.com/en-us/msrc/bounty
[2] https://www.google.com/about/appsecurity/android-rewards/
That seems unlikely to happen as long as it is in commercial software businesses' best financial interests that it not happen.
I don't think it makes much sense for a random one-off script written by some lone developer starting a company to be subjected to all the alphabet soup of regulation, but I don't think letting everyone get away with security breaches forever is a good idea nor is just throwing up our hands and going "oh well, we're going to keep getting break-ins" for another 50 years.
Doesn't sound so bad of a tradeoff, enforced for certain classes of software. Like one expects liability for medical, banking, flight control, etc. software, it could be extended to OSes, server programs, and anything handling user accounts...
Software with less impact if it's not secure, like e.g. a media player or a text editor, could continue to ship free from liability.
I think you would agree that having critical string parsing logic written in C and shipped in an opaque binary is both negligent and more dangerous than a reasonable person would expect. Traditional products liability claims focus on those kinds of questions: was there negligent conduct? is the product more dangerous than a reasonable person would expect?
Attaching some kind of liability is a good way to encourage the industry to settle on standards of non-negligent conduct. If you provide vendors an opt-out mechanism (e.g. disclose your source and there's no liability), it will not dramatically interfere with the cadence of development.
Common law is very well suited to establishing standards in a dynamic environment. It got us through the first industrial revolution.
Secure software lies on a spectrum. We know memory unsafe languages are dangerous yet critical software is still being written in them. Liability would drastically change the domain in terms of discarding practices we've known for decades are deadends, and raise the bar significantly.
It's not about creating 'perfect software', but about holding developers accountable. If a customer accepts a contract with a faulty spec, that's on them, not the developer.
>But even if it is you still have to worry about every layer below you in the software stack on top of the hardware. You might do everything perfectly at great cost and it all gets rendered moot by spectre/meltdown/rowhammer etc.
So what? It's not your fault. There's only one party that should be held liable for spectre, Intel.
Well, if you have formally verified your system, you know that if the underlying systems work correctly, your work will correctly adhere to the spec. So if a breach occurs, there's two options - either the spec was bad or there's a hardware/OS/library bug.
Otherwise, you can't always know it, that doesn't strike me as a realistic goal in the first place. It's not about always being able to perfectly assign liability - obviously if you can't figure out how something was compromised, you simply can't assign any liability. But one could investigate and if you find deviations from the spec, you know where to correctly place blame.
>You'll look around and definitely find bugs in the software though.
If these are deviations from the spec, I think it would only be right to hold the developer liable. If a developer doesn't want this to occur, they could instead formally verify their software.
If the bugs are in the spec instead, the blame should lie with the party that accepted the spec.