Hacker News Reader

Comment by raverbashing | Hacker News Reader

I've worked with embedded device makers that had better ideas about building redundancy and failsafe procedures into their products

While software mistakes do happen, it seems any moron these days thinks they can put out an IoT product because they know how to flash a led on Arduino and they make no effort whatsoever in thinking how to make their devices have minimum reliability

Have a user accessible USB port (from the inside) from which a signed payload can be loaded for cases like these. Would have saved several days for their customers and several shipping fees.

View on HN

You could also do A/B partition booting. Whatever partition is currently running, the other one gets updated. When the update is complete, the system reboots to the updated partition. If it doesn't survive the reboot (like if the system never reaches the app and the hardware watchdog timer stops being reset), then it reboots again, back to the first partition.

This is the common (and correct) solution.

In this scenario, the biggest problem is updating the bootloader, which cannot do this type of fallback.

Generally people avoid updating the bootloader (for good reason) but it is necessary sometimes...

Right. It's important that you leave as little logic in your bootloader as possible.

Have two bootloaders, one is trivially provable and can always be relied on in case the entire world falls apart, and can never be updated. The second one is updatable if needs be.

Of course, the good thing about the bootloader is it is less likely to need updates in the first place, and it's also likely easier to verify an update with a high level of confidence, as its functionality is so tightly bounded.

> (like if the system never reaches the app and the hardware watchdog timer stops being reset)

Worst case scenario is if the system does come up enough for the hardware watchdog to get kicked in a timely manner, but nothing else works.

The watchdog needs to be the last initialized bit of code on the system.

Even so, determining when a partition is "bad" is hard.

Of course. The hardware watchdog is just one of many things that should be taken into account. I picked it as an example, specifically because a lot of people don't know such a thing exists on many embedded systems.

In hindsight I'm sure they will do something different. For folks who are maintaining systems with OTA updates that can't survive an OTA failure, hopefully they now realize they are in the middle of a train wreck and should get to work.

An 8 MB (64 megabits) SPI flash chip[1] is about $2. Two jumpers to power the SPI flash and set the boot/load option is another 3 - 4 cents. These take up less than 30 sq mm of board space. And they give you a tremendous back up in case of failure.

[1] http://www.mouser.com/Semiconductors/Memory/Flash-Memory/_/N...

If you're using external flash rather than on-die flash, doesn't that mean you'll have to copy it into RAM before running it instead of running it directly, and therefore more RAM will also be part of the extra cost?

That depends of course.

All of the ARM chips I'm playing with these days have a serial boot loader in ROM that you can get to by setting a pin on the CPU and powering it up that way. In such a scenario it would take no extra parts.

I believe they emulate or copy the system used by FPGAs to read a configuration from a dedicated SPI chip. Its well understood and a lot of tools can program the SPI chip to have the right sequence of bits in it to do the right thing.

If I were designing a system with OTA updates, then I would design something like this with it for this very reason.

USB would add to the BoM. Much easier (methinks) to have the lock revert to the last known good configuration if you operate an I/O on the inside in an unusual way - say, turning the lock thingamajig towards the 'open' direction for >20s or so.

Also, no update should be carried out unless the user initiated it - for instance, a flashing led or an app notification tells the user an update is available - and it needs to be initiated by using the same controls as would be used for a roll-back, ensuring you do not end up locked out after updating.

That being said, the idea of a lock being online scares the bejeezus out of me. Call me a luddite, but as a minimum I want my burglars or vandals to show up, rather than nessing with me from wherever.

> Call me a luddite, but as a minimum I want my burglars or vandals to show up, rather than nessing with me from wherever.

Well they can't be burglars if they never show up :p

They just tell your roomba to rob you.

https://youtu.be/OwtxWL0P9wA?t=1m54s

Well, good points I suppose. On the other hand if you have a USB port on the thing, what's to keep someone from flashing an update that backdoors the lock? Wait a few weeks, rob the place, rinse, repeat.

Now that I think about it, I hope these devices are validating the certificate of publisher and the updates are signed.

> I hope these devices are validating the certificate of publisher and the updates are signed.

You're joking right? Really, the vast majority of this iot stuff is ridiculously bad in this respect. Plain text data transfer, unsigned updates from any source based on a text message and so on. It's a disaster waiting to happen.

A disaster that has already and continues to happen, you mean.

No, I'm thinking along the lines of large vehicle manufacturer doing an over the air update of the firmware of their vehicles bricking all of them, including the ones on the road. That kind of disaster.

> what's to keep someone from flashing an update that backdoors the lock?

> from which a signed payload can be loaded

All updates should be digitally signed by the lock manufacturer

From a security perspective, absolutely they should.

But then we complain because we don't "own" the hardware and can't hack around on it.

These things really shouldn't be internet connected, and the firmware should be controlled by the owner.

Reply on news.ycombinator.com