Hacker News Reader

Comment by LeoPanthera | Hacker News Reader

I've posted about this a lot but I feel the need to evangelize it:

The cleverest thing the Raspberry Pi can do that almost no-one uses is network boot.

You don't need an SD card at all, you can setup your server such that you can change what OS a Pi boots into just by renaming a symlink.

I have a bunch of Pis around my house that all network boot into specific tasks and it is a joy.

View on HN

And it causes me endless frustration. Because after a power outage the pi will boot and look for somewhere to boot mere seconds after power is back.

But my router supplying DHCP and with info about where it should look takes guessing 3 minutes. And the NAS containing the actual files that the Pi should load is in a virtual machine that takes, guessing, 8 minutes for the hypervisor and VM to start.

So by then the Pi will have timed out and sit there, doing nothing for all eternity.

Fix? An extremely elaborate scheme of Ikea trådfri outlet controlled by the same server taking 8 minutes to boot to power-cycle the pi after it gets up (and notices that the pi doesn't answer to pings).

Has this been fixed yet? Is it even considered a bug?

Not sure about the rationale of giving up booting... Especially when this ought to be an extremely common scenario.

Could you get a small UPS for the router? I would imagine it's not sucking down a lot of watts, should take care of momentary blips and anything less than an hour fairly easily.

Or you know, just insert a $5 SD card like a normal person?

If you have a bunch of them, you cannot centrally manage them this way without a lot more work. Also, if they log to disk, it will wear down the SD card quickly. Netbooting wears down 0 SD cards.

SD cards are less than optimal in Pis for general computing usage, but you could put something like iPXE on the card. That's how I'd work around the issue originally described in this thread. https://en.m.wikipedia.org/wiki/IPXE

At the tech lab I managed IT for back in college we had our own implementation of the preboot environment hacked together from Python and shell scripts on top of a nearly bare linux initrd filesystem. It would poll the router until it got an assignment, then fetch the appropriate files and boot the proper OS. We had to roll our own stuff because it was actually rsync'ing Windows hard drive images and would refresh the drive to clean state before booting every time the power cycled each morning. But you wouldn't need any of that and could probably hack together a solution that does what you want with iPXE.

Even the thought of having an aging battery isn't worth it to me. And not sure if it would help either.

What I have now works for this case, but it definitely put me off using it for other projects.

How often do you have power outages?

Can't remember one in the last 10 years.

10-15 times/year where I live now (suburb of major city) when you count the brownouts. even though power here is underground, there's enough aboveground that it still goes out or blinks often enough to have a UPS.

when I lived on a mountain, quite a few more for sometimes days at a time.

when I lived downtown in the major city in a high-rise, 2-3 times/year when you counted brownouts/blips.

a slack I'm on for the area had outages throughout the city.

so, if you only get it once in the last 10 years, count yourself lucky. I'm on the west coast, not Texas, and there is quite a bit of power generation nearby.

I'm pretty sure I hadn't a single one since I'm in Munich (2009). I remember some planned outages as kid at my parents place, and occasionally we tripped the circuit breakers.

I guess Munich has a really good grid. Everything underground and safe from natural disasters.

> Everything underground and safe from natural disasters.

Well, until stuff gets digged out for construction... I was affected by the arson attack in Berg am Laim last year, took about twelve hours for power to get back.

Wind, snow, and ice affect anything above ground. Doesn’t matter if 90% is buried if 10% isn’t.

Downtown core, everything underground, fire in the vault. That was the last outage I was living there for. Since there were 2 grid operators, I had power after about 6 hours, others across the street went days without.

We get a clock resetter about every other week, and > 20 min outage maybe twice a year in New Hampshire.

What? Do you reset clocks deliberately by cycling the power, or is it a description of an outage so long it will reset clocks?

I think they mean the latter. When I lived in rural Wisconsin we'd get those at least once a month in the summer, usually from thunderstorms. The lights would go out for a second, and the clocks would start flashing 12:00. Haven't seen it happen since moving into town though.

I think what they meant was an outage so short that it will reset the clock on the microwave/stove/bedside alarm but that you don't notice it otherwise. In the northeast, these happen frequently overnight, usually due to minor weather events or grid maintenance.

Most of my clocks are internet connected and battery backed, so the only ones left in my house that are affected are on kitchen appliances.

Home, very seldom. This is in a summer cabin where the infrastructure is really bad (and under reconstruction)... Getting much better but sill one every other month and since it is a summer cabin it can take quite a while before I can do it manually.

Ok, that puts me at ease :D

Change the boot order, so that network boot is after USB / SD card, or disable it completely?

There is no SD card or USB storage attached. To my knowledge it is only booting from network.

I think there's a bit in the boot boot order hex value that you can use to tell it to keep trying indefinitely, but don't quote me on that.

the new rom is supposed to have fixed this

if it's a pi 4 you can upgrade the rom to the latest

if it's a pi 3 you will have to put the rom on an sdcard (no eeprom on the board)

Can't find anything about it, do you have more information?

I think it was this one:

https://github.com/raspberrypi/rpi-eeprom/blob/master/firmwa...

I had the same problem as you, updated my eeprom and the problem was fixed

Thanks! I'll make sure to retry when I get a chance.

Can you get something that responds very quickly, even if only to tell it to reboot?

Strange that it stops trying network boot; most of the systems I have will sit there and retry PXE boot if everything else fails.

How long are your power outages typically? Is it short enough you put the router and NAS on a UPS? Maybe have a backup generator for core services?

I'm not saying this would work for you but for some people, like me, a down and dirty solution might work fine. I live in a village and short power outages (a few seconds to a couple of minutes) are relatively frequent, so a UPS covers many of my needs... for a short period of time, which is most of the time.

For out of the ordinary scenarios, such as the recent Storm Eustice, I just bought a 2.8kW petrol powered backup generator (not a particularly expensive one: equivalent of about US$500). I don't have an automatic failover yet, but that may be the next stage. Still, enough to run fridge/freezer, router, my computer, and possibly my central heating pumps (still need to investigate that).

The problem with UPSs is that they aren't necessarily quiet, and this pisses me off. I don't need loads of fan noise and beeping: I need some batteries in a box that will output a pure sine wave at 230V AC without making a big fuss about it, which is doable, but I'd also like it not to cost an arm and a leg.

"Is it short enough you put the router and NAS on a UPS? Maybe have a backup generator for core services?"

Sometimes you donts want to have tons of equipment hanging around - for example I keep all my mit in a small closet to avoid unhappy wife syndrome.

This is fair. You need to have somewhere to put it. My generator lives in a small shed outside so that wasn't too hard, and you can get some pretty small UPSs, but if you need to keep things up for longer, or have a lot of kit to hook in they get pretty bulky.

can you use this option?

https://www.raspberrypi.com/documentation/computers/config_t...

If your power is sketchy, get a UPS…I have one that powers the NAS, cable modem, switch and WiFi. (And pi hole. )

Until this is "fixed" by the Pi project (indeed, most PCs will boot loop if no boot devices succeed), I'd try something like iPXE (loaded onto SD or USB.) That should be able to loop properly.

https://github.com/ipxe/pipxe

what i'm hearing is that you have insufficient pi. if the dhcp were a pi and the nas were a pi, then they'd be ready before your doing-stuff pi timed out

> But my router supplying DHCP and with info about where it should look takes guessing 3 minutes. And the NAS containing the actual files that the Pi should load is in a virtual machine that takes, guessing, 8 minutes for the hypervisor and VM to start.

And thats why people get paid to manage networked devices, if a contingency event occurs and the network doesnt come up properly, then its not setup properly.

Logical thinking is one of the things the RPi makes people engage in, not superstition.

Really nothing wrong with the setup. Everything on the network handles it fine except for the pi.

I'm curious what you think is wrong with it.

I've been netbooting Pi's since the very early days and have been running networked sound since then (replacing earlier PC-based setup). Still running on the same old Pi 2's with Wolfson/Cirrus cards. SD cards are write-locked and all they do is load the kernel; as soon as the system boots, the root FS is mounted off the server (looping on failure, e.g. server not back up yet). Zero reason for the SD cards to wear out (no writes), the Pi's always come back after power fails (and we get plenty of them).

What software are you using for networked sound? I’ve got three echo dots serving a whole house audio amp, but they’re a little clunky to use and I’ve piles of old RPis sitting around

Icecast driven by mpd on the server; 'mpd' output is also captured as raw PCM and sent over the network to anything that wants to play it back (playback software is just a hack that reads the data and dumps into the soundcard, basically it's pretty much a C program that implements 'netcat | alsa').

Thanks for sharing, didn't know that.

Similar PSA: a common Pi complaint is that SD cards wear out quickly under 24/7 use. Routing most logs to memory rather than disk vastly reduces the wear on the SD: https://github.com/azlux/log2ram

Running backblocks -n will often fix them (non-destructive read/write - it causes a rewrite of every block on the device).

Don't do it often or you'll cause extra wear, but if you find yourself getting random errors from the SD card, give it a try.

Many services log via journald, so to move those off the SD card, you can just add /etc/systemd/journald.conf.d/volatile.conf with contents below and `systemctl force-reload systemd-journald.service`:

    [Journal]
    Storage=volatile

Check what your full config is with `systemd-analyze cat-config systemd/journald.conf`.

The split file approach is very handy for Pi-s. I have a set of configs I install on any new Pi I get (resolved, locale, timesync, etc.).

You can do a very similar thing with services in e.g., /etc/systemd/system/example.service.d/override.conf, which I like to do to constrain their sandbox more than the system package does (ProtectSystem=strict, SystemCallArchitecture=native, SystemCallFilter=@system-service, ~@resources @privileged, etc.)

Thank you for sharing! I looked around and found this cool link:

https://blog.alexellis.io/state-of-netbooting-raspberry-pi-i...

Do you have a preferred tutorial on this?

This is item 1 on google (https://linuxhit.com/raspberry-pi-pxe-boot-netbooting-a-pi-4...) but happy to see whats worked for you.

I don't. But I really want to write one, because all the documentation on how to do it is quite poor.

As I've been playing around with my Raspberry Pis, I've found this too. The hardware technical documentation is good, but, well, _very_ technical. There is a lot of very good information, but it's spread around in blog posts, forum posts (a lot of those), source code and comments, etc.

But not a lot of how-tos for intermediate stuff.

Hope to write up the stuff I'm working on once I figure it out :-)

I would be really interested in learning how to do this. In the past I've looked into PXE booting, but the tutorials all seem pretty opaque ("just make your router send a magic DHCP packet") or too specific to one configuration.

Did you try this official guide? https://www.raspberrypi.com/documentation/computers/remote-a...

What do all these network booted devices do? What do they do better than an SD booted device?

> you can change what OS a Pi boots into just by renaming a symlink.

Just to be clear, this is the symlink on your server right? Is there any advantage in doing this other than convenience of not having to go around your house and individually change out the SD cards of each device?

I've got a bunch of Pis around my house too, but they're just gathering dusk ;) I had all these grandiose ideas, but something, something ADHD.

Reply on news.ycombinator.com