269 karma · joined December 6, 2022
We could debate about strict definitions all day, but I think the vast majority of people differentiate between a Raspberry Pi 5 (not a great example given it needs storage, pick anything with eMMC if you truly want everything on a single board) running a full Debian-based OS, and a Pico running MicroPython (or whatever your poison of choice is) and one task at a time, at least I do when it comes to this kind of thing!
For the 2nd, the file I grabbed initially was https://github.com/geerlingguy/ai-benchmarks/blob/main/obenc... - which I now notice wasn't modified in his repository, so I can check that out, but either way, the same version has been tested across everything thus far.
Armbian do a great job of handling support for a whole host of boards (including most I included in this list), so you'll usually have Debian/Ubuntu-based flavours. Vendor kernels and vendor supplied images will be hit and miss. Mainline Linux support is a flag you filter by on the benchmark comparison site linked in the article, but it's a difficult one to keep up to date and define exactly. It could have some kind of support, but miss out on display functionality, or WiFi yada yada. What would we then class as having mainline support? All hardware etc functioning? If so, very, very few will meet that definition.
I get the desire for the information, and perhaps I should have envisioned these types of questions, but all I initially meant for the post to be was a recap for people following me to see which boards I'd tested that were released last year :D
I don't know if I'd release everything as is, there's a lot of hard coded nonsense that I'd need to filter out and after testing 80 boards, I have a bunch of improvements I want to make. Opening up the test scripts to others isn't/wasn't a high priority item, as I didn't even plan on making this site initially :D
Room temperature data is also nice, but I'll need to draw a line somewhere, on average these tests take around 3-4 hours to run, and with 80 boards right now, best case scenario is 10 straight days of testing (in reality, more with setup and.. life). I could use data on the most popular boards and only do a handful, but I probably have to limit the scope at this point and clearly state the testing conditions and how it may differ in your real world setup.
I can definitely do more to expand on the testing process though, and I can look at sharing the exact commands that are run with my automation so that users can at least mimic the benchmark runs if not the full automation. I really appreciate the responses and feedback, though, so thank you!
Otherwise, there are dual NVMe HATs for the Raspberry Pi 5 but you'll be splitting a single PCIe Gen 2 lane across both drives (unless you go for the much more expensive HATs that have a Gen 3 PCIe switch on them to then share a single Gen 3 lane across them!)
As soon as I have all of that data in there (I think I'm at around 30-40% so far, the initial batch of testing has been a slog, data entry for this took a back seat) I'll be enabling that option and it will all be there to view on comparison pages, and search for to help find/compare on a deeper level.
On the samples from vendors front, yeah, that's definitely a concern, though in the years I've been working with manufacturers and had samples from them, I'm fairly confident that no "golden sample" binning is going on as I've had quite a few shockers, hah! Not to say that it couldn't/wouldn't happen, I wouldn't want to introduce that doubt. Perhaps, like my review website, I should add a quick note/indicator of whether a board was obtained directly from a vendor, or if it was purchased.
I'm currently in talks with a few of the vendors to start filling the database with missing boards from their ranges so we should have more data available in the coming weeks. There should be 2 BeagleBoard (Eco Green and PocketBeagle 2) arriving tomorrow/Tuesday to expand their range on the site!
I wrote a blog post about why I made the site at https://bret.dk/introducing-sbc-compare/ if anyone's interested, but to TL;DR it, I didn't set out to create a site like this, it was a side quest after creating the automation and database to support my reviews, which do indeed focus on the hobbyist trying to explore Raspberry Pi SBCs and their many alternatives.
I have full specifications and hardware capabilities hidden behind a feature flag at the moment as I'm working my way through adding all of that data (currently at 80 SBCs in the database, and I'm only adding those I own and have run tests on) so there should be something similar to what you're asking for soon. Thanks again!