- low power or performance? - ease of use vs. tunability? - closed vs. open-source?
For the server OS, your choices are practically either Linux or FreeBSD (FreeNAS is based on the latter). If you don’t need zfs for NAS, OpenBSD becomes another interesting option - assuming the hardware is supported, is rock solid and easy to administrate, but may have some learning curve.
Tinkering with something like RaspberryPi is not a bad idea - you will spend $50 or so but will learn a ton and in the process will understand your own requirements better. However, if you want NAS, I recommend buying a server-class hardware (like T30 mentioned above) and invest time in understanding how to use zfs and do reliable backups, assuming your data is valueable. I personally don’t think you should aim for lower power for NAS with any serious amount of data.