Get Smart with SmartOS | Hacker News Reader

Get Smart with SmartOS

admin-magazine.com

66 pointsbased229 comments

I'm running my most recent project partially on SmartOS via Joyent's public cloud, Triton. The experience is a mixed bag. SmartOS itself is absolutely fantastic (I also have a few Debian hosts left over, but I'd like to migrate those to SmartOS as well), and Triton is great as long as you avoid the web UI, which is unbearably hard to use if you have a slow connection (or even if you don't, but I use Terraform and Packer so I mostly don't have to look at the UI). The only real problem is that Joyent either doesn't document anything, or it's hard to find, and they don't maintain half the software they advertise (but also won't confirm or deny that it's not maintained or mark it as unmaintained, so good luck finding out what you should be using instead).

When it comes down to it though I prefer Triton to the other big cloud providers just because the firewall product is fantastic, bare-metal machines running SmartOS are fantastic, and best of all it's not Amazon (which is all I really want, if I'm honest).

> The only real problem is that Joyent either doesn't document anything, or it's hard to find, and they don't maintain half the software they advertise (but also won't confirm or deny that it's not maintained or mark it as unmaintained, so good luck finding out what you should be using instead).

Have you sent that feedback, possibly pinged bcantrill on the issue (being Joyent's CTO I'd expect devdocs would fall under his purview)?

Yes, I've mentioned it to support. (EDIT*: removed "multiple times" because I'm not sure that's true; I forget if it was just once or if I followed up about other projects)

Glad to hear that there are aspects of SmartOS that you enjoy, and sorry that it's been a mixed bag. More specifically, my apologies on the documentation: it's an area in which we've always been very short staffed, and our focus on documentation has necessarily been more aligned with our products than our projects.

As for maintaining software: if you can be more specific where you're running into issues, we can figure out what's going on. Generally, we don't have too much software that isn't maintained at all, but we also do reimplement components from time to time, and you might have run into an issue on a component that is being replaced -- but my apologies if or where we weren't upfront about that...

Thanks for the reply; the problem I've had is mostly with things on your GitHub and images you maintain. For example, I asked about the Postgres image and whether it was maintained ages ago and was told that you're not sure if you'll maintain it or not. This is fair, and I don't expect updates the next day, but it's still on Postgres 9.6 and doesn't mention if you plan on ever updating it again. If not, that's fine, but please mark it as such so we know if we should rely on that image or not (we're building our own right now, which is fine, but we'd rather know).

Also, lots of little things on GitHub like the examples of using Prometheus and others have PRs (some of them by me) which have never had any comments, no acknowledgement that they've been seen, but also no indication that the project isn't maintained or is just an example and that we shouldn't submit PRs.

All that being said, in general thanks for a great product.

Most of the application specific images have stopped receiving updates. This is because they were time consuming to create/validate, and were mostly just the base-64 image with whatever package pre-installed via pkgin. This was also happened around the time of the rise of Docker, so most people were opting for docker images produced by upstream maintainers.

In most cases, you can just make a base-64 image and `pkgin in` whatever package you wanted and it's pretty much the same thing.

The Prometheus stuff is heavily used by us internally, and while it's usable, it's pretty experimental (i.e., changing quickly). I don't see any pull requests or issues that are obviously from you, so if you point me at something I can take a look at it.

I wanted to like SmartOS, but the documentation for Arch really had me spoiled. The Joyent/SmartOS docs are a mess of information for different versions that really aren't conducive to a confident first time user. That and it had problems with a pretty vanilla supermicro build. It did pick up my linux ZFS pool without a problem though, which was a nice surprise.

This has been my experience as well; I really love SmartOS and Triton, but until Joyent cleans up the documentation or builds a big enough community to crowd source it I can't generally recommend it to new users, which is sad.

We are run our Triton / SmartOS nodes mostly on SuperMicro hosts. I have never encountered hardware compatibly issues.

Do you recall what issues you bumped into? Might save me some potential headaches / pitfalls in the future ;-)

One of the goals of the openzfs community is to ensure feature compatibility when moving a zpool (a collection of disks that run a storage pool) across FreeBSD, Linux and Illumos (on which SmartOS is based). This was reaffirmed on the openzfs call that happend just 5 hours ago.

See: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HB...

I wish SmartOS was more widely used, and had better hardware support. I tried SmartOS in a VM and loved it, but when I tried to install it on a Supermicro box it didn't support the hardware.

You may want to try it again. We (Joyent) are using some of the latest Supermicro hardware in our cloud.

We've been running our private cloud on Triton / SmartOS with Packer and Terraform for a while now and for us its really working well. I do think however that Triton / SmartOS works best when used in a true cloud native fashion (meaning stateless containers). One could probably use it in a "classic" way but you might not reap all the benefits of the solution.

I also like that Triton has clear upgrade paths when new versions are released. Something that was usually the biggest issue with OpenStack.

I try to run everything as much as possible in native (ie. Illumos) zones since I like SMF better then systemd. But if I can't get it to work I use an LX zone (container with Linux ABI compatibly) with CentOS. Which gets me all the benefits of true containers (Zones) with ZFS and dtrace.

I really wish I could use SmartOS, but I couldn't even install some common Python packages without getting a compilation error. Looking such problems up on issue trackers revealed old requests for compatibility with SmartOS, and no follow-up from the developers.

I'm not sure what the recourse is, I just don't know enough about C to fix this myself. I assumed this would be a common enough problem that the solution would be well documented somewhere but there seems to be nothing out there even mentioning the issue.

I'm happy to do all my work in LX-branded zones, and that's certainly better than the alternatives (for my purposes), but having two operating systems still kinda irks me.

I recommend using the package manager than building yourself. If something needs to be modified about the source, it will first be patched in the widely used package manager. convincing upstream to take in a patch & then waiting for a new version to be released with that patch takes time.

The problem is, I am using the package manager (pip in this case). I guess because the wheels aren't built for SmartOS it has to compile them.

I'd guess they mean "use the system package manager", given that they talk about patches not in upstream?

Ah, I see, thanks. Still, that's a non-starter. Many of the packages I use are not available through any system package manager, and it would preclude using virtualenvs.

OTOH, I desperately want to move away from Python.

Could you let us know which packages you're trying to install via PIP that cause you issues?

On software compatibility unfortunately the actual state (nearly anyone only on GNU/Linux) lead to forget that other OSes exists and many non-POSIX/non-portable features are more and more used anywhere...

That's not an IllumOS/SmartOS or any other unices faults nor something they can really solve...

Love SmartOS! IMHO zones set the bar on containers back in the 2000's, and SmartOS is the evolved logical progression of this. It's truly a ferrari. However, it needs a paint job to accrete community. If you want a cohesive system, and want to become a paas provider, triton-sdc. But I think at the medium and low-end, smartos needs something like proxmox to compete where project fifo appears to be having a hard time. Any takers?

Let me clarify, proxmox (or the vsphere web ui for example), would be a fantastic management ui on top of the smartos engine that would bring some gravity to accrete userbase. Some organizations are just more comfortable with UI tools. Sure, your hairy-knuckled sa's and programmers will favor maintaining version controlled manifests for zones, and keeping things highly automated, using terraform, etc, but there's a huge base out there that will download smartos, stare at the console prompt, and lose interest unless they can treat it like a vsphere or proxmox machine, like they're used to, and are trained on. Project fifo...frankly, needs some good natured competition to breed better product in this space.

'accrete userbase'? Yeesh, where did that bit of geek-speak crawl out from under? Put it back.

Do you hear that whooshing sound? It's the point I was making that flew right by you.

While I was using OpenSolaris (and Solaris before, till 9 6/06) as a personal desktop from SXDE->SXCE->OpenIndiana, I have tried many fork (Belenix, Damm small solaris, Nexenta, OI) I have to say that IllumOS is essentially dead as Irix was before and no one have enough manpower and FOSS culture to bring it back to life.

OpenSolaris nicest features like zfs (with IPS/BE, zones, branz on top), crossbow, SMF, FMA etc may remain or recreated for some time but the whole OS unfortunately is dead.

Irix legacy leave us with OpenGL, even if today nearly nobody know their origin, xfs even if is now EOL fs, STL, even if C++ now is well... Harmful. Many other good things including probably the best CDE out there but it does not exists anymore and it's hw with it. OpenSolaris will probably do the same.

I think people coming from traditional unix companies do not understand the hard lesson they get from Microsoft and GNU/Linux: today's student's, young aspiring users are tomorrow technicians. It doesn't matter how good your software is, if it's not good on their desktop will be marginal, the same for good big iron architectures. OpenSolaris itself start to be known thanks to Ian Murdock (Debian founder, hired by SUN, dead in unclear situation few years ago), not before OpenIndiana, simply because of the lacking of FOSS culture SUN has, even if OpenIndiana does not introduce, at least at start, any particular feature.

I still miss osol, I still hope for a real alternative to GNU/Linux, only to Linux to be more precise, I still hope to see a good architecture at a price and assembly that can be used as alternative to x86 on the desktop (I hope for OpenPower now) but nothing came to the horizon...

After SUN death I've tried to came back to FreeBSD: essentially a pain to use on modern desktop hw, even if I pry for many FreeBSD features from jails to geli passing through securelevel, freebsd-update, a base system consistent with the kernel, a damn simple init (despite raw compared to modern init's), good fw/network performance, rock solid stability, ... but still does not work well on commodity hw. I dream DragonflyBSD hammer storage but I can't really run Dragonfly as my desktop...

In the end I hope we as FOSS users, can came to a modern OS on free hardware and I fear our next desktop will be a kind of mobile-crap with lock-in builtin to the point of actual mobile crap. We need such free solution not only for IT but even for democracy.

SmartOS looks interesting, but I've never had a good chance to try it on a real project. I do actually have an extra box at work that I suppose I could spin up in my copious spare time, but I'm not sure what I'd do with it. Are LX-branded zones complete enough to run something like a build environment based around a vendor-dropped toolchain?

> The Unix-like operating system [...]

I know I'm late to this party, being a bit too young to have direct experience of the Unix Wars and "UN*X" snark and so on, but this really does grate. It feels like trademark law has been twisted toward requiring misrepresentation rather than preventing it.

> Are LX-branded zones complete enough to run something like a build environment based around a vendor-dropped toolchain?

Yes. Ubuntu is even certified by Cannonical iirc. Given that it boots on USB, and ZFS creates a pool in like a second, you can get started in a few minutes providing your internet is fast enough to pull the guest images.

It feels like trademark law has been twisted toward requiring misrepresentation rather than preventing it.

There are technical differences, as well. Most of the time one will never tell the difference (hence "Unix-LIKE": you'll likely never know). But BSD is not Linux, and vice-versa. For a really poor analogy, kind of like the electric version of an ICE automobile. Looks a lot like the ICE version (well, the grill is kind of funny), operates mostly the same, but under the hood it's different.

I don't have a problem with people calling Linux "Unix-like", I have a problem with people feeling like they have to call actual Unix descendants (like forks of BSD and SVR4) "Unix-like" while IBM gets to use "Unix" for z/OS.

Reply on news.ycombinator.com