197 karma · joined March 11, 2020
It's a big ass computer (technically many computers in a rack) that gives you a cloud provider style API to manage on-premises infrastructure. Plug it in, connect to the web console or API, and provision yourself some sweet sweet virtual machines, disks, and VPCs.
I get what you're saying though, and you're right that certain workloads like Kubernetes on the metal that can take advantage of hardware wouldn't be a good fit for Oxide today. We could decide to change that in the future, but I don't see that as a priority for Oxide given all else we have to build first.
Switching to Linux and KVM would gain the ecosystem benefits but Oxide would lose control of the host OS which can impact our security stance. Better to tightly integrate at this layer and expose the primitives that are needed. It's not impossible, but we're already maintaining the existing Helios host OS. Adding another wouldn't be beneficial.
The host OS is an implementation detail since you, the customer, aren't running workloads directly on the host OS. The VMs running on Oxide run on our host OS, much like other cloud providers.
That rack-scale hardware and software co-design allows Oxide to provide higher CPU and memory density in the same footprint at lower power than competitors. That's super helpful for companies operating at scale where data center power matters. The co-design allows us tackle security problems by eliminating the BIOS, providing attestation from the firmware up to the guest VM, and actually updating the firmware that's on Oxide rather than letting it rot like most customers do today.
This is why when people ask which verticals Oxide targets we kinda say "all of them" because different customers benefit from different Oxide features, but all customers need the core VM, disk, VPC abstraction. Some customers come to us because they don't have an API to manage their on-premises compute and Oxide solves that. Others come to us because they need absolute confidence that there's no malicious firmware running in their compute stack. Others come to us because they are tired of paying exorbitant amounts of recurring money just to run on-premises compute.
Our job is to make Oxide an appealing on-premises computing platform for customers to run their public cloud provider workloads and on-premises workloads without it feeling like it's an entirely different platform than you're used to.
The core primitive on Oxide is the instance (virtual machine). We could support some container primitive, but that's a larger product direction discussion. Our host OS is Helios (Illumos) so there are details to iron out there regarding what abstractions we would build and expose to the users. Not impossible but not something we're currently pursuing either given that we have other immediate product asks.
If you're at a scale where compute density, power efficiency, security, and rack-level API management matters then that's where Oxide makes sense for you.
That's what we prototyped before local disk was released and before we started disk hot-plug work.
> Could you attach a single large volume, and do path-based provisioning on that?
Possibly. We'd still want disk hot-plug first. Otherwise, customers would have to create their cluster in a certain shape before using PVCs.
Autoscaling is a requested feature, and Karpenter fits that shape naturally. The nuance is to decide where something like the Cluster API provider ends and Karpenter begins since there's a bit of overlap in concerns. Specifically, both want to manage Kubernetes nodes but for different reasons. We're discussing it though. We have a Kubernetes watercooler meeting today where we'll likely discuss the comments in this post!
I don't necessarily want to match the public cloud experience if there's an opportunity for Oxide to exceed the public cloud experience. Eliminating the overlay is a good example of this. We have customers using external subnets to eliminate the overlay but we haven't integrated that into our controllers yet.
There's an opportunity to more tightly integrate at the network layer but we'll want to get our load balancer released first.
My colleague demo'd Karpeneter internally. We haven't committed releasing it yet but we're discussing it.
[0] https://github.com/kubernetes/cloud-provider/blob/master/clo...
> From the inside, is Oxide a place where a fresh grad can actually be useful? Or is the "complexity floor" of hardware/software co-design so high that you really just need a few decades of experience to be effective?
I believe Bryan spoke about this publicly on podcast episodes in the past but the answer really is no and yes. Generally, the things we're working on at Oxide require a both depth and breadth of knowledge in various areas of the stack that early career folks don't yet possess. We also have limited bandwidth to assist early career folks because of our small company size. I know this sounds like gate keeping, but mostly everyone at Oxide wants to support early career folks. Unfortunately, the general consensus has been that we just don't have the bandwidth for it and it would be a disservice to a candidate to hire them and not support them the way they would need to be supported. That being said, we're getting bigger and will likely have more bandwidth to change this stance as time passes. If you see a role you'd be a good fit for I do recommend applying. The application process itself is valuable for the applicant.
Disclaimer: I work at Oxide.
To be honest there's really no secret sauce in there. It's primarily how to get started with agents, when to abandon your context and start anew, and advice on models, monitoring cost, and prompting. This is not to diminish the value of the information as it's good information written by great colleagues. I just wanted to note that most of the information can be obtained from the official AI provider documentation and blog posts from AI boosters like Thorsten Ball.
There are things in life that have high risks of harm if misused yet people still use them because there are great benefits when carefully used. Being aware of the risks is the key to using something that can be harmful, safely.
Here's what I can knowingly share. We sell full-rack and half-rack SKUs today coming in at 1024/16TiB and 2048/32TiB of CPU cores and RAM respectively. Disk varies depending on the drive size. On top of the cost of the rack you'll pay an annual percentage for support and that's it. No software licensing or other weird fees.
We'll ultimately ship a next-generation rack to take advantage of changing constraints (e.g., physical form factor, power requirements) and we'll provide a mechanism for upgrading to that generation of rack as well. Likely, in the form of multi-rack connectivity to move workloads or perhaps even financial incentives to purchase the latest and greatest.
I don't work on the hardware side at Oxide so I don't have further details to share. Great question!
Selling a customer a contract for on-premises computing and giving them a fake metal box and SaaS is borderline unethical depending on the terms of said contract. I understand the sentiment of that point though. There are many reasons a customer chooses to own instead of rent. Legal requirements, financial incentives, and even control over performance to name a few.
On-premises computing was so good that the cloud providers packaged it up and sold it back to people at a premium that could only ever be rented. The finances of that model don't make sense to many businesses as they look to reignite their on-premises computing with the modernity of the cloud providers. That's where Oxide shines in my opinion- being able to have on-premises computing that combines the efficiencies of the hyper scalers with an API-driven approach to managing resources. We take that a step further by building hardware and software in-house for additional benefits such as power efficiency, control over the networking stack, additional telemetry, etc.
No hiring process will ever be perfect but at least, as the experience mentioned previously touched on, the candidate is left with an artifact that they can then use in future applications. Candidates walk away having learned something about themselves. Open roles are also limited, applying for such roles is also optional, and it's up to each candidate to decide how much time and effort they wish to put into the materials.
We're humans here at Oxide too, and we're doing the best we can to ensure the hiring process is fair and humane as well.