I put together five weekend projects covering trying a different management tool, testing Proxmox HA failovers, reviewing LXC helper scripts, establishing a Docker container patch routine, and exploring CetraFS or MeshXFS for VMFS-like shared storage. The HA and patching projects are useful checks on an existing lab. Also some pretty cool storage level projects. Hopefully these project suggestions will get the ideas spinning for those in their labs this weekend.
Enabling HA in your Proxmox cluster is a step in the right direction. However, it doesn't always mean your app will recover from a failure. I take a look at five practical checks for your HA. We cover host failure, network partitions and fencing, recovery-node compatibility, storage availability, and application failures inside a running VM. The goal is to see when the application becomes usable again, which is better than simply watching the VM restart.
This week’s projects cover a lot of different types of learning. These include what you can do with a single Proxmox server, comparing bare-metal Docker with Docker running in a VM, testing with nested virtualization, adding passkey logins with Pocket ID, and guardrailing your AI agent access with Octelium.
What are you building or experimenting with in your home lab this weekend?
Very cool. I love the ex-thin client setup as well due to their power profile and have been tempted to pull the trigger on a few of those myself. Like you too, I am really thankful I had a little stockpile of storage and RAM resources before everything went crazy!
You may only have the resources for a single Proxmox server to start out in your home lab. But, actually, did you know you can do just about everything with a single node that you can do with a cluster in terms of learning? Check out this post where I take a look at really how many projects and learning experiments you can do having just a single node to work with.
I run a multi-node Proxmox lab with Ceph and 10 GbE. But do you have to have that for a Proxmox home lab to be successful and a great learning playground? No! In my blog post here I take a look at what I think are things that you don't have to have when starting your home lab with Proxmox VE Server. What about you? What upgrade items did you skip?
I took a look at the Pocket ID solution as a small identity provider for self-hosted apps in the home lab. If you have thought that something like Authentik is overkill for what you want to do, I checked this solution out as your passkeys solution for the home lab. we look at the installation of Pocket ID inside Docker, OIDC integration with a test app, permissions, and we also think about recover after losing a passkey for a regular user and admin user.
These are 5 project-based home lab projects that will not only improve your technical skills but also will improve the state of your lab. I take a look at various projects across the lab environment, including the Let's Encrypt change coming, Unifi Network 11, Hermes topics including Netbox and skills, and TrueNAS 27 RC1. Let me know what projects you are working on this weekend.
Let's Encrypt is going to a much shorter certificate lifetime. I think this is something home lab admins should start preparing for now. Especially if you have your scheduled tasks and CRON jobs configured for the current 90 day expirations. I put together a set of things to check if you are using tools like Certbot, Traefik, and reverse proxies to make sure you are ready. Also, I'm curious how you are handling certificate renewals and automation across your home lab. Are you using built-in ACME clients, Certbot, or something else?
Up until now, I have been self-hosting the Unifi Network Application inside of my Kubernetes cluster. But the writing has been on the wall that I needed to make a change as Unifi has deprecated the standalone Network Application in favor of the Unifi OS Server solution.
UOS is the platform that now "runs" the Network Application. I decided to go back to a more traditional self-hosted solution with a Linux VM running UOS and running the Unifi Network Application 11. Take a look at my guide showing how I made the switch and migrated my data. What about you? Have you made the switch yet?
So, one of the first things that I have put my Hermes agent to work doing is populating my home lab inventory and documentation in Netbox. Documentation and inventory are two things that most of us are lazy about keeping updated. However, this was one of the first things that I wanted to use Hermes for in the home lab as it can work tirelessly to gather info, import and update that info in Netbox, and most importantly, keep it updated. Are you using Hermes for this as well? Let me know what tasks you have delegated to it for your home lab?
I put together a short list of Hermes skills for Proxmox, Technitium DNS, Docker, Uptime Kuma, TrueNAS, Portainer, and infrastructure diagrams. The post looks at their capabilities, setup requirements, etc. These are definitely ones I would line up for testing, with read-only acess as the starting point. I’d be interested to hear which infrastructure skills you are using with your home lab agents and how you limit an agent’s access.
This week I explored a ton of new things in the home lab. I considered everything from isolating a Hermes AI agent on its own VLAN, deploying Podman apps with Yantr, bastion host with Warpgate, recovering files from Proxmox backups with pve-flr-portal, understanding ZFS space reservations, and testing WSL containers. PHeww! It has been a week. But any of these would make a great home lab project to try out this weekend. Check out the list.
So, I have been experimenting with Hermes inside my home lab and I decided to build out a dedicated AI agent network that allows easily controlling the traffic that is allowed out of that network and to other parts of my home lab.
I created a dedicated VLAN (999) for my agent. I am currently using a Palo Alto PA-440 that handles my routing and firewall policies. Unifi is carrying my Layer 2 traffic. Then, I created the new network as a new Proxmox SDN VNET.
The interesting part is the traffic controls. Currently I have it setup where my management workstation can connect TO the AI agent box, but not the other way around. Eventually I want to give Hermes limited access to services like Netbox, Proxmox, Kubernetes, etc.
Curious how many of you are isolating self-hosted AI agents and controlling how traffic is sent out into the rest of your home lab.
Have you struggled to find a good solution that allows secure access to your home lab or even production environments? I deployed Warpgate with Docker Compose. This one impressed me as it was able to provide centralized access to home lab services with a well-rounded feature set. The post covers setup, SSH target config, authentication, web access, MFA, roles, and thinking about preventing bypassing it as the secure gateway. One practical snag was permissions on the persistent data directory, which I also cover.
I tested out pve-flr-portal which is a tool that adds a snapshot timeline and file browser to your Proxmox’s existing PBS file-restore. I walk through Docker setup, authentication, permissions, and recovering individual files. It also allows direct restoration into a running VM through the QEMU guest agent.
I recently created a ZFS mirror on a Proxmox host. I then noticed my new 500 GB VM disk reserved 508 GB, instead of just holding only a few gigabytes of data. This led to writing up a list of checks for ZFS, including thin provisioning, block sizes, LZ4 compression, ARC memory limits, and the existing scrub schedule. This post covers what I found, and how you can check these settings. We see which changes affect existing disks you have created vs. new ones.
If you are looking for learning projects in your home lab this weekend, check out these 6 projects from my recent home lab testing. These cover Linux troubleshooting, VLAN isolation, Proxmox self-service, TrueNAS monitoring and storage, and Docker image scanning. Each of these I think are a great starting point for learning new things over a weekend.
I put together seven Linux commands that I think are some of the most often misunderstood or unknown tools that I find useful for troubleshooting. We take a look at ss, lsof, findmnt, journalctl, watch, ip route get, and timeout. If you are looking for a way to find what is listening on a port, identifying processes using files, understanding mounts, tracing routing decisions, and troubleshooting services. These are the tools that can help you do that quickly.
Check out this walkthrough for checking which connections can cross home lab VLANs. It is important to identify the firewall rules that allow traffic, and make sure you test your changes with port tests and firewall logs. We also take a look at Proxmox VM firewall rules for blocking traffic between workloads on the same VLAN.
If you have ever been in need of an easy self-service portal for Proxmox to provide access to developers, or for selling resources, I found a solution that just might fit the bill. PVE Panel is a self-hosted solution that provides a self-service portal for Proxmox where you can define customers and assign specific resources to those customers using the workflows the solution provides. I found it easy to setup and easy to start providing resources. Check it out.
I connected my TrueNAS box to Graphite and Grafana. I had both of these running in a Talos Kubernetes cluster. The metrics were sent over correctly, but the imported dashboard I was using from the Grafana community showed no data because it expected older TrueNAS metric paths. So, I got this fixed after remapping the panels. Then, I found another issue. It interpreted an ARC value in bytes as megabytes and displayed 12.7 PB instead of roughly 12 GB!
Check out my full post detailing what I ran into, why I saw what I did and then how I fixed them. At the end, I was able to have a really good TrueNAS monitoring dashboard in Grafana.
I’ve been testing Dockhand’s vulnerability scanning in my home lab. It can scan a newer image with Trivy or Grype before it does an automatic update to replace the running container. You can also block the update based on the rule you choose which I think is pretty cool. I wrote up a post on configuring this setup and the rule you can choose. Curious how others decide which container updates to automate in your home labs or production?
I think these are good points. I also run a Talos Linux K8s cluster in the home lab and have full GitOps running with ArgoCD. It is a slick way to run services. However, I think many don't want that kind of complexity in the home lab, even though I think it has a lot of advantages to running things just on Docker. But the great thing about self-hosting is we have tons of options.
In case you haven't heard, Portainer 3.0 is moving to a Kubernetes-first codebase. It will keep native Docker, Swarm, and Podman environments available but this is no longer the focus. I looked at what this shift means for home lab users, including KubeSolo, the D2K solution Portainer has developed, continued support for Portainer 2.45 LTS, and we look at whether home lab Docker users need to change anything yet.
I tested the new TrueNAS storage plugin with a three-node Proxmox cluster and a TrueNAS VM. The really cool part was watching a disk created in the Proxmox VM wizard appear as a new zvol in TrueNAS. I also ran into a few setup details worth noting. These include creating unique serial numbers for virtual disks, the iSCSI target configuration, and installing the plugin on each node. The post walks through all of my setup and what I verified in the lab.
I wrote about Pangolin last year, when its clustering capability was a bit cumbersome. It still required a custom deployment engagement with Pangolin. Version 1.23 brings DNS and certificate management into Pangolin and publishes a two-node HA deployment guide. They have now introduced the HA clustering component as a native part of the deployment without jumping through hoops.
You still have to supply your own HA load balancer as part of the solution but the process to get HA for Pangolin now feels within reach for the home lab.
A two-node Proxmox cluster can lose quorum if either of your hosts fail. This is true even if the server that is left is healthy and its workloads are still running. If Proxmox HA is involved, the HA watchdog service will do what is called self-fencing that can also reset an isolated node to keep it from having split brain in the cluster.
I explored how Corosync quorum works, what self-fencing technically does, and how an external QDevice is used. It provides a third vote without requiring another full Proxmox server.
The new official TrueNAS storage plugin I think is going to lead to a broader change for Proxmox. When you combine this with Proxmox introducing 24/7 enterprise support, storage vendors now have a really good reason to build native integrations instead of relying only on generic NFS and iSCSI connections that are found in Proxmox itself.
I look at five trends that could shape Proxmox storage in 2027. These including things like vendor-supported plugins, API-driven provisioning, per-VM storage objects, broader NVMe/TCP adoption, and the role storage support will play in VMware migrations.
Do you think other major storage vendors will follow TrueNAS with official Proxmox integrations?
I put this together as a weekend project list based on tools I have recently tested in my own home lab. It includes a few newcomers like Changerawr, which I have found useful once I worked around a migration bug. Also, along with Proxmox and TrueNAS-related projects. These are genuinely projects that I think don't take that much time, but they provide some really great features for our home lab environments.