Running Windows desktop tests from Linux with disposable VMs | Hacker News Reader

Running Windows desktop tests from Linux with disposable VMs

qawolf.com

94 pointsjadenlemmon32 comments

Not to dismiss this, but when I was working on a problem on Windows recently my LLM (6.1 Sol / oh-my-pi) ended up doing the same thing, although it used Qemu and snapshots to accomplish that.

Eventually it got to a point it could iterate very quickly and was really only constrained by total system RAM which meant a dozen at most copies could be running simultaneously.

Same. It's quite remarkable. Two other things Claude and I have done. I have a spare workstation running proxmox in my garage. Claude can now create, snapshot, delete, extend, start, and restart vms. It's been great. I also have skills that allow it to spin up spot instances when I need to test things that rely on hardware I don't have like certain Nvidia gpus or 100gb networking. What a time to be alive.

To be fair, you never needed LLMs to be able to do all that.

To be fair, often you have good ideas, but not enough time. I have had a lot of fun having Qwen3.8 27b running at home, having it explore and implement ideas that was way to ambitious for the time I had available to tinker with Home Assistant and Proxmox. It is an enabler and can also be a good tutor, when used as such. I have learned a lot.

"be able" implying capability rather than time constraints and opportunity cost which is what most people on here refer to when they mention something cool they're doing with LLMs that they weren't doing before.

True but it was a whole bunch of fragile scripts/functions with lots of ifs. I'll give you a real world example of what's different. Agent spins up a VM to run a build on windows. During the build it realizes it's out of disk space. Agent resizes disk on demand. Build finishes and it launches the app. Runs some tests and realizes it's cpu and memory constrained. Adjust the cpu and memory and runs new tests. tears it down and sends the perf logs back to me. Or it looks at system resources and decides what other vms it can pause while it runs the tests and then turns them back on after.

There are so many unknowns and the ability for the agents to dynamically adjust and make decisions allows for some really powerful autonomous workflows.

Not sure the lower bounds of CPU/memory/disk space that your project needs should be considered "so many unknowns".

Man. HN is brutal. Just one example. And yes, when you're looping agents and builds to identify performance optimizations and build features sometimes you don't know what those bounds are going to be and it's much faster to let the agents work it out, update their learnings, and keep building.

Not to take your credit away but there is something called automation, scripting, etc.

All those actions within my homelab are automated via Ansible and bash scripts (I only run Linux), I am only notified if something goes wrong like a backup failed.

I get it if you never worked in IT, that is fine, but you need zero AI to get the above fully automated.

Does Claude keep versioning of the changes?? I host my own repos via Forgejo, Ansible and the automation process consume the config files from there, one source of truth. If my whole setup burns with fire, I can bring everything back online from the offsite Ansible playbook located at Codeberg.

Why not both? My entire home network (dns, zfs nas, subnets, kube, proxmox hosts, ipa domain, its overkill homeprod) is represented by a git repo. Docs, plans, scripts to bootstrap nodes to domain join, kube config, proxmox tooling, ipa as code, nas as code, cert management, my own ec2 as a service, etc. Making changes is done via a mix of one off tooling, ansible and terraform.

I use Claude or codex to manage the entire thing, interchangeably. They keep detailed daily journals. They keep a central TODO.md up to date. They refresh stale docs or stale plans when new plans invalidate old decisions.

I have so many layers of backups and fail safes that I work entirely in yolo mode. It’s really insane what I’ve been able to deploy myself on nights and weekends. Stuff I’ve wanted to tackle for years but never got around to it.

Why not use snapshots/save states so the VMs start back up right away? This is what I do with QEMU.

Doesn't Windows do some drama if its not activated and has been running without a license for more than 48h or similar?

I don't use windows but have to run windows 11 VM to access special devices. I don't want to bother with licenses, although I have them, and creating a new incus QEMU VM from scratch everytime from an ISO takes ~5m, freeing me up from worrying about all the activation and license drama.

If the activation and license setup was easy/automated in QEMU, I would imagine snapshots would be way easier. I would prefer not needing to connect to the internet.

> Doesn't Windows do some drama if its not activated and has been running without a license for more than 48h or similar

Your info might be like decades out of date, since windows 10 it's just been an overlay and locking out some appearance customizations (you can still set them via other means though).

Ok, so you're saying I can run windows 11 without a license or activation for months on end?

Yes, since the release of Windows 10 in 2014.

Yes, you can also automate fetching a license and activating with https://massgrave.dev/ .

>"There’s a good reason why testing Windows desktop apps was one of the most requested features from customers: no one in the world has the infrastructure to parallelize thousands of runs per month..."

This article makes some excellent points!

If you've ever worked in, or alongside QA in commercial software environments, you know that being able to run a large amount of automated parallel Windows specific QA tasks is indeed a headache! And that's if an organization can accomplish that at all -- many organizations still use single human QA's and/or SDET's and still run many QA tasks manually... Of course, AI should/will help with some of this, but there still is the physical testing infrastructure to take into consideration... there are many barriers, not the least of which include such things as necessity for multiple Windows licences, setting up those Windows environments properly, instrumenting test harnesses properly, aggregating the results from multiple sources, etc., etc.

Phrased another way, there's a large market there -- if you can solve all or some of those problems, in whole or in part...

Anyway, an excellent article on this topic!

We used several different LLM's and our own skills to build out some novel approaches to document creation with new workflows using QEMU in Linux after upgrading our desktops to Ubuntu from Windows 11.

what is the minimum license that I would need to do this legally, with just one or two VMs?

It depends what you need to run. There exist a number of Windows images which can be run without any paid license requirements for test and validation IIRC.

https://learn.microsoft.com/en-us/windows-hardware/manufactu...

it depends on where you are running it. If using a cloud provider, license is typically built into hourly price.

This is super slow solution I did build similar solution based on dockur. And it was sooo slow. Just use windows docker containers on windows server. With some hacks you can run any gui application.

We’ve been providing a similar approach with disposable VMs and emulators/simulators since 2014 at TestingBot. We run our own single-use QEMU VMs with Windows and Linux.

Same here, we used Proxmox as host to spin up ready images built with Packer and starting them with Vagrant to provision the app on the system under test, the whole triggered by the CI build

What's the licensing on Windows 11 these days? Can an individual still run an unregistered Windows 11 installation basically forever with only minor restrictions?

For an individual, mostly yes. Unactivated will just keep running with a watermark & some minor restrictions.

Thank you.

Why is the comments from the OP marked as flagged or dead at the bottom?

Looks like AI written so it got auto-flagged by a bot probably

thanks for the advertisement

Nothing new here. VM + WebRTC + warm pool to absorb boot time

I work at QA Wolf and lead the development of automated testing for Windows desktop apps. I wanted to share what we did and how we did it.

This post covers how we added Windows to a test infrastructure built around Linux runners. The central problem was giving each test a clean, logged-in Windows desktop without waiting 2–3 minutes for a VM to boot.

We extended the device scheduler we already use for mobile testing. It maintains warm pools of Windows VMs on GCE. Each test claims a ready VM, uses it once, and releases it. The VM is then destroyed and replaced, rather than cleaned up for the next test.

The test code stays in a Linux pod. Playwright connects to browsers on the Windows machine over CDP, while WebdriverIO and Appium handle native apps through WinAppDriver. A small Go agent captures the desktop with FFmpeg and streams it over WebRTC for live viewing and recordings.

That separation lets a single test move between a browser and a native Windows app without requiring a separate Windows runner pipeline.

The trade-off is that we keep idle Windows VMs ready and install the app under test on every run. In exchange, tests don’t inherit another run’s files, registry changes, or saved sessions.

I’d love to answer any questions!

Reply on news.ycombinator.com