Looks like the instances they use are not GPU instances, so I’m not sure what the point of this is. Also they do not take into account EC2 spot pricing, which saves a ton of money and are the main reason I use AWS over alternate clouds.
Looks like the instances they use are not GPU instances, so I’m not sure what the point of this is. Also they do not take into account EC2 spot pricing, which saves a ton of money and are the main reason I use AWS over alternate clouds.
https://cloud.google.com/preemptible-vms/
(I work for GCP)
https://aws.amazon.com/blogs/aws/amazon-ec2-update-streamlin...
In fact they have preconfigured Windows instances: https://cloud.google.com/compute/docs/quickstart-windows
Do you have any numbers on that take? Or just a general feel that it works out?
Machine learning stuff will typically happen in batches, openai even went through the effort to publish a k8s library for the task[2] of autoscaling clusters to minimize cost and manage it.
[1]https://aws.amazon.com/ec2/spot/instance-advisor/ [2]https://github.com/openai/kubernetes-ec2-autoscaler
It is ironic that sometimes those pipelines are themselves cheaper to build up than building on fixed cost alternatives. But I don't think it is a given. (I would be surprised, but not shocked, to find I am wrong on that.)
For specific numbers, the AllThePeople.net web scraping and profile compilation runs on Windows m1.small instances. The normal price is 8 cents per hour each while the spot instance costs me only a couple cents per hour each. I set the majority of my spot requests to the minimum possible price. I leave a couple at a higher price point of 8 cents per hour, so at least I have some work getting done if the prices go up. Those numbers are mostly for memory, it’s been a while since I fiddled with it much.