Each GPU is connected with 400Gbps. The rest ist just the normal dataplane which is independent from the GPUs.
Source: Was personally involved in design of that deployment.
18 karma · joined September 3, 2022
We offer a highly efficient and intuitive GPU cloud for scalable machine learning and VFX rendering applications at very affordable prices.
You can reach me via email: thelmich (at) genesiscloud (dot) com
If you try to sell something to me you will be blocked faster than you can imagine.
Source: Was personally involved in design of that deployment.
Genesis Cloud started integration and testing of Gaudi2 quite a while ago. I fully agree with the take of the article.
I can't promise per hour rental, but for longer times they are available! (should you be interested you can find contact details on the website)
Hetzner has excellent connectivity: https://www.hetzner.com/unternehmen/rechenzentrum/ They are always working to increase their connectivity. I'd even go so far to claim that in many parts of the world they outperform certain hyperscalers.
Our lowest at Genesis Cloud at this time are instances with an RTX 3060Ti for 0.20$/hour which adds up to 146$/month ( https://www.genesiscloud.com/pricing#nvidia3060ti ) Though, this includes free storage, no egress fees and has a lot more power than a Jetson.
If you need to optimize for low cost hosting, did you already check whether you actually must have a GPU for your use case? Modern CPU have some impressing capabilities.
Many years ago at university I got to play with a system built by a former student that took wood blocks (children's toys) to build structures on top of a table monitored by kinect cameras. It would then identify features and generate a floor plan. Now imagine combining this! It would allow for a whole new level of exploration of ideas.
Can you share a bit what setup you use to generate the images? Do you run your own GPUs?
Disclaimer: I am the CTO ;)
Why use us?
Competitive prices (billing by the minute, only pay when you actually run an instance). High reliability (professional DCs, customized hardware to suit requirements). Good connectivity (traffic is also free, no in-/egress fees). High security level (full VMs with dedicated GPUs with proper separation of customers instead of shared hosts with docker). Free storage. A great support team. Green energy (no greenwashing by carbon offsetting, we use energy sources that are renewable and carbon free at the source (geothermal/hydro)).
I could go on... Would love it if you just try our services, after sign up there are free credits available for risk free testing.
A quick, very unscientific, test using the oobabooba/text-generation-webui with some models I tried earlier gives me:
* oasst-sft-7-llama-30b (spread over 4x GPU): Output generated in 28.26 seconds (5.77 tokens/s, 163 tokens, context 55, seed 1589698825)
* llama-30b-4bit-128g (only using 1 GPU as it is so small): Output generated in 12.88 seconds (6.29 tokens/s, 81 tokens, context 308, seed 1374806153)
* llama-65b-4bit-128g (only using 2 GPU): Output generated in 33.36 seconds (3.81 tokens/s, 127 tokens, context 94, seed 512503086)
* llama (vanilla, using 4x GPU): Output generated in 5.75 seconds (4.69 tokens/s, 27 tokens, context 160, seed 1561420693)
They all feel fast enough for interactive use. If you do not have an interface that streams the output (so you can see it progressing) it might feel a bit weird if you often have to wait ~30s to get the whole output chunk.
If you want to try it on one reach out to me (email in profile). We rent those out in the cloud. Would allow you to confirm performance before buying one for local use.
Happy to announce that we significantly reduced the prices for our GPU cloud instances.
Let me know what you think!
Tristan from Genesis Cloud here. Please feel free to reach out to me (email is on my profile). Happy to provide some resources for this project.