You can do the same with vscode btw. I also work on remote gpu workstations and vscode remote with ssh works just fine, especially if you use tmux, you can preserve all the history and what you were doing.
You can look into Copilot Labs extension in vscode, it does the editing and bunch of other stuff too (like explain what the highlighted code does, etc). It's not as smooth, but it's getting there.
I believe the internal server error is because of server load, unrelated to the query itself. I've been using chatgpt since it came out, as it got more viral, it started becoming slower and slower and now, it just randomly gives server errors, hopefully it'll be solved as they scale their systems.
The cutoff is sometime early this year iirc, what you can do is copy the wikipedia article and prompt it saying something like "the following event happened: ..." what do you make of it, etc. It has higher likelihood of working for recent events this way.
For what it's worth, while this code generator might not be great, github copilot is pretty damn good at handling wordpress already (no need to specialize it for a specific thing I guess).
We wrote about ~85% of our relatively simple Wordpress plugin[1] using Copilot with no prior WP knowledge, it took maybe 2 hours at most, it's pretty decent and works well, clients are happy. Just had to write a 3-4 line comments of exactly what we wanted, it got it almost right on the first try, it was able to handle writing and reading configuration from the database as well. The only thing it struggled with was posting binary file content which is understandable.
FWIW, you're not the only one. On our product, we experimented adding login when the user tries to use any of our tools and something like ~60-70% of users dropped off. So essentially, whoever implements that is killing their user funnel at the very beginning which I'm guessing will have larger effects down the road (fewer users experience "magical moments" which leads to less word-of-mouth, etc).
On the flip side though, I'm sure it may help metrics in the short term especially since it's expensive to offer a service like this (which requires expensive GPU servers).
I suspect those applications will come from specializing the model. For example, there's people that have avatar generators or automated ad creatives. A cool application I've been toying with is generating icons.
You would likely be limited by the communication latency between nodes, unless you come up with some unique model architecture or training method. Most of these large scale models are trained on GPUs using very high speed interconnects.
You're probably better of using Real-ESRGAN: https://github.com/xinntao/Real-ESRGAN. It's pretty solid and fast, even has portable executables you can just use as is. The upscaler that comes with stable diffusion might work for you, but I suspect it'll probably do a better job at upscaling stable diffusion output rather than a natural images (might be wrong though).
To put things in perspective, the dataset it's trained on is ~240TB and Stability has over ~4000 Nvidia A100 (which is much faster than a 1080ti). Without those ingredients, you're highly unlikely to get a model that's worth using (it'll produce mostly useless outputs).
That argument also makes little sense when you consider that the model is a couple gigabytes itself, it can't memorize 240TB of data, so it "learned".
There are plenty of 3090s on craigslist as well, fairly common to find below 800$, but 4090s are more power efficient and it's easier to manage a single rig with 7 gpus than trying to network 2 rigs with 14 gpus total. It's also faster for inference without any model change.
For machine learning, self funding my startup and can't afford the expensive server grade gpus. Other countries are usually above msrp + shipping + import tax, would be cheaper to buy from a scalper.
A 8xH100 server sells for like 300k+$ (the A100s go for half, still very expensive), I'd imagine the margins are pretty high on that. The demand for ML models (therefore servers) will likely grow now that startups seem to switch from crypto hype to ai hype and big tech derives huge value from ml workloads.
Not sure how much crypto mining going away affects it though, demand is still there, you just can't buy their cards right now. I've been trying to buy a few rtx 4090s since launch day and it's nowhere to be found.
On the off chance you get to put it in your cart, it either disappears by the time you check out or the order gets voided afterwards. It rarely stays in stock longer than a few mins.
Just set the required/requested gpu number to 0 and k8s will let you use same gpu with multiple pods.
You'll need to make sure to not overload gpu mem, otherwise you'll get oom errors. You can use k8s patch functionality to add custom resources, essentially use that to represent gpu mem and include the amount of gpu the pod uses in its definition, that way you don't go over the limit.
While it's great at compressing the image, it's PITA to work with until browsers support it. What's strange to me is that it's the format Apple uses by default on iOS, but even their own browser doesn't support it (!?). So when a user uploads a picture from an iPhone, you either have to load up an extra JS library (~1MB size last I included one in a web app) to process it or upload it to a remote server for processing.
Honestly, that's what I initially thought trying to run ha postgres on k8s, but zalando's postgres operator made things so much easier (maybe even easier than RDS). Very easy to rollout as many postgres clusters with whatever size you want. We've been running our production db on it for the last 6 months or so, no outage yet. Though I guess if you have to have a very custom setup, it might be more difficult.
Nope, I don't touch anything else. This is essentially telling the GPU, don't use more than this power at all, it'll automagically figure out the best configuration and all that. I've ran those power-limited rigs for 24/7 for many weeks, no issues at all and reduced heat is a nice side effect.
Gaming-wise, I think the 3090 struggles at 8k and 4k 144hz gaming, which is very high end gaming experience and people that spend the money on those monitors will spend the money on the 4090.
For other purposes (like scientific computing/ml), while 3090 can do a lot of things that 4090 does, it's half the speed (or even slower) and a lot less energy efficient, say 1x4090 = 2x3090, 4090 uses 450W, dual 3090 uses 700W, of course you can power limit both of them further, but you still get best bang for your buck out of the 4090.
While I do wish they priced lower, I think when you put things into perspective, it blows my mind that I can buy a card with 76 billion transistors(!) for just 1600$. I suspect the demand for top of the line RTXs will come from enthusiast gamers and ML researchers/startups. Most gamers would be more than fine with 3090 which handles almost anything. The market will be very interesting to watch.
I just used `nvidia-smi -pl 250` on linux, it's built into the driver itself, I'm sure there's equivalent tools (or maybe even nvidia-smi itself) for Windows as well.
For what it's worth, you can power limit them and I'd highly recommend it if you plan on running a few of these. In the past we've power limited RTX 3090s to 250W (100W lower than original) while losing negligible amount of performance.