An AMD AI Pro R9700 32GB brand new is $1350 right now.
After some tweaking, I had it running faster than the models the 3090 could run, and it could obviously run with higher context limits and bigger models due to the extra vram.
1,222 karma · joined October 12, 2021
An AMD AI Pro R9700 32GB brand new is $1350 right now.
After some tweaking, I had it running faster than the models the 3090 could run, and it could obviously run with higher context limits and bigger models due to the extra vram.
That was the last major thing the Democrats did, and healthcare has gotten substantially worse...but at least it's well regulated now.
If you can self host it that easily, any Joe can scale it out much like shared web hosting, and shared web hosting or even dedicated rented boxes has always been cheaper than the big cloud providers.
I don't think OpenAI or Anthropic can reasonable compete in the long term if they can't achieve "AGI", and they won't, no matter what shareholders desire.
In terms of electricity, if you aren't using it, even with all the vram loaded, at most your wasting about 30 watts or so.
Prompt processing a large uncached context is annoying, which is why I forced a lower context window, but I don't know if it's any worse in performance than the cloud models I've used.
There's a niceness, to me, knowing I don't have to rent it anymore. If you rent it, the terms can change regularly.
In other words, once people got on it, it was too late.
I'll probably try to figure that problem out in about a month. Worst case is I move it to another even older desktop to replace the 9800 GTX+ inside of that one.
It was suspected to come soon enough, but it was a nice cheap road for my small hobby stuff. When they announced the price changes, I started to explore alternatives, and with the news of Qwen3.6 35B being both and having quality, I figured it was worth a try out, and self-hosting made the most sense to me, since that meant I was free from being a forever-renter.
I can easily run Qwen3.6 35B-A3B with Q5_K_M with a 260k+ context window with some vram to spare. It easily runs probably 80tps. It took me quite a while to find the
Compared to GHCP Claude Sonnet 4.5 or 4.6, I have full parity. The wall clock time is faster for agentic workflows, and rule following is about on par.
With either, doing something kind of novel or obscure takes more hand holding compared to just generate a GUI or crud app. For example, trying to build an actual program that performs a complicated process correctly requires quite a bit of hand holding to get it to properly help.
Sure, it isn't Opus or something, but I think with the right harness, it probably can get close. I think most of the issues these days is the harnesses are lacking.
I admit I know nothing about this though.
But that was my testing strategy already, but writing one of those tests could take legitimately hours with how many columns and crazy rules our area has.
I guess that's why - because extortion is immoral. Eventually, those paying the protection fees will eventually find someone better to pay or will stop paying when the protection doesn't show up.
I still see that better than YAML's indentation problem. Relying solely on indentation is a nightmare.
Having said that, I fear what June 1st brings for copilot It might suddenly be very useless for me.
<project>MAML</>
<tags>
<item>minimal</>
<item>readable</>
</>
<!-- A simple nested object -->
<spec>
<version>1</>
<author>Anton Medvedev</>
</>
<!-- Array of objects -->
<examples>
<item>
<name>JSON</>
<born>2001</>
</>
<item>
<name>MAML</>
<born>2025</>
</>
</>
<notes>
This is a multiline raw strings.
Keeps formatting as-is.
</>If anything, wouldn't a phone extension be more sandboxed than most desktop environments?
Gives you more control over the outcome and more steering anyway.
It might keep up with Sonnet 4.5 with some tinkering.
But long story short: it seems to have better performance and similar quality for a payoff of a year or so compared to cloud models. In the same way you can self host faster/easier/cheaper than cloud hosting, if you are okay with the negatives.
I'm returning my 3090 soon for a R9700 after some more basic benchmarking, since the higher RAM should improve my observations more.
Then COVID happened and employers didn't have 2-4 months of savings built up, and ended up shuttering due to lack of money immediately.
Also, since post COVID, we've had hyper inflation and a locked up housing market. $150 townhomes that a $50 family salary could afford are now going for $300. And rent has gone up to match.
I don't think the numbers not matching is because of anyone's personal financial responsibility. It's more from the Fed's and Congress's horrible actions over the past 2 decades and their financial irresponsibility.
Granted, grumbling about the powers that be doesn't solve the problem, which is why I fear civil turmoil will be here very soon.
Then there's roads with just the center line markers with no road should markings.
Then there's a whole class of roads of lines over "demarked" old lines that weren't demarked well, or lines fading that should've been painted a long time ago.
I'm surprised you've never seen a non-perfect road?
But that gives me a good while to determine if it's worth it or not. I've heard good and bad, so here's hoping for good or close to it.
I wasn't going to fork out $1000 on a chance it might be enough with a rough return strategy.
I think competition will get fierce. We see many people are attracted to the price stability of GHCP - it became clear what a request could do - the problem is that they didn't match results with cost. It's not clear what a 5 hour usage window in Claude Code can do.
There's no reason the harness couldn't provide a quote on the next request, aside from it takes effort and it would be upfront to the user, creating expectations.
But that would entail actual work and effort...and care for user's time and money.
If they wanted to limit what a request can do via their harness, I'm sure they artificially could.
I hate all of the other plans I've seen of here a "credit" or here's a "bucket of usage", and we pull an announced amount from it based on arbitrary info that can't be audited or proven, and most of shat is spent might be entirely useless anyway.
Claude Code has a problem where 1 request could take a significant portion of your 5 hour window, and it's unclear why.
It's much like SEO, where Google sometimes says things that might help, but it's just magic wand eaving hoping something works.