801 karma · joined January 9, 2015
It's good for the engineers involved too. In 5 years, developers not using LLMs for code generation are going to be unemployable.
What is optimized is the ability to find tune medium size models (~200GB) / $
You just can't get 500GB of VRAM for less than $100k. Even with $9k Blackwell cards, you have $10k in a barebones GPU server. You can't use commodity hardware and cluster it because you need fast interconnects. I'm talking 200-400GB/s interconnects. And those take yet another PCIe slot and require expensive Infiniband switches.
Shit gets costly fast. I consternated about this purchase for weeks. Eventually deciding that it's the easiest path to success for my purposes. Not for everyone's, but for mine.
I don't mind paying for models to do things like code. I like to move really fast when I'm coding. But for other things, I just didn't want to spend a week or two coming up on the hardware needed to build a GPU system. You can just order a big GPU box, but it's going to cost you astronomically right now. Building a system with 4-5 PCIE 5.0 x16 slots, enough power, enough pcie lanes... It's a lot to learn. You can't go on PC part picker and just hunt a motherboard with 6 double slots.
This is a machine to let me do some things with local models. My first goal is to run some quantized version of the new V3 model and try to use it for coding tasks.
I expect it will be slow for sure, but I just want to know what it's capable of.
Furthermore, I think we are going to find less and less work for Juniors to do because Seniors are blasting through code at a faster and faster pace now.
I'm not the only one saying that the entry level market is already getting trashed...
Right now, The US and China are in an AI war. The US is doing everything it can to stop China from making progress on AI like it was a nuclear bomb. And it just might be that consequential in 10 years.
Where I am now is past the "3 sleepness nights" of 'Co-Intelligence' fame.
If you haven't seen a properly contexted (50k-100k tokens, depending on the size of the project(s)) LLM work in a code repo, then you have no idea why so many of us are terrified. LLMs are already taking jobs. My company laid off 7% of the workforce because of LLM's impact directly. I say that not because the CEO said it, but because I see it in my day to day. I'm a Principal Engineer and I just don't have need of Juniors anymore. They used to be super useful because they were teachable, and after some training you can offload more and more work to them and free up your time to work on harder problems.
With MCPs, LLMs aren't limited to the editor window anymore. My models update my JIRA tickets for me, rip content from the wiki into it's markdown memory bank which is kept in repo and accelerates everyone's work. It's connecting to databases to find out schemas and example column data. Shit, as I'm typing this it's currently deploying a new version of a container to ECR/ECS/Fargate with terraform for a little project I'm working on.
I believe we are in the very early days of this technology. I believe that over the next ten years, we are going to inundated with new potential for LLMs. It's impossible to foresee all the changes this is going to bring to society. The use cases are going to explode as each tiny new feature or new mode evolves.
My advice is to get off of the sidelines and level up your skills to include LLM integrations. Understand how they work, how to use them effectively, how to program system integrations for them... agents especially! Agents can be highly effective at many use cases. For instance, an Agent that watches a JIRA board for new tickets which contain prompts to be executed in certain repos, then executes the prompt and creates a PR for the changes. All in a context that is fully aware of your environment, deployment, CI/CD, secrets management, etc.
Anything will be possible sooner than we expect. It's going to impact the poorest people the most. A really cyberpunk reality could be upon us faster than we expect, including the starving masses stuggling to get enough to even survive.
I'm considering ordering one of these today: https://www.newegg.com/p/N82E16816139451?Item=N82E1681613945...
It looks like it will hold 5 GPUs with a single slot open for infiniband
Then local models might be lower quality, but it won't be slow! :)
You gotta help me out. What do you see holding it back?
I'm interested in using models for code generation, but I'm not expecting much in that regard.
I'm planning to attempt fine tuning open source models on certain tool sets, especially MCP tools.
Can't wait for it to arrive and crank up LM Studio. It's literally the first install. I'm going to download it with safari.
LM Studio is newish, and it's not a perfect interface yet, but it's fantastic at what it does which is bring local LLMs to the masses w/o them having to know much.
There is another project that people should be aware of: https://github.com/exo-explore/exo
Exo is this radically cool tool that automatically clusters all hosts on your network running Exo and uses their combined GPUs for increased throughput.
Like HPC environments, you are going to need ultra fast interconnects, but it's just IP based.
I'm excited to share that my "Temporal Memory Bank" prompt was just accepted into the official Cline prompts library. I'm incredibly proud of this because it's been a game-changer for how I work with AI, and I hope others find it useful too.
The main idea is to give the AI a deliberate speed bump. I found that even powerful models like Claude 3.7 Sonnet can rush ahead and try to do too much at once, leading to messy results. My prompt is designed to slow things down and enforce a philosophy I call "Baby Steps™".
So, what are Baby Steps™? It's a simple idea: make the smallest possible meaningful change at any one time. Instead of telling the AI "build me a website," I use the memory bank to focus on one, single, well-defined task from a list of known issues.
My workflow now looks like this:
1. __Plan Mode:__ I have the AI read the memory bank and craft a detailed, step-by-step plan to tackle just one issue. We go back and forth until the plan is solid. 2. __Act Mode:__ Once the plan is set, I switch to a more agentic model to execute the steps. The AI's focus is narrow and the goal is clear, which leads to much better outcomes.
It's a way to force a more methodical, human-like approach to problem-solving, and it's made my collaboration with AI far more effective.
A huge shout-out to Nick Baumann ([](https://github.com/nickbaumann98)<https://github.com/nickbaumann98>) for his awesome original work on the memory bank concept, which was the foundation for about 90% of this prompt.
I'm not on Twitter right now but I'm over on bsky:
When I discovered that most online poker sites allow you to export your hand history, I was shook! Over the weekend, I built this site and got it online. It's massively scalable and HA. All the infra is in terraform. Everything is in Alembic. All the code is fairly high performance. And, all of it was vibed coded.
Most important is that all the infra to the best of my knowledge follows all the AWS cloud best practices.
Following a blog from earlier in the year, I discovered the workflow of having GenAI to help me craft my prompts for GenAI. At first, it served as a recommendation engine for options for features I hadn't imagined. (https://harper.blog/2025/02/16/my-llm-codegen-workflow-atm/)
It transformed into a process where I used the prompts and further iteration in Cline's plan mode to build really useful features, testing first, and with all the Markdown documentation I care to generate. (https://cline.bot/)
This meant I was able to move at incredible speed relative to my normal development cycle, and built a reasonably attractive, actually useful site in about 2 days.
Anyways. I don't have friends IRL so I'm sharing with HN
Create an account if you want to try the AI agent. There is no spam and no email confirmation. Here is an example "phh" datafile if you don't have one yourself: https://github.com/uoftcprg/phh-dataset/blob/main/data/pluri...
Things were so much simpler back then.
Opt in systems solve these in open social media platforms, but introduce their own vulnerabilities such as information/reality bubbles and difficulty in growing networks.
> The second is aggregation and discoverability Advertising takes many forms, including word of mouth.
> monetization This is true enough, but most content on the internet isn't monetized by the creator via the platform, and there are many examples of creators who monetize independent of the platform. It also depends on how low you want to go w/ what you call the platform. Some pay for a platform in order to monetize. I don't have stats on this, so maybe it's experiential bias.
This is going to be really really bad isn't it?
I think it's important to realize that this tiny stimulus, whatever it ends up being, $1k, $3k, whatever, is just the sugar to help the massive bailouts go down.
All the companies that took extra profits from the last round of stimulus saved cash and bought back stocks to inflate the stock prices.
The virus was only the needle that popped the good times bubble. Now the market is in free fall, stopping only to bounce off of the branches (circuit breakers) on the way down the cliffside.
How about we accept the individual stimulus and fight against the bailouts this time? These companies aren't too big to fail. They are just too big to be run by profiteering gluttons who fall back on the taxpayer when their business plan is shown to be a failure.