2,439 karma · joined November 27, 2016
I guess there is one thing that Gemini is objectively better at than either, which is long context, and it does seem to be by an order of magnitude. What boggles my mind is why Gemini is still not as good as the open weight frontier models yet. If they got just to parity with that along with their existing long context and strong token pricing, they'd be able to take over the coding market. Are they just biding their time to make their move? Hard to discern.
I'm not saying that they can actually do that per sé; switching costs are so low that if you are doing worse than an existing competitor, you'd lose that volume. Nor am I saying they are deliberately bilking folks -- I think it would be hard to do that without folks cottoning on.
But, I did see an interesting thread on Twitter that had me pondering [1]. Basically, Claude Code experimented with RAG approaches over the simple iterative grep that they now use. The RAG approach was brittle and hard to get right in their words, and just brute forcing it with grep was easier to use effectively. But Cursor took the other approach to make semantic searching work for them, which made me wonder about the intrinsic token economics for both firms. Cursor is incentivized to minimize token usage to increase spread from their fixed seat pricing. But for Claude, iterative grep bloating token usage doesn't harm them and in fact increases gross tokens purchased, so there is no incentive to find a better approach.
I am sure there are many instances of this out there, but it does make me inclined to wonder if it will be economic incentives rather than technical limitations that eventually put an upper limit on closed weight LLM vendors like OpenAI and Claude. Too early to tell for now, IMO.
[1] https://x.com/antoine_chaffin/status/2018069651532787936
My recommendation would be to use other tools built to support pluggable model backends better. If you're looking for a Claude Code alternative, I've been liking OpenCode so far lately, and if you're looking for a Cursor alternative, I've heard great things about Roo/Cline/KiloCode although I personally still just use Continue out of habit.
I've never heard an artist confident in their own ability complain about this because they're not threatened by other competent human artists knocking them off never mind an AI that's even worse at it.
AI not going to out-compete anyone on volume by flooding the marketplace because switching costs are effectively zero. Clever artists can probably find a way to grease controversy and marketing out of finding cases where they are knocked off, taking it as a compliment, and juicing it for marketing.
But I liked the Picasso quote when I was younger and earlier on in my journey as a musician because it reminded me to be humble and resist the desire to get possessive -- if what I was onto was really my own, people would like it and others could try to knock it off and fail. That is a lesson that has always served me very well.
Maybe my biggest disagreement with your view is I think it is simultaneously far too conservative and far too aggressive at the same time. Where there was previously a decision about buy vs build, I disagree with the belief that many tech oriented companies will pull SaaS capabilities inside as GenAI developer tools improve because the cost of "build" is actually not going to get comparatively cheaper compared to the risk of "buying" - if anything, the risk of "building" internally with GenAI tooling that you were considering "buying" is significantly higher unless you are prepared to truly own it, that is go head to head with the entire rest of the market focuses on building and selling that tool as their full-time corporate focus. The actual risk of owning a tool internally making allowances for GenAI tooling is a lot higher than folks realize because GenAI tooling has a lot more risk of creating vibeslop and the only way to avoid that is to be dedicated to producing that tool as one's full time job and serving the needs of the entire market that needs it in that process. This is impossible to do with an internal tool.
The flip side of that same realization is why I also believe that your view is too conservative. The company that might be more empowered to consider building Salesforce internally with better tooling is not competing with Salesforce -- they are competing with the market that is going to /takedown/ Salesforce and a future version of themselves that would've used the future successor to Salesforce. Such a company is probably not in the business of building and selling CRMs although they are likely in the business of using such a CRM. The risk of making that conflation would be to stretch existing resources thin and turn the high interest credit card of tech debt and turn it into a payday loan with GenAI vibeslop.
I do not view GenAI as a democratizer. I view it as an accelerator with the capabilities to accelerate not just "winners take all" dynamics for the top companies that invest enough to avoid vibeslop, but "losers lose all" dynamics in the supply chain for everyone else who tries to wing it to capitalize before inevitably losing against future incumbents, or try in vain to turn a legacy firm into an innovator before inevitably losing against tech debt.
Everyone likes to believe they'll be able to use these great tools to become a future incumbent. But becoming a future incumbent is something that is very hard to do unless your org + book of business + funding + tools is better than someone else's org + book of business + funding + tools. That is why I don't think it actually changes the buy vs build decision that much; the decision should probably still be to buy, it's just changing the question of what exactly should be bought.
When you decide to pull that in house, you are implicitly burdening yourself with the cost of the buildout as well as ongoing maintenance. True, you could probably knock together an okay v1 of CRM yourself inhouse. But are you really going to get it to and maintain production level quality over time at a lower total cost of ownership? I'm skeptical.
The theoretical party you are describing would probably be better served by simply avoiding Salesforce in favor of a next gen CRM that is both more cost effective and easier to customize. In enterprise contexts, even HubSpot is effectively next gen, but there are also products like Attio et al that have a ton of adoption and strong integration ecosystems (albeit not at the Salesforce level).
When you buy enterprise software from a vendor you are buying more than "just software" you are also hiring a company's services. And the inverse is correct as well, when you choose to build it in house, you are implicitly choosing to hire a team internally to resource all of the services you would've expected that vendor to provide.
Certainly, this tradeoff can still make a lot of sense for some companies. The acid test for that, in my opinion, is whether said company could (and would) actually successfully sell the product they build internally on the open market. If the answer to that is "yes", the prospect of turning a cost center into a profit center can potentially bear significant long term ROI to the company.
...no, I haven't lost the plot. I'm seeing another fad of the intoxicated parting with their money bending a useful tool into a golden hammer of a caricature. I dread seeing the eventual wreckage and self-realization from the inevitable hangover.
I can't see it making sense for training workloads if and when I get to them (which I'd put on the cloud). I have a box with a single 3090 to do CUDA dev if I need to but I haven't needed to do it that often. And frankly the Mac Studio has rough computational parity with a bit under a 3090 in terms of grunt, but with an order of magnitude more unified VRAM so it hits the mark for medium-ish MoE models I like to run locally as well as some of the diffusion inference workloads.
Anything that doesn't work great locally or which is throwaway (but needs to be fast) ends up getting thrown at the cloud. I pull it back to something I can run locally once I'm running it over and over again on a recurring basis.
With LLM workloads, you can run some of the larger local models (at all) and you can run them cheap on the unified 128G RAM machines (Strix Halo/Spark) - for example, gpt-oss-120b. At 4bit quantization given it's an MoE that's natively trained at NVFP4, it'll be pretty quick. Some of the other MoEs with highly compressed active parameter models will also be quick as well. But things will get sluggish as the active parameters increase. The best way to run these models is with a multi-GPU rig so you get speed and VRAM density at once, but that's expensive.
With other workloads such as image/video generation, the unified vram doesn't help as much and the operations themselves intrinsically run better on the beefier GPU cores, in part because many of the models are relatively small compared to LLM (6B-20B active parameters) but generating from those parameters is definitely GPU compute intensive. So you get infinitely more from a 3090 (maybe even a slightly lesser card) than you do from a unified memory rig.
If you're running a mixture of LLM and image/video generation workloads, there is no easy answer. Some folks on a budget opt for a unified memory machine with an eGPU to get the best of both worlds, but I hear drivers are an issue. Some folks use the Mac studios which while quite fast force you to be inside the Metal ecosystem rather than CUDA and aren't as pleasant for dev or user ecosystem. Some folks build a multi CPU server rig with a ton of vanilla RAM (used to be popular for folks who wanted to run DeepSeek before RAM prices spiked). Some folks buy older servers with VRAM dense but dated cards (thing Pascal, Volta, etc, or AMD MI50/100). There's no free lunch with any of these options, honestly.
If you don't have a very clear sense of something you can buy that you won't regret, it's hard to go wrong using any of the cloud GPU hyperscalers (Runpod, Modal, Northflank, etc) or something like Fal or Replicate where you can try out the open source models and pay per request. Sure, you'll spend a bit more on unit costs, but it'll force you to figure out if you have your workloads figured out enough to where the pain of having it in the cloud stings enough to where you want to buy and own the metal -- if the answer is no, even if you could afford it, you'll often be most happiest just using the right cloud service!
Ask me how I figured out all of the above the hard way...
Ironically, all the scheming I did about overclocking ended up being very unnecessary and I found it borderline impossible to actually stress the upper limits of the machine's muscle with day to day workloads and so all the research I put into overclocking was not really practically necessary, that it was freeing to not have to even think about the machine and instead focus on the work I wanted to do with the software I was using and building. Surely a lesson that continues to pay dividends, albeit from simpler times...
Many if not most professional producers use MIDI controllers with knobs/sliders/buttons MIDI mapped to DAW controls. As such the skeuomorphism actually plays a valuable role in ensuring that the physical instrument experience maps to their workflows. Secondarily, during production/mastering, producers are generally using automation lanes and envelopes to program parameters into the timeline, and the piano roll to polish the actual notes.
When I've historically done working sessions, the composition phase of what I'm doing tends to involve very little interaction with the keyboard, and is almost entirely driven by my interaction with the MIDI controller.
Conversely, when I'm at the production phase, I am generally not futzing with around with either knobs or the controller, and I am entirely interacting with the DAW through automation lanes or drawing in notes through the piano roll. So I don't really ever use the knob through a mouse and I've never really encountered any professional or even hobbyist musicians who do except for throwaway experimentation purposes.
```
Limitations
I haven't, yet, found a good way to implement filters (low-pass, high-pass, band-pass, etc.). It does not have Fourier transform, and we cannot operate on the frequency domain. However, the moving average can suffice as a simple filter.
```
I wonder if there's a way to implement the FFT using subqueries and OVER/partitioning? That would create a lot of room for some interesting stuff to happen, specifically making it easy/possible to implement filters, compression, reverberation, and other kinds of effects.
Two other primitives that would be valuable to figure out: 1. How to implement FM/phase distortion. You can basically implement a whole universe of physical modeling if you get basic 6 op sine wave primitives right with FM + envelopes. 2. Sampling/resampling - given clickhouse should do quite well with storing raw sample data, being able to sample/resample opens up a whole world of wavetable synthesis algorithms, as well as vocal editing, etc.
Honestly, although the repo's approach is basic, I think the overall approach is wonderful and have wanted to be able to use SQL to write music for a while. I've spent a lot of time writing music in trackers, and being able to use SQL feels like it would be one of the few things that could spiritually be a successor to it. I've looked at other live coding languages, many of which are well built and have been used by talented people to make good music (such as Tidal, Strudel, etc). But all of it seems like a step down in language from SQL. I'd rather have their capabilities accessible from SQL than have to use another language and its limitations just to get those capabilities.
Food for thought -- thanks for the interesting and thoughtful work!
The difference I generally see with shared-state microservices is that now you introduce a network call (although you have a singular master for your OLTP state), and with isolated state microservices, now you are running into multiple store synchronization issues and conflict resolution. Those tradeoffs are very painful to make and borderline questionable to me without a really good reason to sacrifice them (reasons I rarely see but can't in good faith say never happen).
Pertaining to your IoT example -- that's definitely a spot where I see a reason to move out of the cozy RDBMS, which is an access pattern that is predominated by reads and writes of temporal data in a homogenous row layout and seemingly little to no updates -- a great use case for a columnar store such as Clickhouse. I've resisted moving onto it at $MYCORP because of the aforementioned paranoia about losing RDBMS niceties (and our data isn't really large enough for vertical scaling to not just work) but I could see that being different if our data got a lot larger a lot more quickly.
Maybe putting it together, there are maybe only several reasons I've seen where microservices are genuinely the right tool for a specific job (and which create value even with shared state/distributed monolith):
1) [Shared state] Polyglot implementation -- this is the most obvious one that's given leverage for me at other orgs; being able to have what's functionally a distributed monolith allows you to use multiple ecosystems at little to no ongoing cost of maintenance. This need doesn't happen for me that often given I often work in the Python ecosystem (so being able to drop down into cython, numba, etc is always an option for speed and the ecosystem is massive on its own), but at previous orgs, spinning up a service to make use of the Java ecosystem was a huge win for the org over being stuck in the original ecosystem. Aside from that, being able to deploy frontend and backend separately is probably the simplest and most useful variant of this that I've used just about everywhere (given I've mostly worked at shops that ship SPAs).
2) [Shared state] SDLC velocity -- as a monolith grows it just gets plain heavy to check out a large repository, set up the environment run tests, and have that occur over and over again in CI. Being able to know that a build recipe and test suite for just a subset of the codebase needs to occur can really create order of magnitude speed ups in wall to wall CI time which in my experience tends to be the speed limit for how quickly teams can ship code.
3) [Multi-store] Specialized access patterns at scale -- there really are certain workloads that don't play that well with RDBMS in a performant and simple way unless you take on significant ongoing maintenance burden -- two I can think of off the top of my head are large OLAP workloads and search/vector database workloads; no real way of getting around needing to use something like ElasticSearch when Postgres FTS won't cut it, and maybe no way around using something like Clickhouse for big temporal queries when it would be 10x more expensive and brittle to use postgres for it; even so, these still feel more like "multiple singleton stores" rather than "one store per service"
4) [Multi-store] Independent services aligned with separate lines of revenue -- this is probably the best case I can think of for microservices from a first principles level. Does the service stand on its own as a separate product and line of revenue from the rest of the codebase, and is it actually sold by and operated by the business that independently? If so, it really is and should be its own "company" inside a company and it makes sense for it to have the autonomy and independence to consume its upstream dependencies (and expose dependencies to its downstream) however it sees fit. When I was at AWS, this was a blaringly obvious justification, and one that made a lot of intuitive sense to me given that so much of the good stuff that Amazon builds to use internally is also built to be sold externally.
5 [Multi-store] Mechanism to enforce hygiene and accountability around organizational divisions of labor - to me, this feels like the most questionable and yet most common variant that I often see. Microservices are still sexy and have the allure of creating high visibility career advancing project track records for ambitious engineers, even if at the detriment to the good of the company they work for. Microservices can be used as a bureaucratic mechanism to enforce accountability and ownership of one part of the codebase to a specific team to prevent the illegibility of tragedy of the commons -- but ultimately, I've often found that the forces and challenges that lead to those original tragedy of the commons are not actually solved any better in the move to microservices and if anything the cost of solving it is actually increased.
The main time I can see this making sense is when the data access patterns are so different in scale and frequency that they're optimizing for different things that cause resource contention, but even then, my question would become do you really need a separate instance of the same kind of DB inside the service, or do you need another global replica/a new instance of a new but different kind of DB (for example Clickhouse if you've been running Postgres and now need efficient OLAP on large columnar data).
Once you get to this scale, I can see the idea of cell-based architecture [1] making sense -- but even at this point, you're really looking at a multi-dimensionally sharded global persistence store where each cell is functionally isolated for a single slice of routing space. This makes me question the value of microservices with state bound to the service writ large and I can't really think of a good use case for it.
[1] https://docs.aws.amazon.com/wellarchitected/latest/reducing-...