vanuatu

the subjective framework is exactly why its good

prior bms relied mostly on unit tests or synthetic judges which are easily benchmaxxed, which leads to nobody trusting benchmarks

we need people manually checking the data for good code quality

i worked on one of the benchmarks typically found in new model releases

this benchmark looks very good from the methodology. a cog researcher checking the data themselves is very high signal (not scaleable so don't take the benchmark as gospel, but directionally good)

i doubt it, cog wants coding agents to be better because it directly improves their product

they aren't married to a particular lab, most of their usage is their in house model i believe

there are plenty of american cos that bring in tons of money that are outside the mag7.

vc funded companies pay high so they can grow and eventually bring in lots of money, and america has the deepest vc pockets so it reaps the rewards of the biggest exits

I suspect demand for software is nearly infinitely elastic, so far we’ve seen demand and comp for engineers increase as coding agents got better

Academia for comparison doesn’t make money…maybe a better comparison is HFT? Plenty of very very smart people playing a zero sum game, yet their comp has only increased

feel like that narrative has died given the return of RTO. In person work is really valuable

and the talent is just better in the US on average (mostly because of immigration!), software is so levered one good Eng can 1000x the value of a bad one

American eng comp is commensurate with the money American tech brings in, you could even argue underpaid

For that level of experience you can prob get 200k in a MCOL area in the US, or up to 500k+ in HCOL

The rest of the world has already been in a rude awakening, talented engineers should be compensated well no matter where they happen to live

i suspect this is highly dependent on what you're working on

from my experience if you give the models a way to self-verify correctness they succeed basically 100% of the time

nearly any human desire can be set up as a complex software system to satiate it

that is abysmal!

True on the demand/supply curve

I think due to how leveraged software is, the top % of software developers are more desired (and compensated) than ever, and the bottom % will have difficulty finding a role, and there are structural barriers to entering that top % (intelligence, location, etc). Companies have infinite demand for the cream of the crop talent

it is subject to market forces, but there's no clear ceiling you can draw like copywriting, or textiles, or horses and cars

with abstractions and complexity there's essentially infinite demand for software

i don't think this comment contributes much to the discussion. can you elaborate more than saying "no"?

its not obvious to me that salaries go down, my hunch was that salaries go up but the bar is higher. Software becoming easier to produce (still hard to verify and make useful fwiw) raises the ambitions of software projects, and we don't seem to be close to the ceiling of demand for software systems

Employees who get paid a flat rate per hour don't have the incentive to do more than their job

Equity / profit sharing should be commonplace in the age of AI.

i dont think targeted ads are necessarily bad, and definitely not has bad as the data freely shared vs. behind meta's walls

and what is this 3rd use case?

i think some targeted ads are bad

and social media is probably bad

i care that my data isnt being sold

i dont really care that i get targeted ads, in fact i prefer targeted ads vs. ones that are of no use to me

not to defend zuck but its a common misconception that meta sells data

advertisers dont see the personal data they buy for ad placement

all the labs "clean" their pretraining data, and you can have your pretraining data to be minimally ai generated but also spam synthetic post-training data

I think there's been a recent wave of 'fde-washing' with companies adopting the role (or renaming existing roles) without understanding what made it effective in the first place

Imo, the main responsibilities of the fde role are:

1. get the customer to the business outcome (if the product is insufficient, the fde builds what needs to be built to get the outcome, hence the engineer part)

2. take the learnings back into the product. the expected result of 1) and 2) repeating is your revenue and margin increasing and time to onboard decreasing. if your fdes are building from scratch repeatedly then you're going to be priced like a services business

most companies aren't willing to hire the technical talent that makes a good fde and aren't willing to run their R&D off of their fde team

anti-ai article with ai-isms all over :(

the results are in the revenues of appied ai companies!

i work at a company that automates front and back office workflows (being vague on purpose)

all industries, all are willing to pay 10s of millions even with the tech being somewhat immature

I've seen many cases where AI led to ROI with high margins (maybe not enough to justify the entire industry capex though), but they usually share similar features

- AI is a component of a larger product sold

- The product improves the metrics that customers care about, typically autonomously

- The customer is paying for the outcome, regardless of whether or not the product had AI in it

'Copilot' style AI features are much harder to measure ROI on, because they are typically further away from the base metrics that make it easy to measure ROI, and are typically used for specific tasks in a long web of other tasks within a professional job

Hmm compared to film/entertainment yes, but from the perspective of an individual developer worker, your alternatives are not just in film/entertainment

now compare that margin / growth to big tech or hft...

hiring top junior talent is more competitive than it's ever been!

regular tender offers