cornel_io

I don't think that's really true in practice, though at a competence level it absolutely is. Except at the biggest places, even low level programmers are both empowered and expected to make architectural decisions that have big impact on maintainability, testability, and performance. Addition of a bad-but-fast coder to a team is the worst possible outcome because they are really productive at screwing up the whole codebase in ways that will take even more time to peel apart. It's actually worse if their code is in the "functional but ugly" category, since PMs love those people as long as they're really fast, and they hate when other engineers complain about their work.

In construction, the random guy pulled in to swing a hammer may cut something wrong once or twice and waste a little time and money, but he's never going to be trusted to design the way the support beams hold up the whole house, so the damage is very limited. Software is not like that, we expect everyone to do some amount of design and engineering, whether or not they have any ability to do so. If as much software dev was driven by very low-level grunt work as is the case in construction, LLMs would already have revolutionized the field a lot more than they have; as it is, we've probably got another couple years to wait.

That sounds like the correct response if he knows anything about the field: shitty peripherals are an unsolvable problem from top to bottom. The hardware sucks, the software sucks more, there's absolutely nothing you can do to fix it as an end user (unplug it and try again is where "debugging" ends), and the production chain will never improve along the quality axis because the margins are tiny and people are very price sensitive.

That reformulation doesn't let you avoid renormalization, does it?

Sometimes the content of a cell in e.g. a CSV file will be another CSV file. This inevitably happens with pretty much every format, whether CSV, TSV, JSON, etc, so you'll need some sort of quoting rule no matter what you do.

People have been convicted of hacking for merely editing URL strings, under the theory that were knowingly accessing systems in ways that they were not supposed to. This would be similar.

Whether or not that seems reasonable to us is a different matter, but basically it boils down to the fact that "they left the door unlocked" doesn't make it legal to walk in.

Yes, Lina Khan has been very clear that she wants to abandon the previous guidelines that made consumer harm the focal point of anti-trust cases.

I think that's pretty shady and in several of these cases she really just wants to spear any old Big Fucking Whale so it looks like she's been successful. That said, "consumer" really does mean other companies when it comes to Google ads, since they directly pay them a TON of money, so if they can prove that Google's monopoly is distorting the market I think this is fair game, 100%. And the ad market has been incredibly shitty for a very long time because if you don't have an audience in the billions you can't even try to compete, so anything that can open that up to some innovation would be positive, and price competition would allow a lot of products to exist and thrive that otherwise would have failed because cost of user acquisition was too high. That is very good for consumers.

Or it could be really negative, depending how they implement it. The funny thing is, despite the company fighting against it strongly, it's uncertain how good or bad it would be for Google shareholders, oftentimes breakups are very good for their stock because as each piece of the company is unshackled from the others, things that would be profitable for the division but get blocked by other interests at the company all of a sudden become possible: IIRC the combined value of the Baby Bells ended up far greater a few years after the breakup, for one example. The only people it's guaranteed to be bad for are the people at the top at Google, who would have their kingdoms shredded up and taken away from them.

Justice that is not bound by the letter of the law is not just at all.

Leave. You can always hire them/work with them again later (chain hiring of teams is a very common thing even if you're not at a high level), but if you burn out while you're alongside them they might shift their opinion of you...

9 years is a lot to expect it to be under warranty...

I've actually had to fire a technically exceptional Haskell programmer because of the damage they did to our C# codebase (and arguably moreso, the team). Sometimes it's not a matter of talent or skill, but culture fit.

In my experience FP-aligned people on non-FP projects tend to be more likely to overengineer, more prone to argue in favor of the Great Rewrite For No Reason Except Aesthetics, and more likely to abuse "lesser" programmers when they put up PRs. They suck as team players on teams that are not made of language nerds. I am not just talking about the one person here who I fired, this is a legit pattern I've noticed over at least a half dozen people.

Conversely, they are exactly the right people to deploy when you have really tough, self-contained problems to solve that you wouldn't trust the normal Java 9-5ers to tackle.

No matter how they do it, you can always rewrite their working code in a more maintainable language later once it's working, and make it integrate well with the rest of your stack. :D

Simple solution: if you ever run into this, blog about it and mark it as a freak occurrence, because it is. Don't ever in real work worry about stupid edge cases like this or you're wasting your employer's money. I'd reprimand an engineer for spending more than an hour on this, it serves nobody unless you're working on extremely widely used library code.

It depends heavily on location. When I last ended up on Cobra in California it was a touch more expensive than the marketplace plan but my work plan was much better than the best available on the marketplace so it was worth it to go Cobra.

Flagged because the link is wrong...

Different strokes: I prefer it in the PR, with a lot of detail, with commits that are very granular and just explain what they do. PR descriptions should have all the why, test cases, pictures, etc.

Fair enough.

I'm never talking to anyone over insecure channels again.

Google has great lawyers. They don't flinch or buckle.

Yes. The DOJ sometimes (if not often) overreaches because they are taking a shot. These sorts of cases are often attempts to establish precedent that they can use to go after others under similar theories.

Flipping this around, do you think Google's legal team, which is probably equally qualified and vastly more well paid (and with a lot more to lose if they're wrong), would advise employees to do this if they hadn't considered such trivialities?

Right, but I'm not clear, is that actually illegal? I've seen similar policies at many companies.

I use Signal by default for every piece of communication that I can. Am I somehow held to a different standard just because I was paranoid about surveillance from the start?

That seems wrong.

Right on: I've used a single Postgres database on AWS to handle 1M+ concurrent users. If you're Google, sure, not gonna cut it, but for most people these things scale vertically a lot further than you'd expect (especially if, like me, you grew up in the pre-SSD days and couldn't get hundreds of gigs of RAM on a cloud instance).

Even when you do pass that point, you can often shard to achieve horizontal scalability to at least some degree, since the real heavy lifting is usually easy to break out on a per-user basis. Some apps won't permit that (if you've got cross-user joins then it's going to be a bit of a headache), but at that point you've at least earned the right to start building up a more complex stack and complicating your queries to let things grow horizontally.

Horizontal scaling is a huge headache, any way you cut it, and TBH going with something like Spanner is just as much of a headache because you have to understand its limitations extremely well if you want it to scale. It doesn't just magically make all your SQL infinitely scalable, things that are hard to shard are typically also hard to make fast on Spanner. What it's really good at is taking an app with huge traffic where a) all the hot queries would be easy to shard, but b) you don't want the complexity of adding sharding logic (+re-sharding, migration, failure handling, etc), and c) the tough to shard queries are low frequency enough that you don't really care if they're slow (I guess also d) you don't care that it's hella expensive compared to a normal Postgres or MySQL box). You still need to understand a lot more than when using a normal DB, but it can add a lot of value in those cases.

Are you under the impression that he was talking about game clients doing this? That would be absurd, since no gamers (within epsilon of zero) use Linux, and you'd need a completely different bag of tricks on Windows.

He's fully focused on the backend.

Most human authors are frankly far too stupid to be worth reading, even if they do put care into their work.

This, IMO, is the actual biggest problem with LLMs training on whatever the biggest text corpus us that's available: they don't account for the fact that not all text is equally worthy of next-token-predicting. This problem is completely solvable, almost trivially so, but I haven't seen anyone publicly describe a (scaled, in production) solution yet.

Game engines are a terrible business, so I don't know. I'm surprised investors were interested, TBH, as interesting a project as it is.

The fundamental problem with game engines is that the bulk of the users are tiny to small indies, who will complain endlessly about any fee whatsoever and even if it's the best tool ever would never pay more than a couple hundred per year. You'll be lucky to get even a few hundred to pay that. On the other end, BigCo game companies, if they'd ever by some miracle greenlight the use of your product rather than using their own shitty internal engines or Unity/UE, are so few in number that even if you can sign one or two you've got a tiny total addressable market.

Probably a fine business, if it works, for a solo bootstrapped engineer, but beyond that it gets very difficult. I'd love to be proven wrong because it'd be fantastic to see more competition in the space (Unity and Unreal both frankly suck balls, largely because they each essentially monopolize their niches and have no incentive to compete).

There are a few types of acquisition:

- everyone gets rich (including employees with equity)

- founders and investors get rich, nobody else gets shit

- investors get good return, founders get a bit, nobody else gets shit

- investors make mediocre return based on liquidation pref, nobody else gets shit

- nobody gets shit (but people keep their jobs in the acquirer)

In all but the first situation, you as an employee (even a high-up one) are basically being sold to another company without any upside, and whether or not you want to stay will depend 100% what you think about the acquirer.

Yes, the GRE has been way ahead of the curve for a long time because it's an adaptive test: it actually adjusts the difficulty of the questions it presents you based on your performance up until that point in the test, as it tries to hone in in exactly what your correct score should be.

I don't know if the algorithm has changed since the early aughts, and I don't know if that approach is better or worse than the one-size-fits-all non-adaptive SAT, but the difference was always striking to me. You'd think that if the GRE adaptive approach was better statistically (by whatever measure) they'd move towards it soon for the SAT as well since they're going full digital finally.

"Next token predictor" isn't quite the burn that it seems like, because perfect next token prediction would require actual understanding. That's because you can almost always cast any question about understanding into a form where it depends solely on the next token (there are a couple nitpicky exceptions and caveats but not many).

GPT 4 is at a high enough level of performance that mere simple statistics aren't really helping it do any better, it really is developing structures especially in the middle layers that perform some amount of high level understanding.

I don't think that pure next token prediction will always be the optimal way to train and enhance these behaviors, but it's not fair to say that it's unrelated, if this really was just stochastic parroting then LLMs would have topped out way before the level they're at now.

I think he's trying to bring up the racial image gen bias stuff that's going on with Gemini, but for some reason won't say it. He also doesn't appear to realize there is a difference between the two products/teams/launches...

When I use the BQ interface, it estimates the bytes for each query in real time before I run it, does that turn off if the query is too big? I guess that isn't directly a cost estimate, but if I saw hundreds of TB I'd think twice before hitting "Run"...

Sure, but Google provides super helpful responses when doing the exact same search, and as an ex-sysadmin I know exactly what they're asking and could answer perfectly easily.

A ton of us out here don't agree with your goals. I think these models are transformative enough that the value added by organizing and extracting patterns from the data outweighs the interests of the extremely diffuse set of copyright holders whose data was ingested. So regardless of the technical details of copyright law (which I still think are firmly in favor of OpenAI et al) I would strongly opposed any effort to tighten a legal noose here.