tempusalaria

What bizarre choice. Hinton’s work has nothing to do with physics.

Nobel prize jumping on the bandwagon, just like they did for mRNA after covid. At least that was related to medicine.

The first 2 paragraphs of the linked pdf read like a joke. Like it’s a parody announcement.

YC cannot do a ton of due diligence. Their strategy is to back many companies with small amounts of money, basically in the hope that a few companies in the batch end up as unicorns. This means they want to select founders capable of getting those outcomes, which will also select many liars/psychopaths due to the need for variance/risk taking behaviors in these founders.

Now is there a problem where YC and many other VCs basically advise founders to commit fraud and lie to customers/stakeholders? Yes but that has nothing to do with YC admissions strategy. I’ve seen many VCs (including YC) advocate for lying and unethical behavior by their founders. It’s a big problem in the startup industry, and often the biggest brunt of this is taken by unsuspecting employees. One reason I would never advise someone to join a startup is the massive prevalence of unethical and dishonest founders.

We live in the future. I just hope we consumers get easy access to these video tools at some point. I want to make personal movies from my favorite books

It’s more than $5bln. That’s their adjusted loss which excludes stock based comp. Real loss is probably 8-10

Don’t agree. Inference costs have been in a race to the bottom for a while and it’s likely imo that OpenAI is gross profit negative on inference right now

There’s definitely a future for LLMs from an enterprise point of view. Even current capability models will be widely used by companies. But it’s seems that will be highly commoditized space, and OpenAI lacks the deep pockets and infrastructure capabilities of Meta and Google to distribute that commodity at the lowest cost.

OpenAI valuation is reliant IMO on them on them 1) AGI possible through NNs, 2) them developing AGI first and 3) it being somewhat hard to replicate. Personally I’d probably stick 10%, 40%, and 10% on those but I’m sure others would have very different opinions or even disagree with my whole premise.

From an API point of view (I.e. developer/corporate usage), I am quite sure that OpenAI is lower usage than Gemini now, and Anthropic API revenue is like 60% of OpenAI based on recent reporting. OpenAI is definitely not dominant in this area.

The aspect of corporate usage where OpenAI seems to be ahead is in direct enterprise subscriptions to ChatGPT.

They took money from ARKK and SoftBank this round which suggests that there were a lot of funds passing.

Facebook grew 100% and had 45% gaap operating margins the year before their IPO.

Facebooks IPO financials were among the best financials at IPO ever

OpenAI has negative 130% adjusted operating margins.

The product OpenAI got mega traction with was based on the GPT series of papers, all of which were published by OpenAI and all of which are major advancements in research.

Also OpenAI was founded before the transformer paper and spent most of their early years focused on RL style research and products - just look at what they published pre 2019.

This seems completely wrong.

A legal wrapper will use domain specific legal expertise to structure and develop their product and functionality. It will likely have many functions that are specifically optimized for the legal use case.

If a wrapper can easily move across many domains, then the wrapper likely adds little value to the base model.

On the one hand, the financials are terrible for an IPO in this market.

On the other, Nvidia is worth 3trn so they can sell a pretty good dream of what success looks like to investors.

Personally I would expect them to get a valuation well about the 4bln from the 2021 round, despite the financials not coming close to justifying it.

What about Imbue which has raised like 300mln, spams fake hiring ads all the time, and doesn’t have a product…

go to smaller pubs. They don’t have the footfall to justify the exorbitant commercial sports license fees and so don’t have screens. Fancier pubs and gastropubs also tend not to have screens

This is wrong.

The term for models that look only at previous tokens in the sequence is auto-regressive.

Encoder and decoder has nothing to do with this.

I think you’re thinking about things very locally. Of course ChatGPT can help with some coding - I use it for regex quite often cause I never really learned that well.

The problem is that at the average medium sized company code looks like this - you have 1mln lines of code written over a decade by a few hundred people. A big portion of the code is redundant, some of it is incomplete, much of it is undocumented. Different companies have different coding styles, different testing approaches, different development dynamics. ChatGPT does not appreciate this context.

Excel has some similar problems. First of all Excel is 2 dimensional and LLMs really don’t think in 2 dimensions well. So you need to flatten the excel file for the LLM. A common approach to do this with LLMs is using pandas and then using the column and row names to index into the excel.

Unfortunately, excels at companies cannot be easily read using pandas. They are illogically structured, have tons of hardcoding, intersheet referencing is weird circular ways and so on. I spent some time in finance and sell side equity research models are written by highly trained financial analysts and are substantially better organized than the average excel model at a company. Even this subset of real world models is far from suitable for a direct pandas interpretation. Parsing sell side models requires a delicate and complex interpretation before being fed into an LLM.

Being able to create 2048 in 6 hours has basically zero economic value.

Can ChatGPT materially and positively impact the code written by big companies? Can it do meaningful work in excel? Can it do meaningful PowerPoint work? Can it give effective advice on management?

Right now we don’t know the answer to those questions. LLM apps can still improve in many ways - better base models, better integration with common enterprise applications, agentic processes, verifiability and so on - so there is definitely hope that there will be significant value created. Companies and people are excited because there’s huge potential. But it is really just potential right now … current systems aren’t creating real enterprise value at this moment in time

One under appreciated dynamic that has changed is that companies with limited revenue/PMF are raising more money at higher valuations. This mean that more early employees are being hired into less certain, less profitable situations, and the equity they are getting is a smaller percentage behind more liquidity preferences.

E.g. There are numerous AI startups I’ve spoken to in the past year with negative gross margins, 8 figure raises, and >100x ARR multiples. $1mln in paper equity in such a company is probably intrinsically worth <$100k.

Being an early employee at a startup right now is a really bad financial decision imo

It’s probable that there is a separate vision encoder which projects the image tiles into the distribution space of the text tokenizer a la CLIP/LLava

HP did limited due diligence due to their incompetence but there was definitely significant fraud at Autonomy.

A number of quant firms are among the largest global consumers of commercial LLMs

The only area you absolutely can’t use LLMs is in sub ms latency

Every opening phrase is a platitude like ‘sure let’s do it’. So the OpenAI latency is probably higher, they are just using clever orchestration to generate some filler tokens to make latency lower. Unlikely the initial response at OpenAI is coming from the main model.

LLMs can easily produce data not in training dataset.

LLMs do not navigate stored data. An LLM is not a DB of the training data.

Temperature has nothing to do with internals. Temperature is purely to do with how the logits outputted by the network are transformed into probabilities, which is completely deterministic and not learned. In fact, temperature makes it impossible for LLMs to simulate this kind of probability. As a calibrated 80-20 split at a certain low temperature would be a different split with some other temperature.

One thing you could think about is the very simple idea of “I’m walking north on park avenue, how do I get into the park”?

The answer is always ‘turn left’ no matter where you are on park avenue. These kind of heuristics would allow you to build more general path finding based on ‘next direction’ prediction.

OpenAI may well have built a lot of synthetic direction data from some maps system like Google maps, which would then heavily train this ‘next direction’ prediction system. Google maps builds a list of smaller direction steps to follow to achieve the larger navigation goal.

The thing is, YC heavily signals against solo founders.

So it seems like there could be a lot of adverse selection in your data where good solo founders don’t bother applying, or good solo founders take on value destructive cofounders just to satisfy YC.

That’s then a vicious cycle where YC sees increasingly lower quality solo founders which reinforces their opinions and data about solo founder outcomes.

That’s my point. Even in massive outlier upside cases getting a 1% equity deal at seed is not really worth more than just being a standard FAANG worker. And that requires many rounds of funding an and dilution and an exit. It’s worth remembering exits are very hard right now and founders have far more liquidity options. There have been multiple unicorns I’m aware of in the past 5 years with founders taking 8 figures in secondary and employees getting nothing. Now those unicorns have worthless equity for employees and the founders are on the beach.

1% at seed stage also become 0.1% after a few rounds of funding, plus you are way down the preference stack and have no liquidity power.

Startup equity is pretty much objectively a terrible deal except for founders. Be a founder or get paid market comp in cash.

Steve Jobs went to an obscure private school called Reed which seems to be something he became obsessed with due to LSD.

Zuck and Gates both dropped out of Harvard after founding their companies.

I was under the impression that this idea of cofounders being a huge statistical success factor was pretty much just a myth.

Moreover, all reasons given by YC as to why you should have a cofounder are qualitative e.g. https://www.ycombinator.com/library/8h-how-to-find-the-right....

Would love to see the statistical evidence because if anything it seems to be the other way round and this is pretty much just reflective of YC’s personal preference as it were (and many VCs prefer otherwise). Personally I would only advise someone to take on a cofounder if they already know them well and need their skill set. There is so much risk in a poor fit that it far outweighs any theoretical benefit.