gdb

> My "guess" is they're offering $1B worth of Azure services. Which costs MSFT probably much less than $1B.

It's a cash investment. We certainly do plan to be a big Azure customer though.

> My "guess" is that it means MSFT has access to sell products based off the research OpenAI does to MSFT's customers. Having early access to advanced research means MSFT could easily make this money back by selling better AI tools to their customers.

I'm flattered that you think our research is that valuable! (As I say in the blog post: we intend to license some of our pre-AGI technologies, with Microsoft becoming our preferred partner for commercializing them.)

Thank you!

The OpenAI board remains in charge of OpenAI’s AGI-relevant decisions. Microsoft has the right to appoint one board seat, which they have not yet exercised.

(I work at OpenAI.)

Quite the opposite — this is an investment!

We'll be playing generated MIDI without human filtering or manipulation today (though playing them synthetically of course)! If you know of an orchestra looking for some music, we could do the rest of what you describe :).

Not yet, but we plan to release the code & pre-trained model!

Tool is embedded in the blog post: https://openai.com/blog/musenet/#try!

Thanks for the thoughtful reply, it's much appreciated.

> you also want the mission statement for people that aren't motivated by money

I wouldn't agree with this — we want people who are motivated by AGI going well, and don't want to capture all of its unprecedentedly large value for themselves. We think it's a strong point that OpenAI LP aligns individuals' success with success of the mission (and if the two conflict, the mission wins).

We also think it's very important not to release technology we think might be harmful, as we wrote in the Charter: https://openai.com/charter/#cooperativeorientation. There was a polarized response, but I'd rather err on the side of caution.

Would love people who think like that to apply: https://openai.com/jobs

Any returns from OpenAI LP are subject to taxes!

(I work at OpenAI.)

I think this tweet from one of our employees sums it up well:

https://twitter.com/Miles_Brundage/status/110519043405200588...

Why are we making this move? Our mission is to ensure AGI benefits all of humanity, and our primary approach to doing this is to actually try building safe AGI. We need to raise billions of dollars to do this, and needed a structure like OpenAI LP to attract that kind of investment while staying true to the mission.

If we succeed, the return will be exceed the cap by orders of magnitude. See https://blog.gregbrockman.com/the-openai-mission for more details on how we think about the mission.

Our mission is articulated here and does not change: https://openai.com/charter/. As we say in the Charter, our primary means of accomplishing the mission is to build safe AGI ourselves. That means raising billions of dollars, without which the Nonprofit will fail at its mission. That's a huge amount of money and not something we could raise without changing structure.

Regardless of structure, it's worth humanity making this kind of investment because building safe AGI can return orders of magnitude more value than any company has to date. See one possible AGI application in this post: https://blog.gregbrockman.com/the-openai-mission#the-impact-...

The Nonprofit has full control, in a legally binding way: https://openai.com/blog/openai-lp/#themissioncomesfirst

(I work at OpenAI.)

See my tweet about this: https://twitter.com/gdb/status/1105173883378851846

> If the partner gets a vote/profit then a "charter" or "mission" won't change anything

(I work at OpenAI.)

The board of OpenAI Nonprofit retains full control. Investors don't get a vote. Some investors may be on the board, but: (a) only a minority of the board are allowed to have a stake in OpenAI LP, and (b) anyone with a stake can't vote in decisions that may conflict with the mission: https://openai.com/blog/openai-lp/#themissioncomesfirst

The Charter was designed to capture our values, as we thought about how we'd create a structure that allows us to raise more money while staying true to our mission.

Some companies to compare with:

- Stripe Series A was $100M post-money (https://www.crunchbase.com/funding_round/stripe-series-a--a3... Series E was $22.5B post-money (https://www.crunchbase.com/funding_round/stripe-series-e--d0...) — over a 200x return to date

- Slack Series C was $220M (https://blogs.wsj.com/venturecapital/2014/04/25/slack-raises...) and now is filing to go public at $10B (https://www.ccn.com/slack-ipo-heres-how-much-this-silicon-va...) — over 45x return to date

> you don't want to "unduly concentrate power"? How will this work?

Any value in excess of the cap created by OpenAI LP is owned by the Nonprofit, whose mission is to benefit all of humanity. This could be in the form of services (see https://blog.gregbrockman.com/the-openai-mission#the-impact-... for an example) or even making direct distributions to the world.

The nonprofit board retains full control, and can only take actions that will further our mission.

As described in our Charter (https://openai.com/charter/): that mission is to ensure that AGI benefits all of humanity.

Yes, OpenAI Nonprofit is a 501(c)(3) organization. Its mission is to ensure that artificial general intelligence benefits all of humanity. See our Charter for details: https://openai.com/charter/.

The Nonprofit would fail at this mission without raising billions of dollars, which is why we have designed this structure. If we succeed, we believe we'll create orders of magnitude more value than any existing company — in which case all but a fraction is returned to the world.

We believe that if we do create AGI, we'll create orders of magnitude more value than any existing company.

We needed to custom-write rules like:

- Fiduciary duty to the charter - Capped returns - Full control to OpenAI Nonprofit

LP's have much more flexibility to write these in an enforceable way.

No.

Yes, we're planning to release a third-party usable reference version of our docs (creating this structure was a lot of work, probably about 6-9 months of implementation).

We've made the equity grants feel very similar to startup equity — you are granted a certain number of "units" which vest over time, and more units will be issued as other join employees in the future. Incidentally, these end up being taxed more favorably than options, so we think this model may be useful for startups for that reason too.

We've been working on this new structure together for the past two years!

(I gave the talk.)

> I've likely lost any shot I had at contributing in areas that will advance the state of the art since I graduated college 20 years ago.

For a bit of optimism: if you are a good software engineer, you can become a contributor in a modern AI research lab like OpenAI. We hire software engineers to do pure software engineering, and can also train them to do the machine learning themselves (cf https://blog.openai.com/spinning-up-in-deep-rl/ or our Fellows and Scholars programs).

As one concrete example, though not the norm, our Dota team has no machine learning PhDs (though about half had prior modern ML experience)!

(I gave the talk.)

We are already starting to see the nature of data changing. Unsupervised learning is starting to work — see https://blog.openai.com/language-unsupervised/ which learns from 7,000 books and then sets state-of-the-art across almost all relevant NLP datasets. With reinforcement learning, as you point out, you turn simulator compute into data. So even with today's models, it seems that the data bottleneck is much less significant than even two years ago.

The harder bottleneck is transfer. In most cases, we train a model on one domain at a time, and it can't use that knowledge for a new related task. To scale to the real world, we'll need to construct models that have "world knowledge" and are able to apply it to new situations.

Fortunately, we have lots of ideas about how this might work (e.g. using generative models to learn a world model, or applying energy-based models like https://blog.openai.com/learning-concepts-with-energy-functi...). The main limitation right now: the ideas are very computationally expensive. So we'll need engineers and researchers to help us to continue scaling our supercomputing clusters and build working systems to test our ideas.

(I work at OpenAI.)

The code is linked from the top of the blog post: https://github.com/openai/random-network-distillation.

While we do produce lots of open-source code, our mandate is much broader than that: https://blog.openai.com/openai-charter/. We are attempting to build safe AGI and distribute its benefits.

In the short term, that means we focus on building working systems and will sometimes, but not always, release source code.

In the long term, per the charter: "we expect that safety and security concerns will reduce our traditional publishing in the future, while increasing the importance of sharing safety, policy, and standards research."

> The way I would guess this played out was that OpenAI tried multiple variants of the game, and this version ended up being good enough to beat human players

No. We had implemented scripted courier logic for 1v1, and when switching to 5v5 the easiest starting point was to run five copies of that logic. (Dota's Turbo mode also has five invulnerable couriers.) In June, we had more important restrictions to remove — such as wards & Roshan; you personally were focused on the particular heroes we'd chosen: https://news.ycombinator.com/item?id=17392455. The couriers only become our most important restriction in August.

The item purchasing is still scripted, which means that Five receives wards whether it wants them or not. One explanation for the ward dumping is that Five is just trying to free up inventory space.

We estimate it based on when we start being evenly-matched against teams with a given average MMR. (As you might expect, our usual pattern is to lose to a test team consistently, then start being evenly matched, then consistently beat them.) The version of this diagram on our Five page also include bars indicating when we started and stopped testing against various teams: https://openai.com/five/.

We don't claim this is perfect, which is why we label the chart "estimated MMR".

A network blip caused all the players to drop from the game.

Incidentally, we'd just changed the code a few days earlier from "automatically surrender when a human disconnects" to "do nothing if a human disconnects; automatically pause if all humans disconnect". Had done a lot of advance planning for what might break!

(I work at OpenAI.)

Worth noting: it's a well-supported route to join OpenAI without any special graduate training. Many of our teams (including our robotics team!) hire experienced software engineers, teaching them whatever ML they need to know, or our Fellows program lets people do a more formal curriculum (https://blog.openai.com/openai-fellows/). We also have a number of software engineers who focus on what looks like traditional software engineering: see for example https://www.youtube.com/watch?v=UdIPveR__jw.

See our open positions here: http://openai.com/jobs!