The Nonlinear Library LW - OpenAI-Microsoft partnership by Zach Stein-Perlman
Oct 4, 2023
02:03
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: OpenAI-Microsoft partnership, published by Zach Stein-Perlman on October 4, 2023 on LessWrong.
OpenAI has a strong partnership with Microsoft. The details are opaque, as far as I know. It tentatively seems that OpenAI is required to share its models (and some other IP) with Microsoft until OpenAI attains "a highly autonomous system that outperforms humans at most economically valuable work." This is concerning because AI systems could cause a catastrophe with capabilities below that threshold.
(OpenAI may substantially depend on Microsoft; in particular, Microsoft Azure is "OpenAI's exclusive cloud provider." Microsoft's power over OpenAI may make it harder for OpenAI to refuse to share dangerous systems with Microsoft. But mostly this seems moot if OpenAI is just straightforwardly required to share its models with Microsoft.)
If so, then (given that Microsoft is worse on safety than OpenAI) whether OpenAI would do good alignment between training and deployment and then deploy cautiously mostly doesn't matter, because (if OpenAI is leading near the end) whether unsafe AI is deployed will be determined by Microsoft's decisions?
[Edit: I don't think Microsoft has full real-time access to OpenAI's models, given that they launched Bing Chat after OpenAI had RLHF'd GPT-4 but Bing Chat wasn't based on that version of GPT-4, as well as some other reporting. But it's very unclear what access it does have, or why OpenAI and Microsoft aren't transparent about this.]
(The OpenAI-Microsoft relationship seems like a big deal. Why haven't I heard more about this?)
OpenAI says:
by AGI we mean a highly autonomous system that outperforms humans at most economically valuable work. Such a system is excluded from IP licenses and other commercial terms with Microsoft, which only apply to pre-AGI technology.
It's not clear whether OpenAI has to share everything besides AGI with Microsoft.
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
OpenAI has a strong partnership with Microsoft. The details are opaque, as far as I know. It tentatively seems that OpenAI is required to share its models (and some other IP) with Microsoft until OpenAI attains "a highly autonomous system that outperforms humans at most economically valuable work." This is concerning because AI systems could cause a catastrophe with capabilities below that threshold.
(OpenAI may substantially depend on Microsoft; in particular, Microsoft Azure is "OpenAI's exclusive cloud provider." Microsoft's power over OpenAI may make it harder for OpenAI to refuse to share dangerous systems with Microsoft. But mostly this seems moot if OpenAI is just straightforwardly required to share its models with Microsoft.)
If so, then (given that Microsoft is worse on safety than OpenAI) whether OpenAI would do good alignment between training and deployment and then deploy cautiously mostly doesn't matter, because (if OpenAI is leading near the end) whether unsafe AI is deployed will be determined by Microsoft's decisions?
[Edit: I don't think Microsoft has full real-time access to OpenAI's models, given that they launched Bing Chat after OpenAI had RLHF'd GPT-4 but Bing Chat wasn't based on that version of GPT-4, as well as some other reporting. But it's very unclear what access it does have, or why OpenAI and Microsoft aren't transparent about this.]
(The OpenAI-Microsoft relationship seems like a big deal. Why haven't I heard more about this?)
OpenAI says:
by AGI we mean a highly autonomous system that outperforms humans at most economically valuable work. Such a system is excluded from IP licenses and other commercial terms with Microsoft, which only apply to pre-AGI technology.
It's not clear whether OpenAI has to share everything besides AGI with Microsoft.
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org