The Nonlinear Library EA - Impact Assessment of AI Safety Camp (Arb Research) by Sam Holton
Jan 24, 2024
22:38
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Impact Assessment of AI Safety Camp (Arb Research), published by Sam Holton on January 24, 2024 on The Effective Altruism Forum.
Authors: Sam Holton, Misha Yagudin
Data collection: David Mathers, Patricia Lim
Note: Arb Research was commissioned to produce this impact assessment by the AISC organizers.
Summary
AI Safety Camp (AISC) connects people interested in AI safety (AIS) to a research mentor, forming project teams that last for a few weeks and go on to write up their findings. To assess the impact of AISC, we first consider how the organization might increase the productivity of the Safety field as a whole. Given its short duration and focus on introducing new people to AIS, we conclude that AISC's largest contribution is in producing new AIS researchers that otherwise wouldn't have joined the field.
We gather survey data and track participants in order to estimate how many researchers AISC has produced, finding that 5-10% of participants plausibly become AIS researchers (see "Typical AIS researchers produced by AISC" for examples) that otherwise would not have joined the field. AISC spends roughly $12-30K per researcher. We could not find estimates for counterfactual researcher production in similar programs such as (SERI) MATS.
However, we used the LTFF grants database to estimate that the cost of researcher upskilling in AI safety for 1 year is $53K. Even assuming all researchers with this amount of training become safety researchers that wouldn't otherwise have joined the field, AISC still recruits new researchers at a similar or lower cost (though note that training programs at different stages of a career pipeline are compliments).
We then consider the relevant counterfactuals for a nonprofit organization interested in supporting AIS researchers and tentatively conclude that funding the creation of new researchers in this way is slightly more impactful than funding a typical AIS project. However, this conclusion is highly dependent on one's particular views about AI safety and could also change based on an assessment of the quality of researchers produced by AISC.
We also review what other impacts AISC has in terms of producing publications and helping participants get a position in AIS organizations.
Approach
To assess impact, we focus on AISC's rate of net-new researcher production. We believe this is the largest contribution of the camp given their focus on introducing researchers to the field and given the short duration of projects. In the appendix, we justify this and explain why new researcher production is one of the most important contributions to the productivity of a research field. For completeness, we also attempt to quantify other impacts such as:
Direct research outputs from AISC and follow-on research.
Network effects leading to further AIS and non-AIS research.
AISC leading to future positions.
AISC plausibly has several positive impacts that we were unable to measure, such as increasing researcher effort, increasing research productivity, and improving resource allocation. We are also unable to measure the quality of AIS research due to the difficulty of assessing such work.
Data collected
We used 2 sources of data for this assessment:
Survey. We surveyed AISC participants from all camps, receiving 24 responses (~10% of all participants). Questions aimed to determine the participants' AIS involvement before and after camp as well as identify areas for improvement. To ensure honest answers, we promised respondents that anecdotes would not be shared without their direct permission. Instead, we will summarize common lessons from these responses where possible.
Participant tracking. To counter response biases in survey data, we independently researched the career path of 101 participants from AISC 4-6, looking at involvement in AI safety rese...
Authors: Sam Holton, Misha Yagudin
Data collection: David Mathers, Patricia Lim
Note: Arb Research was commissioned to produce this impact assessment by the AISC organizers.
Summary
AI Safety Camp (AISC) connects people interested in AI safety (AIS) to a research mentor, forming project teams that last for a few weeks and go on to write up their findings. To assess the impact of AISC, we first consider how the organization might increase the productivity of the Safety field as a whole. Given its short duration and focus on introducing new people to AIS, we conclude that AISC's largest contribution is in producing new AIS researchers that otherwise wouldn't have joined the field.
We gather survey data and track participants in order to estimate how many researchers AISC has produced, finding that 5-10% of participants plausibly become AIS researchers (see "Typical AIS researchers produced by AISC" for examples) that otherwise would not have joined the field. AISC spends roughly $12-30K per researcher. We could not find estimates for counterfactual researcher production in similar programs such as (SERI) MATS.
However, we used the LTFF grants database to estimate that the cost of researcher upskilling in AI safety for 1 year is $53K. Even assuming all researchers with this amount of training become safety researchers that wouldn't otherwise have joined the field, AISC still recruits new researchers at a similar or lower cost (though note that training programs at different stages of a career pipeline are compliments).
We then consider the relevant counterfactuals for a nonprofit organization interested in supporting AIS researchers and tentatively conclude that funding the creation of new researchers in this way is slightly more impactful than funding a typical AIS project. However, this conclusion is highly dependent on one's particular views about AI safety and could also change based on an assessment of the quality of researchers produced by AISC.
We also review what other impacts AISC has in terms of producing publications and helping participants get a position in AIS organizations.
Approach
To assess impact, we focus on AISC's rate of net-new researcher production. We believe this is the largest contribution of the camp given their focus on introducing researchers to the field and given the short duration of projects. In the appendix, we justify this and explain why new researcher production is one of the most important contributions to the productivity of a research field. For completeness, we also attempt to quantify other impacts such as:
Direct research outputs from AISC and follow-on research.
Network effects leading to further AIS and non-AIS research.
AISC leading to future positions.
AISC plausibly has several positive impacts that we were unable to measure, such as increasing researcher effort, increasing research productivity, and improving resource allocation. We are also unable to measure the quality of AIS research due to the difficulty of assessing such work.
Data collected
We used 2 sources of data for this assessment:
Survey. We surveyed AISC participants from all camps, receiving 24 responses (~10% of all participants). Questions aimed to determine the participants' AIS involvement before and after camp as well as identify areas for improvement. To ensure honest answers, we promised respondents that anecdotes would not be shared without their direct permission. Instead, we will summarize common lessons from these responses where possible.
Participant tracking. To counter response biases in survey data, we independently researched the career path of 101 participants from AISC 4-6, looking at involvement in AI safety rese...