The Nonlinear Library LW - AI Is Not Software by Davidmanheim
Jan 2, 2024
07:22
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI Is Not Software, published by Davidmanheim on January 2, 2024 on LessWrong.
Epistemic Status: This idea is, I think, widely understood in technical circles. I'm trying to convey it more clearly to a general audience. Edit: See related posts like this one by Eliezer for background on how we should use words.
What we call AI in 2024 is not software. It's kind of natural to put it in the same category as other things that run on a computer, but thinking about LLMs, or image generation, or deepfakes as software is misleading, and confuses most of the ethical, political, and technological discussions. This seems not to be obvious to many users, but as AI gets more widespread, it's especially important to understand what we're using when we use AI.
Software
Software is how we get computers to work. When creating software, humans decide what they want the computer to do, think about what would make the computer do that, and then write an understandable set of instructions in some programming language. A computer is given those instructions, and they are interpreted or compiled into a program. When that program is run, the computer will follow the instructions in the software, and produce the expected output, if the program is written correctly.
Does software work? Not always, but if not, it fails in ways that are entirely determined by the human's instructions. If the software is developed properly, there are clear methods to check each part of the program. For example, unit tests are written to verify that the software does what it is expected to do in different cases. The set of cases are specified in advance, based on what the programmer expected the software to do.
If it fails a single unit test, the software is incorrect, and should be fixed. When changes are wanted, someone with access to the source code can change it, and recreate the software based on the new code.
Given that high-level description, it might seem like everything that runs on a computer must be software. In a certain sense, it is, but thinking about everything done with computers as software is unhelpful or misleading. This essay was written on a computer, using software, but it's not software. And the difference between what is done on a computer and what we tell a computer to do with software is obvious in cases other than AI. Once we think about what computers do, and what software is, we shouldn't confuse "on a computer" with software.
Not Software
For example, photos of a wedding or a vacation aren't software, even if they are created, edited, and stored using software. When photographs are not good, we blame the photographer, not the software running on the camera. We don't check if the photography or photo editing worked properly by rerunning the software, or building unit tests. When photographs are edited or put into an album, it's the editor doing the work.
If it goes badly, the editor chose the wrong software, or used it badly - it's generally not the software malfunctioning. If we lose the photographs, it's almost never a software problem. And if we want new photographs, we're generally out of luck - it's not a question of fixing the software. There's no source code to rerun. Having a second wedding probably shouldn't be the answer to bad or lost photographs. And having a second vacation might be nice, but it doesn't get you photos of the first vacation.
Similarly, a video conference runs on a computer, but the meeting isn't software - software is what allows it to run. A meeting can go well, or poorly, because of the preparation or behavior of the people in the meeting. (And that isn't the software's fault!) The meeting isn't specified by a programming language, doesn't compile into bytecode, and there aren't generally unit tests to check if the meeting went well. And when we want to ...
Epistemic Status: This idea is, I think, widely understood in technical circles. I'm trying to convey it more clearly to a general audience. Edit: See related posts like this one by Eliezer for background on how we should use words.
What we call AI in 2024 is not software. It's kind of natural to put it in the same category as other things that run on a computer, but thinking about LLMs, or image generation, or deepfakes as software is misleading, and confuses most of the ethical, political, and technological discussions. This seems not to be obvious to many users, but as AI gets more widespread, it's especially important to understand what we're using when we use AI.
Software
Software is how we get computers to work. When creating software, humans decide what they want the computer to do, think about what would make the computer do that, and then write an understandable set of instructions in some programming language. A computer is given those instructions, and they are interpreted or compiled into a program. When that program is run, the computer will follow the instructions in the software, and produce the expected output, if the program is written correctly.
Does software work? Not always, but if not, it fails in ways that are entirely determined by the human's instructions. If the software is developed properly, there are clear methods to check each part of the program. For example, unit tests are written to verify that the software does what it is expected to do in different cases. The set of cases are specified in advance, based on what the programmer expected the software to do.
If it fails a single unit test, the software is incorrect, and should be fixed. When changes are wanted, someone with access to the source code can change it, and recreate the software based on the new code.
Given that high-level description, it might seem like everything that runs on a computer must be software. In a certain sense, it is, but thinking about everything done with computers as software is unhelpful or misleading. This essay was written on a computer, using software, but it's not software. And the difference between what is done on a computer and what we tell a computer to do with software is obvious in cases other than AI. Once we think about what computers do, and what software is, we shouldn't confuse "on a computer" with software.
Not Software
For example, photos of a wedding or a vacation aren't software, even if they are created, edited, and stored using software. When photographs are not good, we blame the photographer, not the software running on the camera. We don't check if the photography or photo editing worked properly by rerunning the software, or building unit tests. When photographs are edited or put into an album, it's the editor doing the work.
If it goes badly, the editor chose the wrong software, or used it badly - it's generally not the software malfunctioning. If we lose the photographs, it's almost never a software problem. And if we want new photographs, we're generally out of luck - it's not a question of fixing the software. There's no source code to rerun. Having a second wedding probably shouldn't be the answer to bad or lost photographs. And having a second vacation might be nice, but it doesn't get you photos of the first vacation.
Similarly, a video conference runs on a computer, but the meeting isn't software - software is what allows it to run. A meeting can go well, or poorly, because of the preparation or behavior of the people in the meeting. (And that isn't the software's fault!) The meeting isn't specified by a programming language, doesn't compile into bytecode, and there aren't generally unit tests to check if the meeting went well. And when we want to ...