The Nonlinear Library LW - 2023 Prediction Evaluations by Zvi
Jan 8, 2024
44:24
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 2023 Prediction Evaluations, published by Zvi on January 8, 2024 on LessWrong.
It is that time of the year. One must ask not only whether predictions were right or wrong, whether one won or lost, but what one was and should have been thinking, whether or not good decisions were made, whether the market made sense.
The main subject will be the 2023 ACX Predictions, where I performed buy/sell/hold along with sharing my logic. The numbers quoted are from mid-February 2023, first Manifold, then Metaculus.
Section 1: World Politics
Will Vladimir Putin be President of Russia at the end of 2023 (85%/90%)?
Last year I thought markets were too confident Putin would keep power. This year I think this is not confident enough and Metaculus is more accurate at 90%. Metaculus is also doing a better job adjusting as time passes. Things seem to be stabilizing, and every day without big bad news is good news for Putin here on multiple levels. I bought M500 of YES shares, which moved this to 86%.
I increased my position later, and won M179. The market had occasional spikes downward when Putin looked to potentially be in danger, and for a while it failed to decay properly.
Looking back, there was clearly risk that events in Ukraine could have led to Putin's ouster, and he also could have head health issues. It was clear that I could have gotten much better per diem odds later in the year. So even though I won this bet, I don't think it was especially good, and Metaculus was overconfident.
Will Ukraine control the city of Sevastopol at the end of 2023 (14%/8%)?
Getting Sevastopol is a heavy lift. Russia is not about to abandon it, Ukraine has other priorities first, and Ukraine's ability to go on offensives is far from unlimited even in good scenarios. Metaculus is at 8% and once again that sounds more right to me. I bought M250 of NO here and M250 of NO in another similar market that was trading modestly higher, driving the price here to 13%.
I think this was a good bet. Certainly Russia could have completely collapsed, but even then holding onto Crimea was likely. I won M52 here and M50 in the other market. There wasn't much decay until the second half of the year, but also things looked good for Ukraine for a while, so I think the market acted reasonably relative to itself.
Will Ukraine control the city of Luhansk at the end of 2023 (28%/13%)?
This spent a bunch of time near 50% in early January, then went down. Once again Metaculus has been consistently lower and it is down at 13%. That feels very low, I'd probably be closer to 20% although I am doing much less work keeping up with the war these days, but 28% is a lot given how things are progressing right now. I bought M250 of NO shares, driving the price to 25%.
I won M92. I think the assessment of 20% here sounds reasonable. It would not have been that shocking if Ukraine had made it to Luhansk, but it was never likely.
Will Ukraine control the city of Zaporizhzhia at the end of 2023 (81%/69%)?
Metaculus is at 69%. Here I'm more inclined to lean to the Manifold number, and would want to do research before I committed much. It is not great to be selling 81% shots in prediction markets, in general. I bought 10 NO shares to keep tracking.
This resolved YES. I bought out on September 6 at 94%, losing M4 on net. I am not following closely enough to know what the right price would have been.
Will there be a lasting cease-fire in the Russia-Ukraine war at the end of 2023 (25%/24%)?
Here Manifold roughly agrees and is at 24%, down several percent in the last few days which gives me pause about selling. As everyone digs in rhetorically and literally this does seem like it is getting less likely, and the criteria seems easy to not fulfil, but a year is a long time. I bought M10 of NO for tracking purposes.
On reflection I think this was to...
It is that time of the year. One must ask not only whether predictions were right or wrong, whether one won or lost, but what one was and should have been thinking, whether or not good decisions were made, whether the market made sense.
The main subject will be the 2023 ACX Predictions, where I performed buy/sell/hold along with sharing my logic. The numbers quoted are from mid-February 2023, first Manifold, then Metaculus.
Section 1: World Politics
Will Vladimir Putin be President of Russia at the end of 2023 (85%/90%)?
Last year I thought markets were too confident Putin would keep power. This year I think this is not confident enough and Metaculus is more accurate at 90%. Metaculus is also doing a better job adjusting as time passes. Things seem to be stabilizing, and every day without big bad news is good news for Putin here on multiple levels. I bought M500 of YES shares, which moved this to 86%.
I increased my position later, and won M179. The market had occasional spikes downward when Putin looked to potentially be in danger, and for a while it failed to decay properly.
Looking back, there was clearly risk that events in Ukraine could have led to Putin's ouster, and he also could have head health issues. It was clear that I could have gotten much better per diem odds later in the year. So even though I won this bet, I don't think it was especially good, and Metaculus was overconfident.
Will Ukraine control the city of Sevastopol at the end of 2023 (14%/8%)?
Getting Sevastopol is a heavy lift. Russia is not about to abandon it, Ukraine has other priorities first, and Ukraine's ability to go on offensives is far from unlimited even in good scenarios. Metaculus is at 8% and once again that sounds more right to me. I bought M250 of NO here and M250 of NO in another similar market that was trading modestly higher, driving the price here to 13%.
I think this was a good bet. Certainly Russia could have completely collapsed, but even then holding onto Crimea was likely. I won M52 here and M50 in the other market. There wasn't much decay until the second half of the year, but also things looked good for Ukraine for a while, so I think the market acted reasonably relative to itself.
Will Ukraine control the city of Luhansk at the end of 2023 (28%/13%)?
This spent a bunch of time near 50% in early January, then went down. Once again Metaculus has been consistently lower and it is down at 13%. That feels very low, I'd probably be closer to 20% although I am doing much less work keeping up with the war these days, but 28% is a lot given how things are progressing right now. I bought M250 of NO shares, driving the price to 25%.
I won M92. I think the assessment of 20% here sounds reasonable. It would not have been that shocking if Ukraine had made it to Luhansk, but it was never likely.
Will Ukraine control the city of Zaporizhzhia at the end of 2023 (81%/69%)?
Metaculus is at 69%. Here I'm more inclined to lean to the Manifold number, and would want to do research before I committed much. It is not great to be selling 81% shots in prediction markets, in general. I bought 10 NO shares to keep tracking.
This resolved YES. I bought out on September 6 at 94%, losing M4 on net. I am not following closely enough to know what the right price would have been.
Will there be a lasting cease-fire in the Russia-Ukraine war at the end of 2023 (25%/24%)?
Here Manifold roughly agrees and is at 24%, down several percent in the last few days which gives me pause about selling. As everyone digs in rhetorically and literally this does seem like it is getting less likely, and the criteria seems easy to not fulfil, but a year is a long time. I bought M10 of NO for tracking purposes.
On reflection I think this was to...