How could the Good Judgement Project help with managing woodlands?
If you manage a woodland you must be making forecasts about the future. You are probably predicting how your interventions, such as tree-planting or creating clearings will work out for wildlife and timber production in the years ahead. But how good are we are forecasting the effects of our actions and how could we get better at it? In the last few years a "Good Judgement Project" group led by Philip Tetlock at the University of Pennsylvania has been finding out scientifically how we can make more accurate predictions. Oddly the "Good Judgement Project" was originally funded by the CIA who realised that their geopolitical forecasts were weak and needed to be improved. What Tetlock did was to get a large number of volunteers to make forecasts about events that could be checked within a short timeframe and against an objectively checkable outcome, such as "Which party's candidate will win the 2016 Presidential election?" He then scored the results with a technique that rewarded accuracy but allowed people to adjust their predictions periodically. His best forecasters became his group of about 200 "superforecasters".
The results of this exercise were surprising because they showed that this group of self-motivated volunteers could significantly beat the paid experts. Part of this comes down to the their better skill at quantifying probabilities and how effectively they found and analysed the most relevant data. It was also partly because the volunteers were totally focused on making the best predictions whereas so-called "experts" can have very different motivations. For example a TV pundit will be making forecasts to create a newsworthy story, while a political leader may be making over-optimistic forecasts in order to rally his/her supporters or to show them how loyal he/she is to their shared cause. Some forecasters are making forecasts to bolster their careers so they will be likely to reflect group thinking and to avoid outlying possibilities that might make their colleagues think less of them. People working within organisations tend to be over-confident and inflexible - it has been proven that people who express very definite views are perceived as better leaders and more honest.
It turns out that certain ways of forecasting increase your accuracy significantly. One of the most important is to look at an objective comparator so that, for example, if you are at a wedding and someone asks you what is the chance of the couple who are getting married still being together in 20 years, you are likely to put too high a figure on that because of the context. The relevant starting point is probably to look up the statistics and find, say, that only 65% of marriages last for 20 years. So you would start from that figure and perhaps adjust it upwards for other factors. You might also use a technique that is very successful for Philip Tetlock's superforecasters and confer with other analysts. So you might ask around and find out what others think and how they are thinking - so that they would point out things that your didn't know or hadn't thought to consider (are the couple's parents' divorced? How old are the people getting married? How long have they known each other? etc). There are other techniques that are perhaps counter-intuitive but which help good forecasters to make better predictions such as being ready to adjust their probability estimates in the light of new evidence. Human nature is inclined to dig in and give too little weight to new evidence, whereas strong forecasters will be more likely than the average to change their predictions as the evidence changes. There is a tendency too to have a "just world bias": most of us tend to over-estimate the chance of things turning out in the way that we'd like them to. So, if you ask someone the chance of President Assad in Syria being deposed within the next 6 months they tend to over-estimate the likelihood of this happening because they don't like his leadership.
In forestry we have seen quite a few forecasts recently that have been well off the mark. For example, there have been predictions that Horse Chestnut numbers will be severely cut due to the leaf miner, or that Ash die-back will sweep across the UK and eliminate most of the ash trees within a short period or that sudden Oak death will slash the number of oak trees in Britain. Today it looks as though these predictions were probably inaccurate but we can see logic for that: there was a bias in what the media reported because newspapers and TV want stories that will get people's attention. Also the forecasters of gloom and doom sometimes found it hard to get enough perspective - they were so taken up with the issue and were mostly working in the affected regions that they found it harder to "see the wood for the trees". Therefore when we make decisions about how to manage our woodlands we should consider the factors that superforecasters would give most weight to: what has happened when the same intervention has been applied in other parts of the woodland? or in nearby locations? what do others think? have we separated out what we think will actually happen from what we'd really like to happen? etc. In effect we can all do a lot better if we move from optimistic guessing to using careful, scientific forecasting techniques when managing our woodlands for the future.
Comments are closed for this post.