"I knew it all along..." - avoiding hindsight bias after eruptions

“I knew it all along…” – as volcanologists, we need to be careful not to fall into the many traps that come from retrospectively looking at and indeed commenting on crises or catastrophes such as the recent eruption of Ontake.

There is a fantastic book you might want to read: Thinking fast and slow by Daniel Kahneman, which synthesises a huge body of research about how and why we make the decisions we make, particularly when it comes to risk and uncertainty. Many readers of this blog will be familiar with Kahneman’s papers, notably the 1976 “Heuristics and Biases” work with Amos Taversky. Others will be familiar with some of the work by his former PhD student Baruch Fischhoff on (among other things) risk communication*. I was planning on writing a short review of Thinking fast and slow from the perspective of what volcanologists can learn from cognitive psychology, but the eruption in Japan has got me thinking about one particular cognitive trap – the ‘hindsight bias’ or the ‘I knew it all along principle’, first investigated by Baruch Fischoff.

The key message is that as a group we must be very careful that when looking back at past eruptions, particularly when eyeballing monitoring data post-hoc, that we don’t make pronouncements about “missed warning signs” because we interpret things with the benefit of hindsight.

It turns out that it is very difficult for a human mind to reconstruct what we thought about something once we adopt a new belief about it. It leads us to believe that we understand the past, overstating the accuracy of the beliefs that we held (or would have held) at the time, as these are corrupted by what we now know. Kahneman suggests that if we are surprised by an unpredicted event, we adjust our view of the world to accommodate that surprise. Thus when we look back, we forget the state of mind or understanding that we had at the time, and simply think about what we know now.

What hindsight bias can do is lead us to interpret the quality of a decision (such as the recommendation for some kind of mitigative action) on whether the outcome was positive or negative, rather than whether or not the decision making process was sound. This bias leads us to a) overstate our expertise post-hoc, b) neglect the role of luck (or lack of it) in a particular outcome and c) suppress any memory of the effect that uncertainty will have had on our or other people’s interpretations/decisions.

Our natural tendency is to criticise decision making on risk issues when an outcome is negative, and neglect to recognise or praise decision-making when the outcome was good; this ‘outcome bias’ (a facet of hindsight) affects our interpretation of past events far more than we might realise. When considering what might happen at a volcano, a simplistic explanation is that we can consider the probability of an eruption happening given some monitoring signal [P(A|B)]. But, after an event has occurred…it’s quite different! It’s no longer an event that could happen (a chance or likelihood) but a certainty. So when we re-interpret past events, hindsight bias makes it very difficult for us in our present state of certainty, to acknowledge the attendant uncertainty before the eruption occurred. We find it very difficult to reconstruct or understand what our past belief would have been.

Kahneman suggests that these biases make it “almost impossible to evaluate a decision in terms of the beliefs that were reasonable when the decision was made”.

In fact, research suggests that the worse or more shocking a catastrophe is, the more acute hindsight bias becomes (think back to reactions in the aftermath of 9/11). This – in the case of Ontake – is reflected by language such as “failed to forecast” used in many** news articles.

So what does this mean for volcanologists in the wake of a tragedy such as the eruption of Ontake? Well, the first thing we should be aware of is that our opinions post-hoc, about what monitoring data may or may not have shown, or what decisions should or shouldn’t have been made, are prone to huge biases. So, we should be very careful what we voice about these events…particularly to the media! If we are going to retrospectively look at something, let’s do it in a robust and sensible way, such as the work by TheaHinks, Willy Aspinall and others on the 1976 eruption of Soufriére Guadeloupe.

Another point is that from afar – not being a Japanese volcanologist working on Ontake – the availability of information for us to be able make an informed opinion is surely very limited (what Kahneman refers to as the ‘availability bias’ or the ‘what you see is all there is to know principle’). So, just as we should be very cautious about talking about ‘missed signs’, we should also be aware that when we say things like ‘it’s impossible/very difficult to predict such eruptions’ or ‘there were no precursors’, our opinions are perhaps based on very sparse evidence (of course we can draw on other examples from other cases – but hopefully you get my point). In essence, maybe we could do with waiting for a little more information before passing comment.

Hopefully you get the idea that if you haven’t yet read Thinking fast and slow, then please do. It’s very difficult to overcome the various heuristics and biases that affect our opinions and decisions (even Kahneman admits to relentlessly struggling with this) …but being aware of them is an excellent first step.

 * Want to know more about the science of risk communication - read this excellent paper by Nick Pidgeon and Baruch Fischhoff 

** Not all articles/commentaries fall foul of the hindsight bias -  if you want to read some measured and not overly opinionated articles by volcanologists about the Ontake eruption – you might want to look here (Becky Williams) and here (Eruptions blog).

New paper published!

My first ever paper has been published: http://www.appliedvolc.com/content/3/1/11

I'll write a longer blog post about it soonish. It is open access, so anyone can read!

Behind the scenes: making a film about La Soufriére

You can imagine my surprise when I was the first person on a long list not too busy to head out to St Vincent to make a film about La Soufrière…! Having worked on St Vincent before, I sort of knew what to expect – volcanic paradise.

The plan was to make a series of short films about the volcano, describing its recent eruptions, how people responded to eruptions in the past, and what might happen at the volcano in the future. We didn’t want to just make a documentary, or do something just for the sake of ‘outreach’ – we wanted to be able to tell the story of living alongside a volcano through the experiences of those that are.

La Soufrière rising into the clouds, as seen from above the town of Chateaubelair.

For the two volcanologists, Richie Robertson (Seismic Research Centre, UWI) and I – filming in the field with a professional company was a totally new experience. Similarly, the crew from Lambda Films had yet to test their skills in either the tropics or on an active volcano.

La Soufrière is an extraordinary volcano, which dominates the northern third of St Vincent. Its lush green slopes show little sign of its destructive past; the products of large eruptions in 1979 and 1902 barely visible to the untrained eye. This is a common problem to encounter on low altitude volcanoes in the tropics, as the vegetation very quickly obscures where pyroclastic density currents have burnt the landscape. Yet, with a keen eye and, of course, knowing where to look, you can uncover the signs of the past. Our first filming location with Richie was at a sugar plantation, destroyed by the 1902 eruption. Buildings covered in ash and flow deposits up to their roof, only the top of a window and the giant water wheel are still visible. Much industry like this, situated on the volcano’s slopes was damaged to such an extent in 1902, that many plantations and farms never recovered. The same happened in 1979, and although a smaller eruption, the long-term effects were felt by many.

Richie describing the societal effects of past eruptions at a destroyed sugar mill.

We interviewed many people who had lived through the 1979 eruption, and they recounted their experiences for us. It’s really important for memories of past eruptions to be recorded and passed down between generations and shared with communities elsewhere in the world. Although all volcanoes are different, erupting into a vast range of social and economical contexts, many of the struggles and challenges of living with them are shared. With this series of films, we hope to show some of these challenges and how people adapted.

For a researcher who is used to interviewing or conducting focus groups, doing this from behind a camera whilst trying to get responses that would contain sound-bites for a film, was very different! I normally have a very conversational interviewing style, but the camera can inhibit this a little. However, the limitations of using a video camera became insignificant when reviewing the footage. Interviews, descriptions of past events and, most importantly, the people themselves, came to life on screen. We are sure that the footage we collected will be a great legacy for people on St Vincent.

The team interviewing a lady about the 1979 eruption, with La Soufrière in the background.

As well as the interviews, the team hiked up the volcano, to film its crater and the deposits on the flanks. I had a brilliant idea to climb it at 3am – so that we could film sunrise over Georgetown, an at risk town on the eastern flanks. This ‘romantic sunrise trek’, on offer at many volcanoes worldwide, was a total fail – we could only see 10m in front of us…! Nevertheless, at the top of the volcano, any notion of vegetated slopes or a hidden past is literally blown away when you reach the rim of the crater and stare down inside. The clouds swirl and part… three hundred meters below, the crater floor expands into the distance, interrupted by a lava dome, the scale of which is indeterminable to your eyes, is then encircled by the wall on other side – over 1.5km away. A local guide, Delroy Browne, once told me that his favourite thing about climbing La Soufrière is to see the look on people’s faces when they look into it. He isn’t wrong!!

The crater of La Soufrière, with the lava dome from 1979 in the centre.

To get into the crater to do our filming, we had to descend the very steep walls, with a thick rope to stop you from sliding down (probably a long way) if you were to lose your footing. The film crew with their kit, me with their water and tripod, and Richie survived the descent unscathed, and set about a day filming down there.

The team descending the crater walls.

The crater is so important, both physically and symbolically. The exposures of rock in the walls, showing old lava flows and explosive eruptions, tell us a story about the volcano, which is so difficult to see anywhere else. For the people of the island, the crater is a foreboding location, the source of rumor, intrigue and potential devastation. Standing in it, swirling banks of clouds often obscure the walls, wafts of sulphurous gasses coming from the many fumaroles heighten the sense of excitement. We of course unleashed both of the drones that we brought with us to get some aerial shots. Following a good morning filming, including a breakfast of hotdogs – cooked in the dome’s gas vent, we climbed back out and made our way back down.

Shot from the volcanocopter of Lamba Film’s drone filming Richie Robertson in the crater

Despite having been to St Vincent before, or indeed being fortunate enough to work on many incredible volcanoes, the experience of making a film about one is something that I, and I imagine the rest of the team, will never forget. Quite what makes St Vincent feel so special to me, I don’t know.Whether it was incredible street-cooked jerked bbq food, an extraordinary volcano, some of the friendliest people you can find, a rich history or a volcanically threatened future – it has to be near the top of a volcanologist’s bucket-list for fieldwork.
Risk communication can be a tough task, in many cases it is a balancing act between communicating the right message to get people interested/concerned enough about a hazard so that they think about it and plan for it, whilst not overreacting to it. Undoubtedly, St Vincent will experience eruptions from La Soufriére in the future that will threaten communities and ways of life, but we hope that making and showing films like this can help to strengthen resilience there by learning from the past, and telling that story through the eyes of those that lived through it.

(This post was originally written for the London Volcano blog)

The Inefficiency of Compassion

After a large earthquake first news comes of fatalities and numbers people affected, but soon after estimations of the cost of the disaster are reported. This figure tends to rise as time passes, however usually the economic burden is much larger with indirect losses felt in the local communities affected, such as uninsured losses, loss of income, business downtime, etc. But could all of this financial loss be, in fact, not lost and instead used to develop and strengthen resilience in communities?

I heard a story once of a factory in Asia built for use by a western company. The highly seismic area was prone to large earthquakes, so to avoid losses through downtime the factory was built to the highest standards. When a large earthquake came, the factory was in good shape and able to open the next day. However, no one turned up to work. The staff had been made homeless, lost loved ones, were injured or some even killed. From this example it becomes obvious that strengthening works, but the effects of disaster are wider spread and, in order to achieve resilience, investment and strengthening needs to consider multiple aspects.

Recently, a headline used by Care International struck me:

'Fixing the Inefficiency of Compassion'.

The article highlights the ineffective use of funds in the aftermath of disasters, when the same money could protect so many more people and therefore avoid suffering by many. A previous blog post of mine explains that 'for every $1 spent on disaster preparedness, between $2 and $7 is saved in disaster response'. There are various other values calculated by different institutions but the commonality is that it is always better value for money to invest in DRR than to spend on post-disaster recovery.

But how do we fix this inefficient spending? How to we encourage aid donations to be made when there isn't yet suffering. Is it enough to say "Your dollar will go further if you give it now, before hurricane season". Would you donate then?

The organisation Build Change is one example of a proactive organisation that aims to protect communities, instead of helping to 'pick up the pieces'. They have a wealth of technical resources, inspiring projects and opportunities to be involved.

Supporting a charity like gives you the best value for money, almost like a bargain and we all love a bargain.

Top tips for risk communicators

  

An amusing example of precisely inaccurate communication about volcanoes from a tourism company in Baños, Ecuador

Risk communication is successful if it reaches all of those who need it, is comprehended by the end user and received within a timescale that they can make use of it. How we communicate science: what we say, how we say it, who says it, why we say it, is vitally important when the goal is to help people make informed decisions. This takes a wide variety of knowledge and expertise:

“Meeting that goal requires collaboration between scientists with subject matter knowledge to communicate and scientists with expertise in communication processes—along with practitioners able to manage the process” Baruch Fischhoff 

Fischhoff - is a world leader in research on risk communication and this post is focused on a recent publication of his entitled “The sciences of Science Communication”.

Around volcanoes, and also when dealing with other natural hazards, as scientists we often can’t afford to make mistakes when communicating risk. Some do it well, some do it very badly – but it is important for the new generation of scientists working with risk communication to learn how to do it properly. We must ask ourselves: How can we not only get it right, but understand when we do, and crucially why sometimes we get it wrong.

It is true in volcanology that often the best risk communicators do it naturally. Some of this is intuitive; some of it has been learned via a vast number of experiences, including some mistakes. In this world, where we are increasingly interconnected socially, with information available at the click of a finger – we may only get one chance to communicate effectively – with so many alternative sources of information out there…we can only expect to hold the public’s attention through competence, care, integrity, reliability, fairness and openness (some of the dimensions of ‘trust’). So it is important for us to learn from those who have experience and also to get technical about it – in essence to understand the sciences of science communication.

In Fischoff’s recent paper, he describes 4 tasks for a science communicator:

Task 1: Identify the science most relevant to the decisions that people face.

Task 2: Determine what people already know.

Task 3: Design communications to fill the critical gaps (between what people know and need to know).

Task 4: Evaluate the adequacy of those communications.

Fischhoff then goes on to describe a series of mistakes, common to many instances of failed or inadequate risk communication.

Common mistakes:

·      Did they get the science wrong, and lose credibility?

·      Did they get the wrong science, and prove irrelevant?

·      Did they lack clarity and comprehensibility, frustrating their audiences?

·      Did they travel through noisy channels, and not reach their audiences?

·      Did they seem begrudging, rather than forthcoming?

·      Did they fail to listen, as well as to speak?

·      Did they try to persuade audiences that wanted to be informed, or vice versa?

So whether you are currently a risk communicator or think you may be in that position in the future, write these things down, think about them and don’t simply rely on being intuitively good at it. Don’t take my word for it – read Fischhoff’s paper  - it is really great (if you cant get it just ask me for it).

Fischhoff, B. (2013). The sciences of science communication. Proceedings of the National Academy of Sciences, 110(Supplement 3), 14033–14039.

http://www.pnas.org/content/110/Supplement_3/14033.full

Sinabung & dangerous lava domes

Sinabung in Sumatra has been erupting for the last few months, prompting regular evacuations of people living near to it.

(Incandescent lava dome, flow deposit and burning vegetation - Reuters/YT Haryono)

Currently, the style of activity is producing a lava dome: where viscous (less fluid) magma builds up on top of the vent. The lava, although seemingly solid, can have a lot of gas trapped inside at high pressures, and is also still very hot. Sometimes chunks of it fall off the dome, these blocks or slabs break apart, releasing the gas, mixing with and heating the surrounding air, forming pyroclastic flows and surges.

Depending on where the lava is being emplaced on the volcano, material may fall off in different directions, thus the areas most at risk can change quite quickly. Dome building eruptions often produce a LOT of flows - the size of which can often follow a power law relationship: many small events, with a diminishing number of larger events*. Lava domes make both managing and communicating the risk very difficult. Often people are evacuated in the anticipation of larger flows, which may not happen for a while (if at all). After a time of small flows, many people naturally want to return to their homes. Then they may be evacuated again, and subsequently return. This process can occur many times, and ultimately people can become very reluctant to leave.

Unfortunately, this kind of relationship is very different to grasp.  In Montserrat, 1997, this process (among other factors) occured - the danger perhaps obvious to the scientists, but people became used to where flows were going and how big they were. Many thought that they understood the speed of them, and unfortunately thought they could escape in time. On June 25th 1997, people on the slopes of the Soufrière Hills, in areas that they were advised not to visit, were caught off guard (despite numerous warnings from scientists) by a sudden increase in the magnitude and energy of the flows during a partial dome collapse, which lead to a tragic loss of life*. 

This has unfortunately been the case in many similar eruptions, from Soufrière Hills Volcano to Merapi. An added danger is that apart from larger flows related to small collapses...lava domes can also produce large vulcanian explosions - which create even more energetic flows, that can sweep down all sides of the volcano at once.

The key thing is to not expect a volcano to always behave in the same way, but rather to think "what could it do to surprise me?".

Despite the fact that the Indonesian scientists are very capable volcanologists and communicators, we just have to hope that the eruption calms down again.




*Loughlin, S. & Baxter, P., 2002. Eyewitness accounts of the 25 June 1997 pyroclastic flows and surges at Soufrière Hills Volcano, Montserrat, and implications for disaster mitigation. Geological Society of London. 
*Loughlin, S., Calder, E. & Clarke, A., 2002. Pyroclastic flows and surges generated by the 25 June 1997 dome collapse, Sonfière Hills Volcano, Montserrat. Geological Society of London. 

4 years ago, in a far away land...

Four years ago today, an earthquake that will be remembered as one of the worst in our generation shook the caribbean country of Haiti. The world watched in horror as the death toll rose to hundreds of thousands.

Construction in Haiti was known to have been generally poor. Even two years before the earthquake, a school collapsed killing nearly 100 children. When the earthquake struck many other buildings, and human lives, had the same fate.

Experts had warned of the major seismic hazard that the island was exposed to just two years before the event, so it was known that the region could be subject to seismic shaking, as is the case for many places. However, it so often remains a a low priority in favour of more pressing matters - the economy, trade, hurricanes, poverty - viewed as risk for the distant future. Ignored in Haiti, post-disaster the major concerns for the country remain as before: the economy, trade, hurricanes, poverty, with construction standards still not enforced properly. 

I can't help but worry that this devastation could have occurred in so many places in the Caribbean and throughout the world; Haiti was simply the 'unlucky' one that day. But what can be done to avoid a similar disaster elsewhere?

Below is a diagram highlighting the three possible ways forward.

The way forward...

Option 1 is asking for trouble.

Option 2 is the bare minimum.

Option 3 will be hard work and costly; if successful will avoid a disaster on the Haitian scale.

I worry that more needs to be done. That authorities need to be pressured to not take each day for granted. In some places, large earthquake will happen, the only question is whether it's tomorrow or in a 100 years. Lessons can be learnt from others mistakes. Learn this Haitian hotel owners lesson now. 

”This is something that we had forgotten for a long time — that Haiti was subject to earthquakes. Now we know. Now we know that we need to build better.”