What is a cognitive bias?
A cognitive bias is a tendency to make repeated and similar mistakes in thinking. Unlike random errors, which have no pattern, these mistakes are systematically wrong in one direction. They usually arise from simple rules of thumb, or heuristics, which the mind uses to help it perform a task more easily.
As an example, the diagram illustrates the Ebbinghaus Illusion – the orange circle on the right appears larger than the orange circle on the left, to most people. In fact they are both the same size. The heuristic involved here is to take the context into account when estimating the size of things. The bias is to usually overestimate the size of objects when smaller ones surround them – and underestimate the size of objects surrounded by larger ones.
The effects of cognitive biases are not only visual. As another example, an elegant experiment was devised by Drazen Prelec and Dan Ariely at MIT in 2006 (for further details see Ariely’s book Predictably Irrational). In the experiment, people are first asked to write down the last two digits of their social security number and consider whether they would pay that number of dollars for items whose value they did not know, such as wine, chocolate and computer equipment. They were then asked to bid for these items. The result was that the people with higher two-digit final social security numbers submitted bids that were between 60% and 120% higher than those with the lower social security numbers, for the same item.
The heuristic here is the anchoring effect – where estimates start from an initial value that is then adjusted to give a final guess. The bias is that the adjustment is usually insufficient.
Developing a test for cognitive biases
The Cognitive Bias Test was conceived and developed by Dominic Brittain. Dominic spent decades personally defusing bombs and, as the Commanding Officer, taking responsibility for training others to lead bomb disposal teams. He noticed that despite personality testing, different bomb disposal team leaders, with the same training and theoretically the ‘correct’ personality for high stress operations, would behave in various ways when faced with uncertainty.
Some might instantly act, others might gather more and more information, yet others might simply repeat behaviour they had seen successfully used in the past. He realised that there were various pathways between personality and behaviour, and that cognitive biases played a huge part in the thinking process as unconscious faults in logic. He came to realise that success for these men and women came not just from technical ability, but also from the ability to clarify, understand and modify their thinking.
Before retiring from bomb disposal, Dominic partnered up with Vincent Gauthier, an experienced business consultant, in 2008. While working with Vincent to help senior leaders in multinational corporations, he saw corruptions in their thinking, very similar to those that he had found in bomb disposal operators. Again, cognitive biases were at play.
Lovallo and Sibony published their research on cognitive biases (see useful links) in the McKinsey Quarterly in 2010. Dominic realised that he could combine his experience of helping leaders overcome their cognitive biases with Lovallo and Sibony’s typology of biases to develop a simple method of testing for the cognitive biases he was seeing in senior executives. He launched the first version of the Cognitive Bias Test in 2014.
The Cognitive Bias Test measures individual and team cognitive biases so that awareness of them can be improved and techniques implemented that will help mitigate their effects. In short, whether you are defusing a bomb or running a large commercial organisation, if you don’t understand your own thinking and work to improve it, your strategic decisions will be degraded by unconscious cognitive biases.
Cognitive biases in context
One way of putting cognitive biases into context is to explore the links between personality and behaviour. It is commonly accepted that personality is partly innate, partly formed from experiences, and that it drives behaviours consciously and unconsciously.
The ‘Pathways Model’, shows one view of the links between personality and behaviour. It borrows from Freud’s useful (but unproven) model of three sections of the mind, and the Pathways Model shows three similar main pathways between personality and behaviour – the conscious (you are aware of it), the pre-conscious (you can become aware of it if you direct your attention to it) and the unconscious (extremely difficult to become aware of it). The key point is that both the conscious and the pre-conscious pathways are reached via one’s cognitive biases.
The unconscious pathway includes unconscious biases. These are the product of generalisations leading to prejudice, and many are rightly illegal in most countries. Not employing (or promoting) someone simply because of their race, gender, sexual orientation or religion, to name a few examples, is clearly wrong and not an error that most senior executives readily fall into. There have been some interesting ways developed by Harvard to test for such biases, known as the Implicit Association Tests which are available on the Harvard website.
The unconscious pathway also includes the reptilian response. The neurologist Paul MacLean, who posited that there might be three brains in humans – the reptilian, the limbic and the neo-cortex, first described this. In Maclean’s theory each of these brains represents an evolutionary layer, starting with the reptilian brain. Modern research suggests that Maclean’s three layer model is over simplified – for example, the limbic brain is not thought to be a distinct entity; but MacLean’s central concept that the reptilian brain is concerned with survival and the fight or flight response, and that the neo-cortex evolved to control the reptilian brain, gives a model which is a simple, fundamental insight into peoples’ behaviour, particularly when they are under pressure. This reptilian ‘fight or flight’ response is the second part of the unconscious pathway in the Pathways Model. Given the fact that senior leaders in any organisation tend to be judged more on how they react more than how they act, this is usually a response which, like the unconscious biases, is under good regulation by most senior executives – at least most of the time.
The conscious and the pre-conscious pathway both involve cognitive biases – systematic corruptions in thinking that are remarkably stable and persistent. These cognitive biases affect the way that information is gathered and analysed. They are insidious, because leaders think that they are acting rationally, but the corruption in the logic used to reach a decision can lead to some very poor outcomes.
The pre-conscious pathway is where leaders go with their ‘gut feel’, and while senior executives tend to be aware of them, these rapid decisions are heavily influenced by pre-conscious cognitive biases. These instinctive responses can be very useful in solving tactical issues, but fall down when strategic thinking is required, for reasons explained later.
Referring back to the Pathways Model, the conscious pathway has two distinct routes. Conscious biases are those where a leader deliberately chooses a certain way of behaving. The teacher who ‘never smiles before Christmas’ is using conscious biases to assert her authority or overcome a fear of being taken advantage of by the children. The CEO who ‘never flies commercial’ is using conscious biases to reinforce his status, as the MD probably is who ‘never speaks to the employees because it disrupts the chain of command’. These are all actual examples.
The other, more difficult, conscious pathway is via reflection, where a sustained effort is made to understand the workings of one’s mind and to generate alternative viewpoints. This requires knowledge of one’s thinking patterns (which includes cognitive biases) as well as specific techniques for generating and evaluating alternative perspectives.
Finally, debating the merits of a decision with a diverse, wider group can enhance the ability of individual reflection to reduce cognitive biases. Diversity is key here – one more reason why unconscious biases, which always reduce diversity, need to be kept under good regulation. A senior team solely populated by white, heterosexual, older males – the hackneyed stale, male and pale image – is going to struggle to bring the necessary diversity of views to the table to counter cognitive biases – but so would a team exclusively made up of younger, black lesbians.
Debating issues with a diverse team is one good approach taken by senior executives, who will often have some form of Senior Leadership Team to assist them with decision-making, among other functions. The problem that arises here is that this senior team, as they work together over time, develop their own set of shared cognitive biases, which then adversely affect their debates and recommendations.
There are two key aspects to dealing with the effects of cognitive biases on rational decision-making. The first is to understand the cognitive biases that are involved. These can be tested for using the Cognitive Bias Test – both for the individuals on the team, and for the team itself. The second aspect is to introduce techniques that reduce the effects of these biases, both on individual reflection and during team debates.
Strategic leadership and cognitive biases
Forming a good strategy involves answering four key questions:
- Where are we now?
- Where do we need to be and why?
- How will we get there?
- How will we know when we have arrived?
Strategic leaders set the direction by changing the organisation’s internal culture and systems so that they match the external environment – now and in the future. Strategic leaders then make sense of what is emerging and steer the response.
Cognitive biases disrupt strategic leadership because:
- they cloud judgment of both the external environment and the internal culture and systems of the organisation.
- when making sense of what is emerging, they filter perceptions, so that some facts can be downplayed or even ignored, while assumptions may be mistaken for facts.
- when responding to what is emerging, they restrict the range of options considered.
This is where the downsides of cognitive biases become apparent. Experts who go with their gut feel to solve tactical problems quickly and well, are often poor predictors of the future, even in their field of expertise, as Philip Tetlock shows in his book Superforecasting.
As the Pathways Model shows, the conscious biases that experts use to solve problems reinforce cognitive biases, and these restrict a realistic appraisal of future probabilities. The complex nature of strategic thinking makes an awareness of the cognitive biases involved, and a sound grasp of the techniques that can be used to mitigate them, essential to making good strategic decisions.
Measuring cognitive biases
There are nearly 150 recognised cognitive biases. The original approach to measuring them in individuals involved 8 weeks of observation over a prolonged series of simulations. This is impractical outside of highly specialised areas – such as bomb disposal.
Macleod, Matthews and Tata developed a basic measurement of bias toward optimism or pessimism in 1986. A version of this test, the dot-probe task, can be taken on line. This is a useful, simple test but it does not go into the cognitive bias issue in any depth.
Measurements in the field of risk intelligence – which looks at some wider aspects of the cognitive bias picture, have been developed by Dylan Edwards. These are explained in his excellent book Risk Intelligence – how to live with uncertainty. His Risk Intelligence Test can also be taken online.
The on-line Cognitive Bias Test is designed to look for all cognitive biases that tend to affect senior leaders. This test makes the measurement of cognitive biases in leaders and their senior teams relatively straightforward, but dealing with them is more challenging. It requires mental effort and a systematic approach, but one piece of good news relates to what happens at the end of the Pathways Model. Behaviours lead to outcomes, and better outcomes are a product of reduced cognitive biases. Better outcomes lead to better experiences which, when coupled with an increased understanding of the workings of the mind, can help reduce the grip that cognitive biases have over the next decision.
Mitigating cognitive biases
Some research has concluded that it is difficult to remove cognitive biases – that they are highly persistent and will always remain. We would agree that the process is difficult, and that cognitive biases always exist, but would strongly argue that awareness, coupled with specific techniques, particularly when individually tailored to deal with each category of biases, do have a marked impact on reducing the effects of cognitive biases on executives’ strategic thinking. We would also argue that unconscious biases are similarly difficult to eradicate, and would firmly agree that there is still more work to be done in this area, but it remains a fact that huge strides have been made in dealing with these apparently intractable biases over the last 40 years.
Our cognitive bias mitigation techniques were pioneered over decades of helping counter terrorist bomb disposal leaders overcome their effects on thinking. These have been modified over recent years and refined for use with senior executives in the corporate world.
Workshops
Workshops are conducted for Senior Leadership Teams by qualified Cognitive Bias Test (CBT) facilitators.
- Cover the benefits and drawbacks of the individual biases found in the CBT report.
- Explore and practice strategies to overcome these individual biases.
- Cover the benefits and drawbacks of the team biases found in the CBT report.
- Explore and practice strategies to overcome these team biases.
- Explore and seek to explain differences between individual perceptions of the team’s biases and the group’s perceptions of the team’s biases, as identified by the CBT report.
Workshops are conducted worldwide at a location of the team’s choosing. Cognitive biases can be strongly reinforced by a familiar location (such as corporate offices) and off-site venues for these workshops are strongly recommended. A typical workshop lasts for one and a half to two days, with individual follow-ups in the subsequent months conducted by telephone.
Useful links
Insight Leadership provide qualified facilitators for CBT report debriefs, and associated strategic thinking workshops. .
» Insight Leadership Website
A simple test related to cognitive bias is known as the dot-probe task. This tests for an optimistic or pessimistic bias and is available from McGill:
» Dot-Probe demonstration
The Harvard Implicit Association Test surveys unconscious biases in several areas.
» Project Implicit
Dylan Evans’ Risk Intelligence Test can be found at:
» Risk Intelligence Test
The Lovallo and Sibony article published by McKinsey, which gives the original classification of the five types of biases surveyed by the CBT, can be found here:
» The case for behavioral strategy
The Harvard Business Review has a series of articles on cognitive biases which can be accessed here:
» The Magazine – HBR
A sample Cognitive Bias Test report can be downloaded from here:
» Download Cognitive Bias Test sample report
Dominic Brittain’s show reel gives a good overview of his background:
» Watch Dominic Brittain Showreel
FAQs
How much does it cost?
Workshops are charged at US$5,000 per day per facilitator plus expenses (which are charged at cost). One additional day is billed for preparation, which includes the CBT reports for each participant.
Can we just get the CBT report without the workshops?
No. The issues involved are complex and require a facilitated workshop to ensure that both the biases identified, and the mitigation techniques that should be used, are thoroughly understood.
Can a workshop be arranged anywhere in the world?
Broadly, yes. Workshops are provided on all continents except Antarctica.
How long does it take to prepare a CBT Report?
The longest period of time is usually collecting the responses from the executives (typically 10 days). Once they have all completed the survey, the CBT reports are typically sent out to each workshop participant in 5 working days.
What does a CBT report show?
Each report shows the executive’s individual cognitive biases map, their view of the Senior Leadership Team’s biases and the team’s view of its biases. These are displayed in three easy to read, proprietary bias maps. The report also contains background information on cognitive biases and the test, and explores the benefits and drawbacks of each category of cognitive biases. A sample CBT report can be downloaded from the useful links section.
How do executives take the Cognitive Bias Test?
A link is sent to each executive who then takes the CBT on line. It typically takes 20 minutes to complete.
Can executives retake the CBT after a period of time to see if their biases have changed?
Yes, and this is excellent practice. Costs for retests are reduced, with the debrief being typically conducted by conference call. A minimum period of 6 months is recommended between tests.
Are the results of the CBT confidential?
Yes, individual and team CBT results are never disclosed or discussed with any other party. Senior leadership team executives discuss their reports with each other during the workshops, but answers to individual questions are not disclosed, either in the report or to any person. The pattern of the data is what the report measures and shows, not individual responses to questions.
Can the CBT be used effectively for more junior executives?
No, the CBT and the workshops are designed for more senior executives working on the strategic level. Typically for a large Multi-National Corporation (MNC) this would be defined as the country manager/MD and one level below. For some clients we have had success with one level below that, but strongly do not recommend that it is used with more junior executives.
Is the CBT available in other languages?
No, the CBT is only available in English. Measuring cognitive biases is a tricky process, and experience has shown that it does not survive translation well. For the level of executives the CBT is designed for, English language skills are not normally an issue.
Can individual executives take the CBT?
Yes. The full CBT measures both individual cognitive biases and team biases. It requires all executives in a leadership team to take the CBT within a reasonable time frame (typically 10 days). However, the Individual Cognitive Bias Test (ICBT) only measures individual cognitive biases. Debriefing sessions following the test can be conducted in person or by telephone call. ICBT reports only provide one bias map, and while the data is valuable, it is less insightful that the full CBT report, which provides the standard three bias maps.
Further reading
One of the iron rules of cognitive bias mitigation is: read more than one book. The following give a useful, alternative view of the cognitive bias problem.
Daniel Kahneman is one of the giants in the field of cognitive biases. His book Thinking Fast and Slow is a highly readable insight into the research he conducted with another of the greats Amos Tversky. As a bonus, their original research paper Judgement Under Uncertainty: Heuristics and Biases is included at the back of the book. If you read nothing else on cognitive biases, read that.
Philip Tetlock’s book Superforecasting: The Art and Science of Prediction is a must read if you are involved in strategic thinking.
Dylan Edwards’ book Risk Intelligence – how to live with uncertainty is a practical and fun introduction to the concept of risk intelligence and how you can improve your understanding of it. This is highly recommended for all leaders – not just those working at the strategic level.
Dan Ariely’s book Predictably Irrational contains some useful insights, and details some interesting, elegant experiments – others less so. It inadvertently provides a good example of the dangers of over-using college students for psychological testing and research. Worth skimming.
While not specifically about cognitive biases, Angela Duckworth’s work on grit as a desirable characteristic is thought provoking. Her grit questionnaire can be a useful development tool for up and coming leaders. Her book is GRIT The power of Passion and Perseverence – good material but in danger of lapsing into the trite of the self help genre. Like Ariely’s book, it is worth dipping into.
Enquiries
For further information on Cognitive Bias Tests and workshops please contact enquiries@cognitivebiastest.com