Disciplined Thinking

Figuring out how to make the act of decision-making “commensurate with the complexity and importance of the stakes” is a huge problem, in Kahneman’s view, to which the business world does not devote much thought. post

He cited some disturbing evidence about the professional judgment of experts: “You put the same X-ray in front of radiologists, and about 20% of the time in some experiments they don’t reach the same diagnosis.”

His recommendation is to sit down with a committee of people who are knowledgeable about the situation and make a list of five or six dimensions. More than eight is probably unnecessary. “If you create good ranking scales on those dimensions, and give them equal weight, you will typically do just as well as with a very sophisticated statistical algorithm.” And do just as well — typically much better — than experts on the average, he added.

“Global rating is very good — and intuition is very good — provided that you have [first] gone through the exercise of systematically and independently evaluating, the constituents of the problem,” he explained.

He is concerned that as AI becomes more sophisticated, it is moving beyond simply helping humans achieve disciplined thinking to actually being able to execute professional judgment on its own.

“All of this depends on the availability of data: this is how intuition develops,” he noted. “We develop intuition with the data we collect in a lifetime. AI will be able to do better.