Thursday, July 12, 2007

Overcoming bias

Not strictly virtualization-related, but overcoming bias is one of the more consistently interesting cross-disciplinary blogs I've come across. The same sorts of cognitive biases that cause people to make bad decisions about their happiness, finances, political preferences, etc., often cause errors in software engineering. Pawing through Wikipedia's list of cognitive biases, I can associate almost every single bias with one (or more) mistakes I've made in my professional life. Optimism bias? Zounds, what engineer isn't guilty of it on a daily bias? Information bias has often led me to waste hours in debugging collecting useless trivia. Some have even entered the lore of computing; ingroup bias is nothing but "NIH syndrome."

While "overcoming bias" is certainly a laudable goal in many professional fields, I wonder about its tractability. These biases are clearly hardwired, and I suspect no amount of mindfulness will prevent us from committing them. Getting rid of one of these well-attested biases would probably require the neurological equivalent of, e.g., making humans able to digest grass, or giving a human the VO2Max of a horse. Oh well; if nothing else, at least being aware of our limitations as thinking machines might prevent us from trying anything too impossibly stupid...


Blogger Robin Hanson said...

Keith, once we appreciate just how biased we are, we are more willing to create social institutions, like betting markets, that discourage bias.

7:00 AM  
Blogger Keith Adams said...

Cool! Mr. Overcoming Bias himself weighs in...

So, how does one apply betting markets to software engineering? Or diagnostic medicine, etc.? Since I work at a publicly traded company, perhaps we can say that there's already a "market" conception of how well I can really do my job, reflected in our share price. This is so far removed from the day-to-day, tactical decisions where bias creeps in that it doesn't provide much of a counterweight.

Colleagues of mine have suggested, quite seriously, that we schedule software projects with a betting pool: then, engineers will be motivated to provide realistic estimates rather than optimistic ones.

9:44 AM  

Post a Comment

<< Home