Software Bugs That Kill

You might remember the rash of unintended acceleration incidents that occurred in Toyota vehicles a few years ago.  Perhaps the worst incident happened near me in Southlake, Texas where four people were killed.  I remember thinking at the time that these incidents had all the indicators of a software problem.  Well it turns out that is most likely the case.  Research from an embedded software expert as part of an Oklahoma trial indicates that a stack overflow may be responsible. 

The Toyota issue reminded me of the story of the Therac-25.  Every computer science student should be required to read it.  The Therac-25 was a medical linear accelerator that used electrons to create high-energy beams to destroy tumors in cancer patients.  Eleven of these devices were built and used in the 1980s. Software bugs in the Therac-25 caused massive overdoses of radiation that killed patients.

Here are some quotes from the story.  It reads like a novel.

she felt a "tremendous force of heat . . . this red-hot sensation." When the technician came in, the patient said, "You burned me." The technician replied that that was not possible.

She completely lost the use of her shoulder and her arm, and was in constant pain. She had suffered a serious radiation burn, but the manufacturer and operators of the machine refused to believe that it could have been caused by the Therac-25.

the patient said that he felt like he had received an electric shock or that someone had poured hot coffee on his back: He felt a thump and heat and heard a buzzing sound from the equipment. Since this was his ninth treatment, he knew that this was not normal. He began to get up from the treatment table to go for help. It was at this moment that the operator hit the "P" key to proceed with the treatment. The patient said that he felt like his arm was being shocked by electricity and that his hand was leaving his body.

Software quality is really important.  The reality is that some bugs can lay hidden for a very long time because they surface only under a very rare set of circumstances.  A race condition (multiple threads competing for the same resource) is a good example of this.  Another example is the security flaw in MySQL that allowed a 1 in 256 chance of *any* password to work.

Fortunately, most developers don't write code that can cause direct bodily harm, but I think it's good to be familiar with these types of cases and hopefully avoid repeating history.