Richard G. Epstein, Westchester University of Pennsylvania
Mike Melamed, CWRU 2000
The Case of the Killer Robot is a detailed scenario that combines elements of software engineering and computer ethics.
The scenario consists of fictitious articles that touch on specific issues in software engineering and computer ethics. The articles discuss programs such as programmer psychology, team dynamics, user interfaces, software process models, software testing, the nature of requirements, software theft, and privacy. A major consideration is "when is the software good enough?"
The articles in the scenario begin with the indictment for manslaughter of a programmer who wrote faulty code that caused the death of a robot operator. Slowly, over the course of many articles, students are introduced to factors within the software company that also contributed to the accident. They are shown software development as a social process. It is hoped that students will begin to realize the complexity of the task of building real-world software and to see some of the ethical issues intertwined in that complexity.
This scenario is about 70 pages long and includes some tongue-in-cheek humor.
For permission see author information.
Special to the Silicon Valley Sentinel-Observer
Silicon Valley, USA
Sources close to the development of Silicon Techtronics' Robbie CX30, the so-called killer robot that caused the death of an assembly line worker last May, have told the Sentinel-Oberver that the project was controversial from the beginning. Two groups, committed to different software development philosophies, nearly came to blows during the initial planning meetings for the Robbie CX30, nearly two years ago. At issue was whether the Robbie CX30 project should proceed according to the "waterfall model" or the "prototyping model" of software development.
The waterfall and prototyping models are two common methods for organizing a software project. In the waterfall model, the software goes through definite stages of development. In the first stage, called requirements analysis and specification, an attempt is made to agree upon the detailed functionality of the system. As the project passes from one stage to the next, there are limited opportunities to change earlier decisions. A drawback of this approach is that potential users are not able to interact with the system until very late in the process.
In the prototyping model, great emphasis is placed on producing a working model or prototype early in the process. The prototype is built to help arrive at a final specification of the functionality of the proposed system. Potential users interact with the prototype early and often until the requirements are agreed upon. This approach allows potential users to interact with the prototype system long before the final system is designed and coded.
In a memo dated December 12 of the year before last, Jan Anderson, a member of the original Robbie CX30 project team, bitterly attacked project manager Sam Reynolds' decision to employ the waterfall model. The Sentinel-Observer has obtained a copy of Anderson's memo, which is addressed to Reynolds, and Anderson verified the authenticity of the memo for this reporter. Reynolds fired Anderson on December 24 of the same year, two weeks after she wrote the memo.
The Anderson memo refers to a meeting on December 11, during which an angry exchange occurred relating to software development philosophy. Anderson underlined the following passage in her memo: "I did not intend to impugn your competence at our meeting yesterday, but I must protest most vehemently against the idea that we complete the Robbie CX30 software following the waterfall model, which you have used in previous projects. I need not remind you that those were data processing projects involving the processing of business transactions. The Robbie CX30 project will involve a high degree of interaction, both between robot components and between the robot and the operator. Since operator interaction with the robot is so important, the interface cannot be designed as an afterthought."
Randy Samuels, the programmer who has been charged with manslaughter in the death of robot operator Bart Matthews, was in attendance at the December 11 meeting. In a conversation in the living room of her suburban townhouse, Anderson said that Samuels had not had much to say about the waterfall-prototyping controversy, and that she would give her "eye teeth" to have Samuels exonerated.
"The project was doomed long before Samuels misinterpreted those formulas," Anderson said emphatically.
Anderson did her best to explain the waterfall-prototyping controversy in lay terms: "The main issue was really whether we could agree on the system requirements without allowing actual robot operators to get a feel for what we had in mind. Reynolds has been in the data processing business for three decades and he's good at that, but he never should have been made manager of this project."
According to records obtained by the Sentinel-Observer, Silicon Techtronics moved Sam Reynolds from the data processing division, which takes care of inventory and payroll, to the robotics division just three weeks before the December 11 meeting referred to in Anderson's memo.
Reynolds was transferred to the robotics division by Silicon Techtronics president Michael Waterson. Reynolds replaced John Cramer, who managed the earlier Robbie projects, CX10 and CX20. Cramer was placed in charge of the CX30 project, but he died unexpectedly in a sky-diving accident. By placing Reynolds in charge of the CX30 project, Waterson was going against the advice of the robotics division chief Ray Johnson, sources say. According to these sources, Johnson strongly opposed the choice of Reynolds to head the Robbie CX30 project. These sources tell the Sentinel-Observer that Waterson's choice of Reynolds was purely a cost-saving decision. It was cheaper to move Reynolds to the robotics division than to hire a new project leader from outside the corporation.
The anonymous source that the Sentinel-Observer calls "Martha" described the situation in this way: "Waterson thought it would be cheaper to move Reynolds to robotics rather than try to find a new manager for the Robbie project from outside. Also, Waterson tended to be suspicious of people from the outside. He often sends down memos about how long it takes people to master the Silicon Techtronics way of doing things. In Waterson's view, Reynolds was a manager and he was moved to his new position in robotics as a manager and not as a technical expert. Clearly, Reynolds saw himself as both a manager and as a technical expert. Reynolds was not aware of his own technical limitations."
According to Martha, Reynolds was very reluctant to manage a project that would not use the waterfall model, which had served him so well in data processing: "He attacked prototyping as a fad at the meeting on December 11, and after a few verbal exchanges back and forth things got pretty personal."
"Anderson was especially vocal," Martha recalled. "She had lots of experience with user interfaces, and from her perspective the operator-robot interface was critical to the success of CX30 since operator intervention would be frequent and at times critical."
In her interview with the Sentinel-Observer, Jan Anderson also commented on this aspect of the December 11 meeting: "Reynolds was vehemently opposed to wasting time -- to use his words -- on any kind of formal analysis of the user interface and its human factors properties. To him, user interfaces were a peripheral issue."
"Anything new was a fad to him," Anderson added. "Computer interfaces were a fad, object-oriented design was a fad, formal specification and verification techniques were a fad, and most of all, prototyping was a fad."
Exactly one week after the December 11 meeting, the Robbie team received a memo from Sam Reynolds concerning the project plan for the Robbie CX30. "It was the waterfall model, right out of a textbook," Anderson said as she reviewed a copy of the project plan memo. "Requirements analysis and specification, then architectural design and detailed design, coding, testing, delivery and maintenance. In Reynolds' view of things, there was no need to have any user interaction with the system until very, very late in the process."
The Sentinel-Observer has learned that the first operator actually to use the Robbie CX30 robot in an industrial setting was Bart Matthews. This initial use of the Robbie CX30 in an industrial setting was covered by the media, including this newspaper. In a great irony, the Silicon Techtronics Annual Report for Shareholders, published last March, has a picture of a smiling Bart Matthews on its glossy front cover. Matthews is shown operating the very same Robbie CX30 robot that crushed him to death barely two months after the photograph was taken.
The site is updating and expanding and is currently in beta form. We encourage you to browse and see all that the Online Ethics Center has to offer. We are actively working to improve your experience on the site and hope you find what you are looking for.
We would be delighted to hear your thoughts on the site reorganization, and if you need help finding a specific resource, please contact us via email at firstname.lastname@example.org.