Author(s): Michael McFarland, S.J.
Originally published by the Markkula Center for Applied Ethics
Wayne Davidson is a software engineer in the aerospace division of Occidental Engineering, a large engineering firm. For the past two years he has been working as a test engineer for Operation Safe Skies, a project to build a prototype of the next generation air traffic control system. This project, which is funded by a contract from the Federal Aviation Agency (FAA), is a very important one for Occidental. With all the cutbacks in defense spending, the...
Wayne Davidson is a software engineer in the aerospace division of Occidental Engineering, a large engineering firm. For the past two years he has been working as a test engineer for Operation Safe Skies, a project to build a prototype of the next generation air traffic control system. This project, which is funded by a contract from the Federal Aviation Agency (FAA), is a very important one for Occidental. With all the cutbacks in defense spending, the aerospace division has been losing business. The Safe Skies project has provided much needed business, and could lead to a much larger contract if successful. Mindful of its strategic importance, the company had bid very aggressively for the original contract. In fact they had "low-balled" it, bidding less than it would take to do the work properly. They felt that was the only way they could beat out their competitors, who were just as hungry for the work. Because of their somewhat shaky financial position, the company was not willing to take a loss on the project, so the project has been underfunded and understaffed. Nevertheless those working on the project have made a heroic effort, working eighteen hour days seven days a week to meet the deadline, because they know how much it means to the company, not to mention their own jobs. They are now very close to success.
A version of the prototype has been completed and turned over to Wayne for testing. He has run extensive simulations on it and found that it works as it should except for one little problem. When there are too many aircraft in the system, it will sometimes lose track of one or more of them. The "forgotten" aircraft will simply disappear from the screen, there will be no trace of it anywhere, and it will be ignored by all of the collision avoidance and other safety tests. Wayne has been working with the software designers to identify the cause of the problem, and they have traced it to a subtle error in memory allocation and reuse. They are confident that they can fix it, but it will take a month or more to do the redesign, coding and testing.
Wayne meets with his boss, Deborah Shepherd, the project manager, to discuss the implications. She tells him that what he is asking for is impossible. The contract requires that the company deliver a fully certified, working version of the software in three days for system integration and test. The government has developed a new, get-tough policy on missed deadlines and cost overruns, and Occidental is afraid that if they miss this deadline, the government will make an example of them. They would be subject to fines and the loss of the remainder of the prototype contract; and they might not be allowed to bid on the contract for the full system. This would have a devastating effect on the aerospace division, resulting in thousands of lost jobs.
They consider whether they can do a quick patch to the software before turning it over, but Wayne adamantly refuses to release any code that has not been tested thoroughly. There is always a chance that the patch would interact with some other part of the program to create a new bug.
"Then we'll have to deliver the software as is," Deborah says. "I can't jeopardize this project or the jobs of my people by missing that deadline."
"We can't do that!" exclaims Wayne. "That's like delivering a car with defective brakes."
"Don't worry," Deborah reassures him. "We have contacts in the FAA, so we know their testing plans. They will do a lot of simulations to make sure the software works with the hardware and has all the functionality in the specs. Then they will do live tests, but only at a small airport, with a backup system active at all times. There is no way they will overload the system in any of this. After that they will have some change requests. Even if they don't, we can give them an updated version of the program. We can slip the bug fix in there. They will never see the problem. Even if they do, we can claim it was a random occurrence that would not necessarily show up in our tests. The important thing is no one is in any danger."
"Maybe they won't find the bug, but I know it's there. I would be lying if I said the system passed all the necessary tests. I can't do that. Anyway, it would be illegal and unprofessional."
"You can certify that it is safe, because it is, the way they are going to use it."
And so he does. In the end Wayne signs off on the software. It is delivered to the FAA and makes it through all the preliminary tests, including live tests at a small airport in the Midwest. As a result of these tests, the FAA requests some changes in the user interface, and when Occidental delivers the new software it includes a robust solution to the problem of the disappearing aircraft. No one outside of Deborah's group ever learns of the problem. In fact Occidental's success with the prototype leads to major contracts for air traffic control software, giving much-needed business to the aerospace division. This saves hundreds of jobs, and allows the company to add hundreds more.
Wayne Davidson, however, takes early retirement once the prototype project is finished, in order to write a book on software testing. He feels that the book should have a chapter on ethics, but he can never bring himself to write it.
What do you think about Wayne's decision? Was it ethical?
Next: Tutorial on Ethical Decision Making
Michael McFarland, S.J., a computer scientist, is the former president of College of the Holy Cross and was a visiting scholar at the Markkula Ethics Center. June 2012
Author(s): Michael McFarland, S.J.
Originally published by the Markkula Center for Applied Ethics
Why is there so much disagreement on ethical judgements, if we have these well-established ethical principles? How, for example, can Wayne and Deborah come to such different conclusions about reporting unfavorable test results in the case described earlier? Clearly the above principles do not always lead to clear and unambiguous solutions to ethical issues.
There are a number of reasons why reasonable people of good will can differ in their ethical judgements, even when they agree on the basic principles.7 First of all, there can be empirical differences, that is, different readings of the facts of the case. Sometimes they disagree about the facts themselves: who fired first in a confrontation, for example, or how many people were killed in a battle. Even more often, they differ on the implications, especially in predicting the consequences of certain policies or actions. In the Occidental case, Deborah believes that it is completely safe to release the flawed system because its flaws will never show up in the environment where it will be used. Wayne seems more skeptical. More aware, perhaps, of how often systems fail because of unanticipated circumstances, he does not see the operating environment as quite so benign as Deborah does.
Another problem is that the principles are very general, and therefore need a great deal of adaptation and interpretation when they are applied to individual acts and policies. For example, what does it mean to tell the truth? Does it mean always telling the all the facts exactly as they are, as Wayne insists, or just avoiding significant deceptions, as Deborah seems to imply? What does it mean to avoid harming someone? Does it refer only to bodily harm, or does it include intangibles such as freedom or privacy? Is it limited only to direct, immediate harm to a well-defined victim, or does it also include indirectly increasing the chance of harm to an unspecified population? In war, is bombing a city's water supply, which will lead to widespread disease and death, the same as machine-gunning civilians in the streets?
Other differences arise because the basic ethical duties sometimes conflict when applied to a particular situation. They cannot all be satisfied in all cases, so it is necessary to prioritize them and decide to what extent each one must be observed. In our example, Wayne insists strongly that the duty to tell the truth, which comes from the duty of fidelity, is paramount and must not be violated. Deborah puts much more emphasis on the duty to avoid harm to others, especially her employees who could lose their jobs if the full truth is disclosed. As another example, a government might adopt economic and social policies that bring prosperity to much of its population but leaves rural peasants largely impoverished and on the margins of society, thus putting the duty to do good to a large number of people before the duty to show justice by ensuring that all participate equally in the goods of the society. On the other hand, a rebel group might seize the land of wealthy landowners by violent means and redistribute it to the peasants, giving the demands of justice higher priority than the duty to do no harm. There can also be differences in the priorities given to different groups affected by a decision. For example, Deborah seems to put her company and her employees first, while Wayne is more sensitive to the interests of their customers, including both the FAA and the ultimate users of the system.
Politicians often have to choose between the narrow interests of their constituents and the good of the country as a whole, or of the international community. This occurs, for example, when a member of Congress must decide whether to vote for a "pork barrel" project that will help the economy of his or her district but is a waste of the taxpayers' money from every other standpoint. Finally, legitimate disagreements can arise because of different modes of ethical reasoning. This refers to the methodology used to interpret and apply the fundamental ethical principles in particular circumstances, to reason from the general principles to specific ethical judgements.
A number of different systems have been proposed; and while these complement one another in some ways, they can lead to different conclusions in some cases. We will study these different methodologies in the next section.
Return to Occidental Engineering Case Study
Next "Ethical Reasoning: Part 5"