The Ethical Analysis of Risks and Potential Benefits in Human Subjects Research (Research Involving Human Participants V2)

Author(s): Charles Weijer, Dalhousie University

Summary

This paper addresses three questions central to the ethical analysis of risks and potential benefits in human subjects research:

  1. How was the ethical analysis of risk understood by the members of the U.S. National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research (National Commission)?
  2. What conceptual framework should guide the ethical analysis of risk?
  3. What changes to U.S. regulations would the implementation of such a framework require?

Three distinct views on the ethical analysis of risks and potential benefits in research can be found in the National Commission's opus: analysis of entire protocols; analysis of protocols with particular components; and analysis of components. Basing the moral analysis of risk on the categorization of research into therapeutic or nontherapeutic research poses two problems: 1) therapeutic research is a contradiction in terms, and 2) research subjects are inadequately protected, as any number of procedures not for the benefit of subjects may be added to a therapeutic study. Recognizing these shortcomings, the National Commission adopted an analysis of risk that focused on whole protocols with particular components. New problems arise with this approach. Little guidance is given for the analysis of research that presents less than minimal risk; the concept of minimal risk is applied to both therapeutic and nontherapeutic procedures but sets a threshold for allowable risk only for nontherapeutic procedures; and, since clinical research often contains a mixture of procedures, differing rules for whole protocols may apply simultaneously, leading to confusion and conflict.

The ethical analysis of the various components in a research study seems to present a number of advantages:

  1. It acknowledges that clinical research often contains a mixture of procedures, some administered with therapeutic intent and others solely to answer the research question.
  2. Therapeutic procedures and nontherapeutic procedures are, by definition, administered with differing intent. This difference is morally relevant.
  3. Therapeutic procedures are justified by their potential to benefit the subject, while nontherapeutic procedures are justified by their potential to generate knowledge. These two types of benefit are largely incommensurable.
  4. Rigorous separate moral calculi for therapeutic and nontherapeutic procedures protect research subjects better than previous approaches. They prevent the justification of risky nontherapeutic procedures by the benefits that may flow from therapeutic procedures that are components of the same study.
  5. It is a parsimonious model for analysis and thereby avoids confusion and conflict.

The model advocated in this paper establishes the separate ethical analysis of therapeutic and nontherapeutic procedures in research. Therapeutic procedures are those study interventions administered with therapeutic intent. The Institutional Review Board (IRB) must ensure that such procedures fulfill the requirements of clinical equipoise - that is, they must ensure that, at the start of the study, genuine uncertainty exists in the community of expert practitioners as to the preferred treatment. Nontherapeutic procedures are not administered with therapeutic warrant and are administered in the interest of answering the research question. The IRB must ensure that the risks associated with such procedures are 1) minimized and 2) reasonable in relation to the knowledge to be gained.

Minimal risk is a concept through which the risks of nontherapeutic procedures are compared to the risks of daily life. Minimal risk is used in regulation as a sorting mechanism and as a protection for the vulnerable. As a sorting mechanism, minimal risk is used to direct the attention of the IRB to risky research. Protection of the vulnerable is, however, its most important role. Groups may be vulnerable for one or more of three reasons: they may be unduly susceptible to risk; they may be incapable of providing informed consent to study participation; or they may be in circumstances that throw the voluntariness of their consent into question. Protections for vulnerable groups include ensuring that the study hypothesis requires the inclusion of the vulnerable group; seeking consent from a proxy decisionmaker when subjects are incapable of giving consent; and limiting the amount of nontherapeutic risk to which subjects may be exposed to minimal risk or a minor increase over minimal risk.

A number of changes to the Common Rule and Department of Health and Human Services (DHHS) regulations are required if the regulations are to be consistent with this more comprehensive framework for the ethical analysis of risk:

  1. Ambiguity in current regulations caused by a multiplicity of conceptual models must be eliminated. A single conceptual model should underlie all regulations for the protection of research subjects.
  2. Definitions for therapeutic and nontherapeutic procedures should be added.
  3. The IRBs general obligations regarding the ethical analysis of potential benefits and risks of research participation must be stated more clearly.
  4. The definition of minimal risk should be clarified.
  5. Standards for expedited review must be more rigorous.
  6. Regulations for the ethical analysis of risk in research on children should be greatly simplified.
  7. A new subpart detailing protections for incapable adults should be added.

Back to Top

Introduction

The IRB is a social oversight mechanism charged with the mandate of protecting research subjects. Performing this task competently requires that the IRB scrutinize informed consent procedures, the balance of risks and potential benefits, and subject selection procedures in research protocols. It may be that IRBs spend too much time editing informed consent forms and too little time analyzing the risks and potential benefits posed by research.1 This imbalance is clearly reflected in the research ethics literature. A review of articles published between 1979 and 1990 in IRB: A Review of Human Subjects Research, for example, reveals a large number of articles on informed consent and confidentiality (142 articles), and considerably fewer on risk-benefit assessment (40), study design (20), and subject selection procedures (5).2 The obligation to ensure that study participation presents a favorable balance of potential benefits and risks to subjects is central to upholding the ethical principle of beneficence and fulfilling the IRBs protective function.3 Some believe it to be the single most important determination made by the IRB. It ensures that potential research subjects - be they sick or well, young or old, capable or not - are presented with the option of entering a research study only when agreeing to study participation would be a reasonable choice.

Accordingly, the Common Rule requires that the IRB ensure that:

  1. Risks to subjects are minimized: (i) by using procedures which are consistent with sound research design and which do not unnecessarily expose subjects to risk, and (ii) whenever appropriate, by using procedures already being performed on the subjects for diagnostic or treatment purposes.
  2. Risks to subjects are reasonable in relation to anticipated benefits, if any, to subjects, and the importance of the knowledge that may reasonably be expected to result. In evaluating risks and benefits, the IRB should consider only those risks and benefits that may result from the research (as distinguished from risks and benefits of therapies subjects would receive even if not participating in the research). The IRB should not consider possible long-range effects of applying knowledge gained in the research (for example, the possible effects of the research on public policy) as among those research risks that fall within the purview of its responsibility (45 CFR 46.111(a)).

The moral analysis of risk is neither obvious nor intuitive. Rules, including those of the Common Rule, are not self-interpreting. They must be situated within a conceptual framework that facilitates their interpretation by the IRB. The articulation of a conceptual framework for the ethical analysis of risk might therefore be a project assisting IRBs in fulfilling their mandate - the protection of research subjects.

Of the analysis of risk in research, the authors of the Belmont Report observe that [i]t is commonly said that benefits and risks must be balanced and shown to be in a favorable ratio. The metaphorical character of these terms draws attention to the difficulty of making precise judgments.3 Unpacking these metaphors will occupy the bulk of this paper. We will focus on three questions:

  1. How was the ethical analysis of risk understood by the members of the National Commission?
  2. What conceptual framework should guide the ethical analysis of risk?
  3. What changes to U.S. regulations would the implementation of such a framework require?

The work of the National Commission receives special consideration in this paper. No other ethics body has had as much influence on the development of research ethics and regulation. As we shall see, pivotal conceptual advances in the moral analysis of risks and potential benefits can be traced back to work of the National Commission.

The last papers on the ethical analysis of risk written for a major U.S. ethics body were written almost 25 years ago by Robert J. Levine, a staff member and consultant to the National Commission. In The Boundaries Between Biomedical or Behavioral Research and the Accepted and Routine Practice of Medicine, Levine recognizes that many clinical studies are complex activities, involving both therapeutic and nontherapeutic procedures.4 In the second paper, The Role of Assessment of Risk Benefit Criteria in the Determination of the Appropriateness of Research Involving Human Subjects, he comprehensively describes the risks and benefits presented by research to subjects and society.5 He argues convincingly that quantitative approaches to risk analysis will, at best, be of limited use to the IRB.

As we shall see, there have been considerable refinements in our understanding of the ethical analysis of risk in the last 25 years. Nonetheless, this paper, solicited by yet another commission, the National Bioethics Advisory Commission (NBAC), relies heavily on the solid intellectual work that precedes it. All of the work of the National Commission is a source for learning; much of it should be preserved in our current understanding and regulation. Levnie's papers for the National Commission remain foundational in research ethics. This paper will assume familiarity with them.

Back to Top

Risks and Potential Benefits in Research Involving Human Subjects

Risk is a multidimensional concept involving both the probability and magnitude of harms to research participants.6 All too often, risk is equated with the magnitude of the outcome, e.g., death or serious disability. The proper ethical analysis of risk requires that both the magnitude of the harm and its probability of occurring be considered. A risk of death of one in one million is properly treated differently than a risk of death of one in ten. Benefit, on the other hand, is the magnitude of a positive outcome without reference to its probability.

In reference to the comparison of risks to benefits, reference is often made of the need to consider the risk-benefit ratio presented by study participation. This is not a parallel construction, and, hence, it is, strictly speaking, incorrect. One speaks accurately of harms and benefits or risks and potential benefits. Research subjects may be exposed to a broad array of risks and potential benefits as a result of study participation. Risk is not a concept exclusive to biomedical research; social science studies also present risks to participants. Indeed, there is a surprising degree of overlap between the kinds of risks presented in biomedical and social science research. As study methodologies continue to cross conventional disciplinary boundaries, we can expect increasing convergence in the risks and potential benefits in biomedical and social science studies. We will thus need to consider whether the moral calculi involved in risk assessment suffice for the assessment of risks of research in a variety of disciplines. Consider the risks to participants in the following four case studies:

Study A: Placebo controlled trial of a drug for people with acutely symptomatic schizophrenia.
The study involves schizophrenic patients who are newly hospitalized with acute symptoms of their disease.7 Despite the existence of effective treatment for such symptoms, patients are randomized to a new antipsychotic drug, a standard drug, or placebo. Patients are treated in hospital for four weeks, and a variety of psychometric scales are measured. Risks to subjects include the possibility that the new medication may have serious adverse effects, some of which may be irreversible; patients assigned to placebo will be deprived of needed treatment for a month; patients may suffer from continuing hallucinations or paranoia; patients may be at increased risk of suicide; and, finally, patients may pose a risk to others. (The ethics of placebo controlled trials in schizophrenia is discussed in detail elsewhere.8)
Study B: Hypnotic induction of partial deafness to see whether paranoid symptoms result.
Hypnotically suggestible but otherwise healthy college students are randomly allocated to three different hypnotic suggestions: partial deafness without awareness of the cause; partial deafness with awareness of the cause; and no deafness but an ear itch.9 The hypothesis is that persons in the first group, compared to the other two groups, will demonstrate more symptoms of paranoia. Subjects are assessed with a variety of measures, including psychometric scales and scoring of observed behavior. After evaluation, subjects are hypnotized again, debriefed at the end of the study, and reassessed at one month. The study poses a variety of risks to participants, including distress associated with paranoia and hearing loss, suicide, the possibility of harm to others, and uncertain sequelae from hypnosis. (Some of the ethical issues raised by this study are discussed elsewhere.10)
Study C: Questionnaire examining adolescent sexual practices.
The study involves the administration of a pencil and paper questionnaire to 400 Minneapolis high school students during regularly scheduled health classes.11 The survey seeks to document attitudes and behaviors related to HIV prevention. Accordingly, adolescent participants are asked whether they are sexually active, what types of sexual activity they have experienced (e.g., oral, vaginal, or anal intercourse), and the gender(s) of their partners. A variety of risks are presented by this study to participants: teachers or parents may become aware of undisclosed sexual activity; others may become aware of same-sex relationships; and participants might become aware that they are at risk of developing HIV. (The ethical issues raised by this study are thoroughly reviewed elsewhere.11)
Study D: Genetic epidemiology of BRCA1 and BRCA2 mutations in Ashkenazic Jews.
The BRCA1 and BRCA2 mutations are known to be associated with an increased risk of breast and ovarian cancer. The study seeks to determine what proportion of Ashkenazic Jews carry the mutations in question and what risk is conferred by them in a nonhigh- risk population.12 Participants who respond to advertisements will be asked to give a blood sample and fill out an epidemio-logical survey including questions on health, family history of cancer, and family members who might also be willing to participate. Personal identifiers will be destroyed before genetic tests are conducted, and test results will not be disclosed to participants. Risks to participants are the risks of a venipuncture, the risk of anxiety provoked by answering questions related to family history of cancer, and risks of genetic testing, including unwanted disclosure of risk, discrimination, and stigmatization. (A review of ethical issues in genetic epidemiology studies may be found elsewhere.13)

As illustrated by these four examples, research participation may expose the study participant to a wide spectrum of risks. Levine classifies risks into four categories: physical, psychological, social, and economic.6 Let us consider each briefly:

Physical risks.
The research subject may suffer bodily harm - minor or serious, temporary or permanent, immediate or delayed - as a result of his or her participation in the study.
Psychological risks.
Study participation may impact upon the research subject's perception of self and cause emotional suffering, e.g., anxiety or shame, or may induce aberrations in thought or behavior.
Social risks.
Research findings, or even study participation itself, may expose subjects to the possibility of insurance or employment discrimination or other forms of social stigmatization.
Economic risks.
Research subjects may directly or indirectly bear financial costs related to research participation.

So defined, risk is an inherently inclusive concept. As demonstrated by the above examples, a given study may present a variety of types of risk. For example, study C (sex questionnaire) posed both psychological and social risks. Furthermore, no category of risk is exclusive to medical or social science studies: study B (deafness and paranoia), a social science study, presented physical risks, and studies A (schizophrenia trial) and D (breast cancer genes), medical studies, generated psychological risks. Despite the various disciplinary backgrounds involved, all four of the study examples posed nontrivial risk to research subjects.

Levine provides a comprehensive description of particular potential benefits and risks presented to research subjects and society by biomedical and social science research, and the listing will not be repeated here.5

Back to Top

The Analysis of Risks and Potential Benefits in the Work of the National Commission

The first major question to be addressed regards how the ethical analysis of risk was understood by the members of the National Commission. The National Commission sat from 1975 to 1978 and issued a total of ten reports on differing aspects of human subjects research. The National Commission's work represents the first sustained in-depth exploration of the moral analysis of risk in research. As such, it has had a lasting influence on research ethics scholarship and federal regulation. Little recognized is the fact that the National Commission's views on risk analysis evolved over its four-year term. Three distinct views on the ethical analysis of risks and potential benefits in research can be found in the National Commission's opus: analysis of entire protocols; analysis of protocols with particular components; and analysis of components.

Six reports of the National Commission were selected for this analysis based on their impact on public policy and the perception by National Commission staff of the overall success of the report.14 These reports are Research on the Fetus (1975);15 Research Involving Prisoners (1976);16 Research Involving Children (1977);17 Research Involving Those Institutionalized as Mentally Infirm (1977);18 Institutional Review Boards (1977);19 and the Belmont Report (1978).3 What follows is a critical review of approaches to the ethical analysis of risks and potential benefits found in each of the reports. The interpretation is my own and is based on a review of the primary source documents.

The ethical analysis of risks and potential benefits presented by entire protocols.

Research on the Fetus was the first of the National Commission's reports. It was produced under several constraints.15 Congress required the completion of the report in only four months, and it imposed a moratorium on fetal research pending the completion of the report. Thus, Levine observes that:

As a consequence of these time constraints, the Commission completed its report, Research on the Fetus, before it had the opportunity to address the general conceptual issues in its mandate. If the conceptual clarifications had preceded the report, it is likely that the Commission would have developed substantially different recommendations.20

In the report, the National Commission defines research as the systematic collection of data or observations in accordance with a designed protocol (p. 6).15 The schema for risk analysis presented in Research on the Fetus relies on separating whole research proposals into two types: therapeutic research and nontherapeutic research. Therapeutic research is that which is designed to improve the health condition of the research subject by prophylactic, diagnostic, or treatment methods that depart from standard medical practice but hold out a reasonable expectation of success (p. 6).15 Nontherapeutic research, on the other hand, is not designed to improve the health condition of the research subject by prophylactic, diagnostic, or treatment methods (p. 6).15 Separate recommendations are presented for each type of study. Recommendation 1 addresses therapeutic research directed toward the fetus. Under this provision:

[t]herapeutic research directed toward the fetus may be conducted or supported, and should be encouraged, by the Secretary, DHEW, provided such research (a) conforms to appropriate medical standards, (b) has received the informed consent of the mother, the father not dissenting, and (c) has been approved by existing review procedures with adequate provision for the monitoring of the consent process (p. 73).15

Recommendation 4 outlines different ethical criteria for the assessment of nontherapeutic research. It states that:

[n]ontherapeutic research directed towards the fetus in utero (other than research in anticipation of, or during, abortion) may be conducted or supported by the Secretary, DHEW, provided (a) the purpose of such research is the development of important biomedical knowledge that cannot be obtained by alternative means, (b) investigation on pertinent animal models and non-pregnant humans has preceded such research, (c) minimal or no risk to the well-being of the fetus will be imposed by the research, (d) the research has been approved by existing review procedures with adequate provision for the monitoring of the consent process, (e) the informed consent of the mother has been obtained, and (f) the father has not objected to the research (p. 74).15

While there is intuitive appeal in categorizing studies as a whole, as either therapeutic or nontherapeutic, the validity of this approach has been criticized. Levine points out that this distinction invariably leads to deep conceptual problems. This is illustrated by inserting the National Commission's definition of research into its definition of therapeutic research, as Levine does here:

There is, of course, no such thing as a systematic collection of data or observations designed to improve the health condition of a research subject that departs from standard medical practice. Thus, the Commission developed recommendations for the conduct of a nonexistent set of activities.20

A further problem exists with this approach. The inclusion of one or more therapeutic procedures in a study will lead it to being identified as therapeutic research. Once this categorization has taken place, there is no limit to procedures without therapeutic intent that might be administered to research subjects. Thus, this approach not only leads to confusion, it leaves research subjects without adequate protection.

Levine correctly observes that all ethical codes, regulations, and commentaries relying on the distinction between therapeutic and non-therapeutic research contain serious errors.20 The Declaration of Helsinki, perhaps the source of the National Commission's approach to risk in Research on the Fetus, relies on the distinction. Article III.2 requires of nontherapeutic biomedical research that [t]he subjects should be volunteers - either healthy persons or patients for whom the experimental design is not related to the patient's illness. This would seem to require that research into the pathophysiology of disease be conducted (absurdly) on those who are either healthy or who have a disease other than that of interest. This sort of thinking continues to pervade the work even of well-known thinkers in research ethics. Baruch Brody, for instance, in his recent book The Ethics of Biomedical Research concludes that phase I chemotherapy studies are nontherapeutic research.21 Despite endorsing the Declaration of Helsinki, he fails to recognize the entailment that such toxic studies, posing a risk of death to participants, must be done on healthy volunteers or persons with some disease other than cancer.22 Despite its shortcomings, this approach to the ethical analysis of risk is found in current DHHS regulations on the protection of fetuses in research. The regulations divide research on the fetus into two categories: research to meet the health needs of the particular fetus, i.e., therapeutic research; and research for the development of important of biomedical knowledge, i.e., nontherapeutic research (45 CFR 208(a)). As this approach to the ethical analysis of risk is not found elsewhere in the federal Common Rule or DHHS regulations, one interpretation is that it is a historical artifact of Research on the Fetus in current regulation.

The ethical analysis of whole protocols with particular components.

Recognizing these problems with the distinction between therapeutic and nontherapeutic research, the National Commission largely abandoned the use of these terms in subsequent reports. In the preface to Research Involving Prisoners they state: The Commission recognizes problems with employing the terms therapeutic and nontherapeutic research, notwithstanding their common usage, because they convey a misleading impression (p. x).16 In Research Involving Prisoners the therapeutic research category is replaced with research on practices which have the intent and reasonable probability of improving the health and well being of the subject (p. xi).16 While cumbersome, this manner of speaking at least avoids the conceptual confusion pointed to by Levine supra. The National Commission recognizes that:

Additional interventions over and above those necessary for therapy may need to be done, e.g., randomization, blood drawing, catheterization; these interventions may not be therapeutic for the individual. Some of these interventions may themselves present risk to the individual - risk unrelated to the therapy of the subject (p. xi).16

Despite this, it remains unclear in the report just how one is to determine whether such nontherapeutic risks are at an acceptable level. Indeed, Recommendation 4 merely states, in part, that [a]ll research involving prisoners should be reviewed by at least one human subjects review committee or Institutional Review Board. [T]he committee or board [IRB] should consider at least the following: the risks involved (p. 20).16 Clearly, IRBs require more detailed guidance on the ethical analysis of risks and potential benefits in research than is provided in Research Involving Prisoners.

It has been suggested that this and other failings of the report may be due to the fact that members of the National Commission confused the need for prison reform with the need for protection of prisoners in research.23 Be this as it may, the report does contain early ruminations about the notion of minimal risk. Minimal risk is referred to in Research on the Fetus, but only in Research Involving Prisoners does one see recognizable beginnings of what would become a central concept in the moral analysis of risk. A standard similar to that of minimal risk is articulated for research without therapeutic procedures:

Research designed to determine the effects on general health of institutional diets and restricted activity, and similar studies that do not manipulate bodily conditions (except innocuously, e.g., obtaining blood samples) but merely monitor or analyze such conditions, also present little physical risk and are necessary to gain some knowledge of the effects of imprisonment (p.15).16

Furthermore, there is an explicit recognition that in determining which risks should be acceptable, comparison is to be made between risks of research and those of daily life, in this case, the daily lives of persons who are not incarcerated:

The risks involved in research involving prisoners should be commensurate with risks that would be accepted by non-prisoner volunteers. If it is questionable whether a particular project is offered to prisoners because of the risk involved, the review committee might require that non-prisoners be included in the same project (p. 20).16

Both of these standards find expression in current DHHS regulations (45 CFR 306(a)(2)(A); 45 CFR 46.303(d)).

The concept of minimal risk is first fully expressed in the National Commission's report Research Involving Children.17 It is perhaps natural that the most detailed recommendations regarding the analysis of risks and potential benefits are found in this report. Levine explains that:

Because infants and very young children have no autonomy, there is no obligation to respond to it through the usual devices of informed consent. Rather, respect for infants and very small children requires that we protect them from harm. No discernable risk seemed to the commission to be virtually impossible; therefore, they stipulated a definition of minimal risk as the amount that would be acceptable without unusual standards for justification.24

The National Commission defines minimal risk as the probability and magnitude of physical or psychological harm that is normally encountered in the daily lives, or in the routine medical or psychological examination, of healthy children (p. xx).17 This definition differs from that found in the DHHS regulations in its stipulation of healthy children; DHHS does not so limit minimal risk (45 CFR 46.120(i)). The National Commission provides a number of prima facie examples of procedures that pose no more than minimal risk, including routine immunization, modest changes in diet or schedule, physical examination, obtaining blood and urine specimens, and developmental assessments [emphasis added] (p. 20).17 Again, this differs from DHHS regulation in its inclusion of a procedure, routine immunization, administered with therapeutic intent.

The concept of minimal risk is central to the schema for risk analysis presented in Research Involving Children. Recommendation 2 requires that the IRB ensure that [r]isks are minimized by using the safest procedures consistent with sound research design and by using procedures performed for diagnostic or treatment purposes whenever feasible (p. 2).17 Thus, if a blood sample is needed from a child, one should, where possible, use blood left over from a venipuncture done for therapeutic purposes. If the research does not involve therapeutic or nontherapeutic procedures that present more than minimal risk, it may be approved provided the above condition is fulfilled. Recommendation 3 states:

Research that does not involve greater than minimal risk to children may be conducted or supported provided that an Institutional Review Board has determined that: (A) the conditions of Recommendation (2) are met; and (B) adequate provisions are made for assent of the children and permission of their parents or guardians, as set forth in Recommendations (7) and (8) (p. 5).17

Separate recommendations, as follows, apply to research involving therapeutic or nontherapeutic interventions that exceed the minimal risk threshold.

If research involving a therapeutic intervention poses more than minimal risk, the IRB must ensure that the balance of potential benefits and risks of the intervention is at least as favorable as alternatives.

Recommendation 4 follows:

Research in which more than minimal risk to children is presented by an intervention that holds out the prospect of direct benefit for the individual subjects, or by a monitoring procedure required for the well-being of the subjects, may be conducted or supported provided that an Institutional Review Board has determined that:

  1. such risk is justified by the anticipated benefit to the subjects;
  2. the relation of anticipated benefit to such risk is at least as favorable to the subjects as that presented by available alternative approaches;
  3. the conditions of recommendation (2) are met; and
  4. adequate provisions are made for assent of the children and permission of their parents or guardians, as set forth in Recommendations (7) and (8) (pp. 56).17

In short, the IRB should evaluate such interventions in the same way as they are evaluated in clinical practice:

It should compare the risk and anticipated benefit of the intervention under investigation (including the monitoring procedures necessary for the care of the child) with those of available alternative methods for achieving the same goal, and should also consider the risk and possible benefit of attempting no intervention whatsoever (p. 7).17

If, on the other hand, the research involves a nontherapeutic intervention that poses more than minimal risk, the provisions of Recommendation 5 apply:

Research in which more than minimal risk to children is presented by an intervention that does not hold out the prospect of direct benefit for the individual subjects, or by a monitoring procedure not required for the well-being of the subjects, may be conducted or supported provided an Institutional Review Board has determined that:

  1. such risk represents a minor increase over minimal risk;
  2. such intervention or procedure presented experiences to subjects that are reasonably commensurate with those inherent in their actual or expected medical, psychological or social situations, and is likely to yield generalizable knowledge about the subject's disorder or condition;
  3. the anticipated knowledge is of vital importance for understanding or amelioration of the subject's disorder or condition;
  4. the conditions of Recommendation (2) are met; and
  5. adequate provisions are made for assent of the children and permission of their parents or guardians, as set forth in Recommendations (7) and (8) (pp. 78).17

Risks presented by nontherapeutic procedures are justified, therefore, in part by the importance of the knowledge to be gained from the research study as a whole. However important the knowledge, risks associated with the nontherapeutic interventions are effectively limited to a minor increase over minimal risk. (Risks exceeding this threshold require the approval of a National Ethics Advisory Board and the Secretary of the responsible federal agency [Recommendation 6].) The majority of the members of the National Commission defend this threshold for permissible risk as posing no significant threat to the child's health. The added requirement that such risks be commensurate to the child's experience ensures that such risks will be familiar. Such activities, then, would be considered normal for these children (p. 139).17 Importantly, if the research involves both a therapeutic intervention and a nontherapeutic intervention that exceed minimal risk, then both Recommendations 4 and 5 are to be applied by the IRB.

This provision (Recommendation 5) was the subject of the most enduring disagreement among members of the National Commission. Turtle dissented from the provision arguing that it should be impermissible to expose children to nontherapeutic procedures that pose more than minimal risk. He objected strenuously to the suggestion that sick children might be exposed to greater nontherapeutic research risk than healthy children:

Children, who through no fault or choice of their own, are subjected to greater risks incident to their condition or treatment, cannot ethically be assumed to qualify for additional increments of risk. To do so, is to add to the potential burdens that result, directly or indirectly, from the child's illness (p. 148).17

It scarcely needs to be observed that these provisions for the moral analysis of risk are complex. The recognition that a study may involve therapeutic procedures, nontherapeutic procedures, or both is a substantial leap forward over the schema for risk analysis found in Research on the Fetus. The members of the National Commission have solved both of the shortcomings associated with the attempt to classify research as therapeutic or nontherapeutic discussed supra. The solution nonetheless suffers from a number of problems of its own:

  1. The concept of minimal risk is applied to both therapeutic and nontherapeutic procedures in both the examples provided and in Recommendation 3. It is unclear, moreover, in what meaningful way minimal risk can apply to therapeutic procedures. According to Recommendation 4, therapeutic procedures that are more than minimal risk are justified as they are in clinical practice. In other words, there is no limit to the risk that may be posed by such procedures so long as they are reasonable in relation to potential benefits. Only nontherapeutic procedures should be subject to a threshold for permissible risk, such as a minor increase over minimal risk.
  2. The National Commission's use of the concept of minimal risk in the recommendations seems at odds with its definition. Recall that the National Commission defines minimal risk as risks commensurate to those of daily life of healthy children. Fixing the standard to the daily lives of healthy children seems designed to protect sick children from being exposed to more nontherapeutic research risks than healthy children. This presumed intention is contradicted by Recommendation 5 which allows nontherapeutic risks that are a minor increase over minimal risk so long as such intervention of procedure presents experiences to subjects that are reasonably commensurate with their experience (p. 7).17 Thus, a spinal tap done purely for research purposes may be permissible in a child with a neurological disorder in which such procedures are common, but not in a healthy child. The definition of minimal risk would be consistent with its use in this section if it omitted reference to healthy children, as is the case in current DHHS regulation (45 CFR 46.102(i)).
  3. Little guidance is provided for the analysis of risks and potential benefits for procedures that pose no more than minimal risk (Recommendation 3). Recommendation 2 requires that [r]isks are minimized by using the safest procedures consistent with sound research design (p. 2).17 This cannot, however, sensibly apply to risks posed by therapeutic procedures, as considerations of research design are largely irrelevant to them. One might reasonably ask the following question: What ethical test should the IRB apply to research involving a therapeutic procedure posing no more than minimal risk? No answer is forthcoming in this report.
  4. Research may involve both therapeutic and nontherapeutic procedures. Indeed, I think it is fair to say that this is often or always the case in clinical research. If a study involves a therapeutic intervention and a nontherapeutic intervention, then multiple recommendations may apply. The various possibilities are summarized in Table 1. If both procedures present only minimal risk, then only Recommendation 3 applies. If the therapeutic procedure is more than minimal risk but the nontherapeutic procedure is minimal risk, then Recommendations 3 and 4 apply. If the reverse, then Recommendations 3 and 5 apply. Finally, if both procedures present more than minimal risk, then Recommendations 4 and 5 apply. Since each of the recommendations refer to a research study as a whole involving a particular type of intervention, it is unclear how multiple recommendations are to be applied to a particular study. Without doubt, it is a cumbersome approach and, worse, it may easily lead to confusion or conflict.

Applicability of Differing Recommendations from Research Involving Children in a Mixed Clinical Study


 

Nontherapeutic
Procedure
Therapeutic procedure
No more than minimal risk More than minimal risk
No more than minimal risk Recommendation 3 only Recommendation 3 and Recommendation 4
More than minimal risk Recommendation 3 and Recommendation 5 Recommendation 4 and Recommendation 5

Despite these difficulties, the model for risk assessment found in Research Involving Children is clearly reflected in current DHHS regulations for the protection of children in research. Indeed, there is a one-to-one correspondence between certain regulations and recommendations made by the National Commission.

45 CFR 46.404, Research not involving greater than minimal risk, corresponds to Recommendation 3; 46.405, Research involving greater than minimal risk but presenting the prospect of direct benefit to the individual subjects, corresponds to Recommendation 4; 46.606, Research involving greater than minimal risk and no prospect of direct benefit to individual subjects, but likely to yield generalizable knowledge about the subject's disorder or condition, corresponds to Recommendation 5; and 46.407, Research not otherwise approvable which presents an opportunity to understand, prevent, or alleviate a serious problem affecting the health or welfare of children, corresponds to Recommendation 6. Note that the conceptual model for risk analysis underlying 45 CFR 46.404-407 differs from that underlying protections for the fetus, 45 CFR 46.208(a), noted supra.

The schema for the analysis of risks and potential benefits of research found in Research Involving Those Institutionalized as Mentally Infirm is essentially identical to that found within Research Involving Children.18

Accordingly, only a few comments need to be added at this point. The report refers primarily to persons who are both incapable of providing informed consent and institutionalized. It addressed problems of including such persons in research by incorporating elements of Research Involving Prisoners and Research Involving Children. The definition of minimal risk refers to the risk normally encountered in the daily lives of normal persons (p. 8).18 Thus, the risks associated with institutionalization may not be used to justify exposing subjects to greater research risks. Recommendations 1 through 5 map onto Recommendations 2 through 6 found in Research Involving Children, and they will not be further elaborated here.

Perhaps the most remarkable fact about the report is its failure to be translated into regulation. Like children, adults incapable of providing informed consent are a vulnerable population in need of protection. Some have suggested that this failure is in part the result of the reports exclusive focus on institutionalized incapable persons. On this point, the National Commission merely responded to the charge provided to it by the Congress. The President's Commission (19801983) repeatedly called for the entrenchment of protections for incapable adults in regulation.25,26 According to Levine:

The Secretary of the Department of Health and Human Services (DHHS) responded that, while continuing to consider specific issues regarding protections for institutionalized mental patients, the Department is not intending to issue additional regulations in the near future. He provided two justifications: first, that the rules proposed by the Department in November 1978 had produced a lack of consensus and, second, that the basic regulations on human subjects research adequately respond to the recommendations made by the National Commission to protect persons institutionalized as mentally disabled.27

This is a remarkable assertion considering the fact that DHHS regulations contain no special protections for incapable adults in research.

The ethical analysis of risks and potential benefits of components of a research study.

The final works of the National Commission are typified by a move towards a model of the analysis of risks and potential benefits of components of studies, be they therapeutic interventions or nontherapeutic interventions. The move is, however, incomplete. Previous work of the National Commission has focused on risk analysis for particular vulnerable populations. In Institutional Review Boards, members of the National Commission articulate for the first time ethical standards to apply to the review of all human subjects research. The report acknowledges explicitly that a protocol may contain therapeutic procedures, nontherapeutic procedures, or both:

A research project is described in a protocol that sets forth explicit objectives and formal procedures designed to reach those objectives. The protocol may include therapeutic and other activities intended to benefit the subjects, as well as procedures to evaluate such activities (p. xx).19

Risks must be analyzed systematically and should involve a procedure-by-procedure review of risks, benefits, and alternatives. In the words of the National Commission, [t]his evaluation should include an arrayal of alternatives to the procedures under review and the possible harms and benefits associated with each alternative (p. 23).19 The risks associated with particular procedures are acceptable only if risks to subjects are minimized by using the safest procedures consistent with sound research design and, wherever appropriate, by using procedures being performed for diagnostic or treatment purposes; [and] risks to subjects are reasonable in relation to anticipated benefits to subjects and importance of knowledge to be gained. (Recommendation 4; pp. 19 20).19

The Belmont Report surprisingly provides little additional detail with regard to this model for the ethical analysis of risk. It famously articulated three ethical principles guiding the conduct of clinical research: respect for persons, beneficence, and justice. Beneficence demands that one 1) do no harm and 2) maximize possible benefits while minimizing harms.19 The translation of this principle into practice requires that the IRB ensure that research participation presents subjects with a favorable balance of possible benefits and risks. The Belmont Report once again emphasizes that this is to be done in a systematic and rigorous manner:

the idea of systematic, nonarbitrary analysis of risks and benefits should be emulated insofar as possible. This ideal requires those making decisions about the justifiability of research to be thorough in the accumulation and assessment of information about all aspects of the research, and to consider alternatives systematically. This procedure renders the assessment of research more rigorous and precise, while making communication between review board members and investigators less subject to misinterpretation, misinformation and conflicting judgments.19

While it encourages the IRB to be explicit about precisely how risks and potential benefits are analyzed, it is itself not more explicit than the previous report, Institutional Review Boards.

Levine, a staff member and consultant to the National Commission, renders the thinking of the National Commission somewhat clearer in two papers contained within the appendix to the Belmont Report. In The Boundaries Between Biomedical or Behavioral Research and the Accepted and Routine Practice of Medicine, the existence of complex activities in research is recognized.4 Such activities involve procedures administered with different intent in the research setting. Some interventions may be administered for therapeutic purposes, while other procedures are done solely to answer a scientific question. It is this difference in intent that drives the ensuing moral analysis of components of research.

Levine illustrates this point in a lengthy but instructive example of just such a complex research study. In it he weighs the risks and potential benefits of each component of the research separately:

the benefit [to the subject] will ordinarily derive from those aspects of the complex activity that may be considered practice rather than research. For example, if one wishes to study the effects of chlorothiazide (a diuretic) on sodium balance in patients with congestive heart failure and if one selects subjects in whom chlorothiazide is indicated and administers the drug in appropriate doses, the subject may receive direct therapeutic benefit. This study might be accomplished in a metabolic research ward. It might involve a period of two or three weeks of eating a constant diet with precise control of sodium content. It might involve repeated sampling of venous blood and collection of all urine excreted during those two or three weeks for purposes of sodium assay. The patient may be expected to receive direct therapeutic benefit through administration of the drug; however, this is only technically a research intervention. The subject will not benefit ordinarily from repeated blood sampling and urine collection. The subject may or may not benefit from the constant diet; it might be more or less nutritious and/or palatable that the diet to which he [sic] is ordinarily accustomed. The subject might also benefit from a two or three week period of relative rest on a metabolic research ward. If the subject required hospitalization for that long a period of time anyhow it is likely that he will find the accommodations better on the metabolic ward than on the usual hospital ward; however, this is not true in all hospitals. If the subject did not require hospitalization for therapeutic purposes, the period of incarceration might be viewed as more an inconvenience than as a benefit. Further, owing to the customary practices of very careful scrutiny of all activities on metabolic research wards, the subject might derive additional benefits as follows: Any adverse effects of the chlorothiazide are likely to be discovered earlier than they would in the course of the ordinary practice of medicine. Thus, the risks of taking the drug would be reduced accordingly. Further, any complications of the subject's basic disease are likely to be found and tended to quite promptly. Additional ramifications may be provided if desired.5

The view is further elucidated in comments by Levine in his book Ethics and Regulation of Clinical Research on the work of the National Commission. He states:

the Commission calls for an analysis of the various components of the research protocol. Procedures that are designed solely to benefit society or the class of children of which the particular child-subject is representative are to be considered as the research component. Judgements about the justification of the risks imposed by such procedures are to be made in accord with other recommendations. For example, if the risk is minimal, the research may be conducted as described in Recommendations 3 and 7 [of Research Involving Children], no matter what the risks are of the therapeutic components. The components of the protocol that hold out the prospect of direct benefit for the individual subjects are to be considered precisely as they are in the practice of medicine.24

Levine's description is clearly at variance with the actual text of Research Involving Children. The passage is significant, I believe, as an account of Levine's own views on the ethical analysis of risk, as developed for the National Commission. It may also be an accurate description of the view of the National Commission itself as reflected in Institutional Review Boards and the Belmont Report. A formal articulation of what we have called a component analysis of the potential benefits and risks of research would not, however, come until long after the close of the National Commission's work.

It is this last model of risk assessment, component analysis, that serves as the conceptual framework for the analysis of risk found within the Common Rule. Risks associated with nontherapeutic procedures must be minimized and reasonable in relation to the importance of the knowledge that may reasonable be expected to result (45 CFR 46.111(a)). Risks associated with therapeutic procedures must be reasonable in relation to anticipated benefits to subjects (45 CFR 46.111(a)). Thus, reflecting their historical origins, DHHS regulations protecting fetuses, children, and research subjects in general are based on three different approaches to the ethical analysis of the risks and potential benefits in research.

Back to Top

Toward a Comprehensive Approach for the Ethical Analysis of Potential Benefits and Risks in Research

What conceptual framework should guide the ethical analysis of risk? In this paper's introduction we noted that:

The moral analysis of risk is neither obvious nor intuitive. Rules, including those of the Common Rule, are not self-interpreting. They must be situated within a conceptual framework which facilitates their interpretation by the IRB. The articulation of a conceptual framework for the ethical analysis of risk might therefore be a project assisting IRBs in fulfilling their mandate - the protection of research subjects.

Our historical analysis reveals that differing aspects of current DHHS regulations are supported by differing and mutually incompatible conceptual frameworks for the moral analysis of risk:

  • Regulations for the protections of fetuses in research (45 CFR 46.208(a)) reflect a whole protocol approach to risk analysis, which requires that protocols be classified as either therapeutic or nontherapeutic.
  • Regulations for the protection of children in research (45 CFR 46.404-407) reflect a protocols with particular components approach. This approach defines separate standards for protocols with either therapeutic or nontherapeutic components. Recognizing that a given study may contain both a therapeutic and nontherapeutic procedure, it allows for both standards to apply simultaneously to a given study.
  • The Common Rule, outlining general protections for research subjects, relies on component approach to risk analysis. Procedures administered with therapeutic intent are justified when the benefits to subjects outweigh the risks. Procedures administered without such warrant, so-called nontherapeutic procedures, are justified only if they are minimized and if the risks are reasonable in relation to the knowledge to be gained.

The proliferation of conceptual frameworks underlying current regulation is obviously problematic. It has surely lead to ambiguity in regulation and confusion among IRBs attempting to implement the regulations in a consistent manner. One conceptual framework should guide the moral analysis of risks and potential benefits in research.

Of the three historical approaches to risk analysis it is clear that an approach based on component analysis is preferred. A whole protocol approach suffers from two problems: 1) Therapeutic research is a contradiction in terms and describes the null set and 2) research subjects are inadequately protected as any number of procedures not for the benefit of subjects may be added to a therapeutic study. An analysis of protocols with particular components also suffers from shortcomings:

  1. The concept of minimal risk is applied to both therapeutic and nontherapeutic procedures, but sets a threshold for allowable risk only to nontherapeutic procedures.
  2. The anchoring the concept of minimal to the risks of daily life for healthy, persons seems to run counter to the use of the notion of commensurability.
  3. Little guidance is given for the analysis of research that presents less than minimal risk.
  4. Since clinical research often contains a mixture of procedures, differing rules for whole protocols may simultaneously apply leading to confusion and conflict.

The ethical analysis of the various components in a research study presents a number of advantages:

  1. It acknowledges that clinical research often contains a mixture of procedures, some administered with therapeutic intent and others that answer the research question.
  2. Therapeutic procedures and nontherapeutic procedures are, by definition, administered with differing intent. This difference is morally relevant.
  3. Therapeutic procedures are justified by their potential to benefit the subject, while nontherapeutic procedures are justified by their potential to generate knowledge. These two benefits are largely incommensurable.
  4. Rigorous separate moral calculi for therapeutic and nontherapeutic procedures protect research subjects better than other approaches. It prevents the justification of risky nontherapeutic procedures by the benefits that may flow from therapeutic procedures.
  5. It is a more parsimonious model for analysis than other alternatives, and therefore avoids confusion and conflict.

A comprehensive approach to the ethical analysis of research risk was first formalized by Freedman and colleagues.1,28 This approach is summarized in Figure 1. Three main topics will be discussed here: the moral analysis of potential benefits and risks presented by therapeutic procedures; the moral analysis of potential benefits and risks of nontherapeutic procedures; and the role of the concept of minimal risk in the protection of vulnerable research subjects.

Figure 1. The Ethical Review of the Potential Benefits and Risks in Research

Protocol

Therapeutic Procedures - procedures administered with therapeutic warrant, must pass test of Clinical Equipoise - At the start of the study there must exist a state of genuine uncertainty in the community of expert practitioners as to the preferred treatment

IRB review:

  • Review of justification
  • May use literature review
  • May consult with impartial experts

Nontherapeutic Procedures - Procedures administered without therapeutic warrant. Risks must be minimized

  • Utilize procedures conducted for therapeutic purposes
  • Consistent with sound scientific design

Risks Reasonable in Relation to Knowledge to Be Gained

  • Assessment of the study's value
  • Requires input from experts and community representatives

Acceptable only if ethical tests for both therapeutic and nontherapeutic procedures are passed

Therapeutic procedures.

Therapeutic procedures are those interventions in research - drug, surgical procedure, device, or psychological procedure - administered with therapeutic intent (Figure 1). This category also encompasses monitoring procedures that optimally guide the administration of treatment, even if these procedures are not routinely administered in clinical practice. Let us consider what procedures might be considered therapeutic in the four example studies from the beginning of this paper.

  • In study A, a novel antipsychotic drug is compared with placebo. Both of these procedures are therapeutic interventions. The use of psychometric scales may be therapeutic if they are used routinely in clinical practice to guide treatment or if their use would reflect ideal practice. We do not have enough information to make this judgement, so we will assume that they are nontherapeutic.
  • In study B, hypnosis is used to implant a variety of suggestions related to deafness. Hypnosis is used therapeutically in certain circumstances, but in this case the use is nontherapeutic. The study population is not in need of any treatment. They are healthy college students and are participating solely for the purpose of testing a hypothesis.
  • In study C, a questionnaire related to sexual activity is administered to high school students. Obviously, this is not a therapeutic intervention.
  • In study D, an epidemiological survey is administered and genetic tests for mutations associated with breast cancer are done on blood samples. The study is directed at all adult members of a community, and not merely those who may require a detailed work-up for genetic predisposition to breast cancer. Furthermore, results will not be given to participants. These interventions are, therefore, nontherapeutic.

Having determined which procedures are administered with therapeutic warrant, how do we determine whether they are morally acceptable?

Therapeutic procedures must pass the test of clinical equipoise (Figure 1).29 A major competing notion, the uncertainty principle, has recently been shown inferior to clinical equipoise.30,31,32 Clinical equipoise is normally developed in response to the following question: When may the ethical physician offer trial participation to her patient? It begins from the recognition that competent medical practice is defined as that falling within the bounds of standard of care - that is, practice accepted by at least a respectable minority of expert practitioners. The innovation of clinical equipoise is the recognition that study treatments - be they experimental or control treatments - may be consistent with this standard of care. Thus, a physician, in keeping with his or her duty of care to the patient, may offer trial enrollment when [t]here exists an honest, professional disagreement among expert clinicians about the preferred treatment.29

A state of clinical equipoise may arise in a number of ways. Evidence may emerge from early clinical studies that a new treatment offers advantages over standard treatment. Alternatively, there may be a split within the clinical community, with some physicians preferring one treatment and other physicians preferring another. This latter scenario is well documented in the literature and calls for a randomized controlled trial (RCT) to settle which is the better treatment.33 Clinical equipoise permits these important RCTs. It would have physicians respect the fact that their less favored treatment is preferred by colleagues whom they consider to be responsible and competent.29

When evaluating a study containing one or more therapeutic procedures, the IRB must take reasonable steps to assure itself that a state of clinical equipoise exists. This will involve a critical evaluation of the study's justification. In selected cases, it may also require searches of the medical literature or consultation with relevant experts who have no connection with the study or its sponsor. A variety of treatment-related factors are also likely to contribute to this determination: the efficacy of the treatment; side effects, both reversible and irreversible; ease of administration; patient compliance; and perhaps even cost. It is important to recognize that clinical equipoise does not require numeric equality of treatment risks (or benefits, for that matter). It is more accurate to say that equipoise requires approximate equality in treatments therapeutic index - a compendious measure of potential benefits, risks, and uncertainty. Thus, a novel treatment may pose considerably more risk to subjects, so long as it also offers the prospect of considerably greater benefit. With novel interventions, the uncertainty associated with their effects will almost always be greater than treatments currently used in practice.

Study A is the only one of our four examples that involves the use of therapeutic procedures. The question the IRB must ask itself is as follows: Does a state of clinical equipoise exist among the new antipsychotic, placebo, and alternatives available in clinical practice? It follows from clinical equipoise that placebo controls will generally only be permissible for first generation treatments, when no standard treatment is available. Once effective treatment exists, new interventions must be tested against best available standard treatment. Freedman describes five circumstances in which placebo controls may be employed legitimately: 1) when there is no standard treatment; 2) when standard treatment is no better than placebo; 3) when standard treatment is placebo; 4) when the net therapeutic advantage of standard treatment has been called into question by new evidence; and 5) when effective treatment exists but is not available due to cost or short supply (although caveats apply to this criterion).34 Effective treatment exists for the treatment of schizophrenia, and, hence, the use of placebo in this case is impermissible.35 The IRB must not approve the study unless either an active control is used or the patient population is restricted to those who have no response to standard therapy, including any routinely used second- or third-line agents. A detailed rebuttal of scientific arguments made in favor of the routine use of placebo controls can be found elsewhere.8,36,37

Nontherapeutic procedures.

The remaining procedures administered in a clinical study are, by definition, not administered with therapeutic warrant and are properly referred to as nontherapeutic procedures (Figure 1). Such procedures are administered solely for scientific purposes, to answer the research question at hand. As all research is a systematic investigation designed to develop or contribute to generalizable knowledge (45 CFR 46.102(d)), it is difficult to imagine a study that does not include a nontherapeutic procedure. A nontherapeutic procedure may be as simple - and innocuous - as randomization, chart review, a questionnaire, an interview, or data that is recorded in some other manner; it may, however, be invasive or otherwise fraught with risk, as with genetic testing, organ biopsy, or the collection of information related to illegal practices. All four of the examples discussed at the beginning of this paper include nontherapeutic procedures:

  • Study A (trial of new medication in schizophrenia) proposes to test subjects regularly with psychometric scales. Filling out such forms is time consuming, potentially upsetting, and may expose subjects to the risk of discrimination.
  • Study B (hypnosis and deafness) involves a number of nontherapeutic procedures. Subjects will be hypnotized solely for research purposes, and various suggestions will be provided. Subjects will be observed, will fill out psychometric scales, and will be hypnotized again to remove the hypnotic suggestion. Distress and paranoia may result from the hypnosis, the effects of the implantation of these suggestions are uncertain, and there are risks associated with the administration of psychometric tests (supra).
  • Study C (adolescent sexual practices) again involves only nontherapeutic procedures. The questionnaire addresses a number of sensitive areas of inquiry, including sexuality and practices that predispose to HIV transmission. Subjects may find the questions anxiety provoking, and others may learn of deeply private matters, leading to stigmatization.
  • Study D (breast cancer genes) also involves only nontherapeutic procedures. The epidemiological survey and genetic tests may generate information that is anxiety provoking or that indeed may lead to workplace or insurance discrimination. Beyond risks to the individual study participants, the Jewish community as a whole may be wrongly labeled as cancer prone and subjected to discrimination and stigmatization.

By definition, risks associated with nontherapeutic procedures cannot be justified by the prospect of benefits to individual research subjects and, hence, a risk-benefit calculus is inappropriate to assessing their acceptability. The IRB must first ensure that the risks associated with nontherapeutic procedures are minimized by using procedures which are consistent with sound research design and which do not unnecessarily expose subjects to risk, and whenever appropriate, by using procedures already being performed on the subjects for diagnostic or treatment purposes (Figure 1) (45 CFR 46.111(a)(1)). Second, the IRB must ascertain that the risks of such procedures are reasonable in relation to the knowledge to be gained (Figure 1) (45 CFR 46.111(a)(2)). Thus, the ethical analysis of risks associated with nontherapeutic procedures involves a risk-knowledge calculus. The knowledge that may result from a study is essentially its scientific value. Freedman has argued that the proper assessment of the scientific value of a study requires not only the opinion of experts from relevant disciplines, but also of representatives of the community-at-large.38

In study A, the IRB will wish to ensure that all of the tests administered are required and consider whether psychometric tests administered routinely might provide equivalent information. In study B, hypnosis and hypnotic suggestion present worrisome risks. Can the information be gained in another way, for example, by studying those who are already deaf? Can the risks associated with hypnosis be minimized? Study C also presents nontrivial risk, in part because the questionnaire is administered in a high school setting. Paying careful attention to the protection of anonymity, allowing students to opt out of the questionnaire (or certain questions) unobtrusively, and seating students so they cannot see one the answers of others will minimize risk. In study D, risks to subjects of genetic information are considerably alleviated by destroying identifiers and by not informing participants of the results of genetic testing. In all cases, the risks of these procedures must be reasonable in relation to the knowledge to be gained.

Study D poses one category of risk that is not dealt with by this model - risks to the community. The Ashkenazi community has expressed the concern that such studies may lead to discrimination:

Such findings, which have already lead to Jewish groups being targeted as a potential market for commercial genetic tests, could create the perception that Jewish people are unusually susceptible to disease. As a result anyone with a Jewish sounding name could face discrimination in insurance and employment as companies struggle to keep down health care costs.39

The protection of communities in research is a novel area of inquiry in research ethics. Another paper commissioned by NBAC argues for a new ethical principle of respect for communities.40 Subsequent work has detailed possible protections for communities in research.41 Most recently, a rational schema for mapping appropriate protections onto specific communities, such as Ashkenazic Jews, has been reported.42 More work will be required to determine how the ethical analysis of risk for communities in research should proceed.

Minimal risk.

Minimal risk is a widely used concept in the regulation of research internationally. It can be found in contemporary guidelines from Australia,43 Canada,44 CIOMS,45 the Council of Europe,46 the United Kingdom [Physicians, 1996 #220], and the United States (45 CFR 46). That a research study poses minimal risk means that the risks of harm anticipated in the proposed research are not greater, considering probability and magnitude, than those ordinarily encountered in daily life or during the performance of routine physical or psychological examinations or tests (45 CFR 46.102(i)).

Minimal risk has been the subject of considerable debate and confusion in the literature. As we have seen, the concept of minimal risk was applied to both research with a therapeutic procedure and research with a nontherapeutic procedure in Research Involving Children. In the context of our schema for the ethical analysis of risk, this makes little sense. If a state of clinical equipoise exists, it follows that the therapeutic indices of the various study treatments (and alternatives available in clinical practice) are roughly equivalent. Thus, when considering the limits of risk to which research subjects may be exposed, we must focus on nontherapeutic risks. The risks of nontherapeutic procedures are the incremental risks associated with study participation.

Freedman and colleagues have argued that the definition of minimal risk found in the Common Rule is best understood as a core definition with examples.47 Minimal risk refers to risks ordinarily encountered in daily life" - or, shorter, risks of daily life (45 CFR 46.102(i)). The second part of the definition provides two examples of minimal risk: procedures encountered during the performance of routine physical or psychological examinations or tests (45 CFR 46.102(i)). The concept has been criticized on the grounds that it is difficult to know what counts as a risk of daily life and that the quantification of such risks is elusive.48 Freedman and colleagues conclude that the first claim is untrue and the second irrelevant.47 The risks of daily life are familiar to us all. Minimal risk does not refer to any risk encountered by any person, as some individuals engage in hazardous professions and pastimes. Rather it refers to the risks that are common to us all - driving to work, crossing the street, exchanging information over the internet, or getting a blood test at the doctor's office. While it may be difficult to quantify the precise probability of given outcomes associated with each of these behaviors, we can nonetheless easily identify them as risks of daily life. As Freedman and colleagues observe: We are, by definition, each of us acquainted with them; and, almost by definition, if we are unsure whether they belong within the set of common tasks then they don't.47 The assessment of whether a procedure is minimal risk is not primarily a quantitative determination; rather, it is a qualitative or categorical judgment made by the IRB. Research interventions may be determined to be of minimal risk because either the procedure is in fact encountered in daily life or it is sufficiently similar to those routinely encountered.

The threshold of a minor increase over minimal risk corresponds to the custodial duty that parents have for their children. Responsible parents makes decisions regarding new activities for their child based on the daily life of the child (minimal risk) and make allowances for the importance of new experiences (a minor increase over). While the majority of researchers and parents are scrupulous, some are not. The IRB acts in loco parentis by evaluating nontherapeutic risks as a responsible parent would, thereby ensuring that parents, scrupulous or not, will have the opportunity to enroll a child only in a study that would have passed such a test.

The concept of minimal risk serves two basic functions in regulation. First, it may be used as a sorting mechanism, directing the attention of the IRB to studies posing greater risk. Second, it serves as a threshold limiting the amount of nontherapeutic risk to which vulnerable research subjects may be exposed. The provision in the Common Rule allowing for expedited review is an example of the use of minimal risk as a sorting mechanism. If a study is found to pose only minimal risk, it may, with certain other caveats, receive approval by the IRB chair without full IRB review. The regulations state:

An IRB may use the expedited review procedure to review either or both of the following:

  1. some or all of the research appearing on the list and found by the reviewer(s) to involve no more than minimal risk,
  2. minor changes in previously approved research during the period (of one year or less) for which approval is authorized.

Under an expedited review procedure, the review may be carried out by the IRB chairperson or by one or more experienced reviewers designated by the chairperson from among members of the IRB. In reviewing the research, the reviewers may exercise all of the authorities of the IRB except that the reviewers may not disapprove the research. A research activity may be disapproved only after review in accordance with the non-expedited procedure set forth in 46.108(b) (45 CFR 46.110(b)).

Several problems are apparent with this provision. First, the requirement that nontherapeutic risks be both minimal risk and included in the list of research activities which may be reviewed through expedited review procedures (45 CFR 46) is curious. The list is obviously designed to include procedures that pose minimal risk to healthy adult subjects. For example, moderate exercise by healthy volunteers and collection of blood samples by venipuncture from subjects 18 years of age or older are permitted procedures. This effectively eliminates any study involving venipuncture in children or exercise testing of adults with illness from expedited review. This seems inconsistent with minimal risk as defined, which does not limit the standard to healthy persons or adults (45 CFR 46.102(i)).

Second, the expedited review provision as stated must surely be an incomplete set of criteria. A given study might pose only minimal risk to subjects and yet raise serious ethical concerns that should make it ineligible for expedited review. One such case is a study that involves a vulnerable population. Studies involving vulnerable populations require special scrutiny by IRBs and should not be eligible for expedited review. It may be that the current regulation attempts to so restrict the use of expedited review by limiting approvable activities to those administered to healthy adults. A more direct (and effective) regulatory stance on this issue would be preferable. Another such case is a study that has serious methodological flaws. Freedman observes that the ethical requirement that a study have a sound research design (validity) is absolute.38 Thus, a study should be eligible for expedited review only if three conditions are fulfilled: 1) the study poses no more than minimal risk to participants; 2) it does not involve a vulnerable population; and 3) no serious methodological flaws are apparent.

Most important is minimal risk's role as a threshold concept for allowable nontherapeutic risk in research on vulnerable populations (Figure 2). Vulnerable populations in the Common Rule include children, prisoners, pregnant women, mentally disabled persons, or economically or educationally disadvantaged persons (45 CFR 111(b)). Given the heterogeneity of these groups, vulnerability itself must be a complex notion. Indeed, it encompasses groups who have one or more of the following characteristics: undue susceptibility to harm; incapability of providing informed consent to study participation; or being so situated as to render the voluntariness of consent suspect.49 In light of these characteristics, the vulnerable are entitled to special protections in research. Three protections are often invoked. First, members of a vulnerable group may be included in research only when their participation is essential to the hypothesis being tested. Second, if persons are incapable of providing informed consent, the consent of a proxy decisionmaker is required. Third, the amount of nontherapeutic risk to which persons may be exposed is limited to either minimal or a minor increase over minimal.

The importance of the last protection can scarcely be over emphasized. Clinical equipoise ensures that therapeutic procedures in a study are comparable with each other and alternatives in clinical practice in terms of their therapeutic indices. Thus, the incremental risk posed by study participation is that posed by nontherapeutic procedures. If vulnerable populations, such as children or incapable adults, are to be protected in any meaningful way, the risks of nontherapeutic procedures to which they may be exposed must be limited to a minor increase above minimal risk. As we have discussed, the standard has the advantage of mirroring the custodial duties of parents to children and caretakers to incapable adults.

NBAC proposes to eliminate this important protection.50 In its report Research Involving Persons with Mental Disorders That May Affect Decisionmaking Capacity, no limit is placed on the nontherapeutic risk to which an incapable adult may be exposed, provided certain consent provisions obtain (Recommendation 12). This is shortsighted. When the limit of a minor increase above minimal risk is eliminated as a threshold for permissible nontherapeutic risk, no amount of risk is ruled out for research involving incapable persons. So long as the research question is important enough (and informed consent provisions fulfilled), any amount of nontherapeutic risk is permissible. This change, if translated into regulation, will effectively undermine protections for incapable persons in research. Incapable persons will then be exposed to exploitation legitimated by the very regulations that were to protect them.

Figure 2. The Ethical Review of the Potential Benefits and Risks in Research Involving a Vulnerable Population

Protocol

Study Hypotheses Requires Inclusion of Vulnerable Pop'n

Therapeutic Procedures - procedures administered with therapeutic warrant

Nontherapeutic Procedures - Procedures administered without therapeutic warrant, must pass test of Clinical Equipoise - At the start of the study there must exist a state of genuine uncertainty in the community of expert practitioners as to the preferred treatment

IRB review:

  • Review of justification
  • May use literature review
  • May consult with impartial experts

Risks Must Be Minimized

  • Utilize procedures conducted for therapeutic purposes
  • Consistent with sound scientific design

Risks Reasonable in Relation to Knowledge to Be Gained

  • Assessment of the study's value
  • Requires input from experts and community representatives

No More Than a Minor Increase Above Minimal Risk

No more than a minor increase above the risks of daily life for the study population in question

  • Applies only to nontherapeutic procedures
  • A qualitative judgment made by the IRB

Acceptable only if ethical tests for both therapeutic and nontherapeutic procedures are passed

Back to Top

Implications for U.S. Regulations Protecting Research Subjects

What changes to U.S. regulations would the implementation of such a framework require? As we remarked supra, rules are not self-interpreting, and a conceptual framework is required. What follows is a summary of the changes to aspects of the Common Rule and DHHS regulations pertaining to risk analysis required to achieve this end. Proposed text is highlighted in bold. Specific recommendations for changes to subparts B (pregnant women) and C (prisoners) are not included.

  1. It is clear that the U.S. regulations protecting research subjects found in the Common Rule and the DHHS regulations were profoundly influenced by the works of the National Commission. The differing models of risk analysis with which the National Commission worked influenced different parts of the regulations. These inconsistencies must surely be corrected. IRBs require a single conceptual framework for the ethical analysis of the risks and benefits in research if they are to apply regulations consistently. The component approach described in the last section of this paper is the preferred conceptual model.
  2. The concepts of therapeutic and nontherapeutic procedures should be included and defined, as they are central to this approach to risk analysis. 46.102(k) Therapeutic procedures are study interventions administered with the intent of providing direct benefit to the research subject. 46.102(l) Nontherapeutic procedures are study interventions that are not administered with therapeutic intent, and are only intended to answer the scientific question of the study.
  3. The IRBs general obligations regarding the ethical analysis of the potential benefits and risks of research should be stated more clearly. 46.111(a) In order to approve research covered by this policy the IRB shall determine that all of the following requirements are satisfied: 1) Therapeutic procedures fulfill the requirements of clinical equipoise. That is, at the start of the study there must exist a state of genuine uncertainty in the community of expert practitioners as to the preferred treatment. 2) The risks associated with nontherapeutic procedures must be minimized i) by using procedures which are consistent with sound research design and which do not unnecessarily expose subjects to risk and ii) whenever appropriate, by using procedures already being performed on the subjects for diagnostic or treatment purposes. Risks of nontherapeutic procedures must be reasonable in relation to the knowledge to be gained. Both 46.111(a)(1) and 46.111(a)(2) must be satisfied if a given study is to be approved.
  4. The definition of minimal risk has been a source of considerable controversy and confusion. The definition should be simplified and clarified. 46.102(i) Minimal risk means that the probability and magnitude of harm is no greater than that encountered in daily lives of all (or the great majority) of persons in the population from which research subjects are to be recruited. It refers only to the risks associated with nontherapeutic procedures.
  5. The role of the concept of minimal risk in expedited review needs to be clarified. The use of a list of procedures drawn up only for healthy adults is inconsistent with the concept's definition and use. Furthermore, minimal risk is not a sufficient condition for a research protocol to receive expedited review. Generally speaking, the study protocol must also be methodologically sound and not involve a vulnerable population. 46.110(a) deleted 46.110(b) An IRB may use the expedited review procedure to review either an entire protocol or a protocol amendment provided the review(s) determine:
    1. The study methods are valid;
    2. The study does not involve a vulnerable population; and
    3. The study poses no more than minimal risk.
  6. The ethical analysis of risk as pertains to children as research subjects can be simplified greatly with this conceptual approach. Simplifying these regulations will avoid confusion and help IRBs protect children who are research subjects. 46.404 delete 46.405 delete 46.406 delete 46.407 delete 46.404 (new) In order to approve research involving children covered by this policy the IRB shall determine that all of the following requirements are satisfied: a) the conditions of 46.111(a)(1), 46.111(a)(2), and 46.111(a)(3); b) answering the study's scientific hypothesis requires the inclusion of children as research subjects; and c) risks associated with nontherapeutic procedures are no more than a minor increase over minimal risk.
  7. A new section must be added to the DHHS regulations detailing protections for adults incapable of providing informed consent. The protections for incapable adults will for the most part be similar to those for children. 46.500 In order to approve research involving incapable adults covered by this policy the IRB shall determine that all of the following requirements are satisfied: a) the conditions of 46.111(a)(1), 46.111(a)(2), and 46.111(a)(3); b) answering the study's scientific hypothesis requires the inclusion of incapable adults as research subjects; and c) risks associated with nontherapeutic procedures are no more than a minor increase over minimal risk.

Back to Top

Acknowledgments

I am grateful for the helpful comments on earlier drafts of this paper by Drs. Chris MacDonald, Eric Meslin, Paul Miller, and Marjorie Speers.

Back to Top

Footnotes

  • 1Weijer C. Thinking clearly about research risk: Implications of the work of Benjamin Freedman. IRB: A Review of Human Subjects Research 1999; 21:15.
  • 2Freedman B, Shapiro S. Ethics and statistics in clinical research: Towards a more comprehensive examination. Journal of Statistical Planning and Inference 1994; 42:223240.
  • 3National Commission on the Protection of Humans Subjects of Biomedical and Behavioral Research. The Belmont Report: Ethical Principles for the Protection of Human Subjects of Biomedical and Behavioral Research. Washington, D.C.: DHEW Publication no. (OS) 78-0012, 1978.
  • 4Levine R. The boundaries between biomedical or behavioral research and the accepted and routine practice of medicine. In: National Commission on the Protection of Humans Subjects of Biomedical and Behavioral Research. The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research, Appendix 1. Washington, D.C.: DHEW Publication no. (OS) 78-0013, 1978.
  • 5Levine R. The role of assessment of risk benefit criteria in the determination of the appropriateness of research involving human subjects. In: National Commission on the Protection of Humans Subjects of Biomedical and Behavioral Research. The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research, Appendix 1. Washington, D.C.: DHEW Publication no. (OS) 78-0013, 1978.
  • 6Levine R. Balance of Harms and Benefits. Ethics and the Regulation of Clinical Research (2nd ed.). New Haven: Yale University Press, 1988:3765.
  • 7Chouinard G. A placebo-controlled clinical trial of remoxipride and chlorpromazine in newly admitted schizophrenia patients with acute exacerbation. Acta Psychiatrica Scandinavica 1990; 358:111119.
  • 8Weijer C. Placebo-controlled trials in schizophrenia: Are they ethical? Are they necessary? Schizophr Res 1999; 35:211218; discussion 227236.
  • 9Zimbardo P. Inducing hearing deficit generates experimental paranoia. Science 1981; 212:15291531.
  • 10Lewis M, Zimbardo P. The ethics of inducing paranoia in an experimental setting [letter and response]. IRB: A Review of Human Subjects Research 1981; December:911.
  • 11Phillips S. Asking the sensitive question: the ethics of survey research and teen sex. IRB: A Review of Human Subjects Research 1994; 16:17.
  • 12Struewing JP, Hartge P, Wacholder S, Baker SM, Berlin M, McAdams M, Timmerman MM, Brody LC, Tucker MA. The risk of cancer associated with specific mutations of BRCA1 and BRCA2 among Ashkenazi Jews [see comments]. N Engl J Med 1997; 336:14011408.
  • 13Glass K, Weijer C, Lemmens T, Palmour R, Shapiro S. Structuring the review of human genetics protocols part II: Diagnostic and screening studies. IRB: A Review of Human Subjects Research 1997; 19:111, 13.
  • 14Gray B. Bioethics commissions: What can we learn from past successes and failures? In: Bulger R, Bobby E, Fineberg H, eds. Society's Choices: Social and Ethical Decision Making in Biomedicine. Washington, D.C.: National Academy Press, 1995:261306.
  • 15National Commission on the Protection of Humans Subjects of Biomedical and Behavioral Research. Research on the Fetus: Report and Recommendations. Washington, D.C: DHEW Publication no. (OS) 76-127, 1975.
  • 16National Commission on the Protection of Humans Subjects of Biomedical and Behavioral Research. Research Involving Prisoners: Report and Recommendations. Washington, D.C.: DHEW Publication no. (OS) 76-131, 1976.
  • 17National Commission on the Protection of Humans Subjects of Biomedical and Behavioral Research. Research Involving Children: Report and Recommendations. Washington, D.C.: DHEW Publication no. (OS) 77-0004, 1977.
  • 18National Commission on the Protection of Humans Subjects of Biomedical and Behavioral Research. Research Involving Those Institutionalized as Mentally Infirm: Report and Recommendations. Washington, D.C.: DHEW Publication no. (OS) 78-0006, 1978.
  • 19National Commission on the Protection of Humans Subjects of Biomedical and Behavioral Research. Institutional Review Boards: Report and Recommendations. Washington, D.C.: DHEW Publication no. (OS) 78-0008, 1978.
  • 20Levine R. The Fetus and the Embryo. Ethics and Regulation of Clinical Research. New Haven: Yale University Press, 1988:297320.
  • 21Brody B. The Ethics of Biomedical Research: An International Perspective. New York: Oxford University Press, 1998.
  • 22Weijer C. Trial by error. Hastings Center Report 2000; in press.
  • 23Levine R. Prisoners. Ethics and Regulation of Clinical Research (2nd ed.). New Haven: Yale University Press, 1988:277295.
  • 24Levine R. Children. Ethics and Regulation of Clinical Research (2nd ed.). New Haven: Yale University Press, 1988:235256.
  • 25The President's Commission for the Study of Ethical Problems in Medicine and Biomedical and Behavioral Research. Protecting Human Subjects: The Adequacy and Uniformity of Federal Rules and Their Implementation. Washington, D.C.: U.S. Government Printing Office, Stock no. 040-000-00452-1, 1981.
  • 26The President's Commission for the Study of Ethical Problems in Medicine and Biomedical and Behavioral Research. Implementing Human Research Regulations: The Adequacy and Uniformity of Federal Rules and of Their Implementation. Washington, D.C.: U.S. Printing Office, Stock no. 040-000-004781-8, 1983.
  • 27Levine R. Those Institutionalized as Mentally Infirm. Ethics and Regulation of Clinical Research (2nd ed.). New Haven: Yale University Press, 1988:257276.
  • 28Freedman B, Fuks A, Weijer C. Demarcating research and treatment: a systematic approach for the analysis of the ethics of clinical research. Clin Res 1992; 40:65360.
  • 29Freedman B. Equipoise and the ethics of clinical research. N Engl J Med 1987; 317:141145.
  • 30Peto R, Pike M, Armitage P, Breslow NE, Cox DR, Howard SV, Mantel N, McPherson K, Peto J, Smith PG. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design. British Journal of Cancer 1976; 34:585612.
  • 31Peto R, Baigent C. Trials: The next 50 years. British Medical Journal 1998; 317:11701171.
  • 32Weijer C, Shapiro SH, Cranley Glass K. For and against: Clinical equipoise and not the uncertainty principle is the moral underpinning of the randomised controlled trial. British Medical Journal 2000; 7263:756758.
  • 33Taylor KM, Margolese RG, Soskolne CL. Physicians reasons for not entering eligible patients in a randomized clinical trial of surgery for breast cancer. New England Journal of Medicine 1984; 310:13631367.
  • 34Freedman B. Placebo-controlled trials and the logic of clinical purpose. IRB: A Review of Human Subjects Research 1990; 12:16.
  • 35Kane J. Schizophrenia. New England Journal of Medicine 1996; 334:3441.
  • 36Freedman B, Weijer C, Glass KC. Placebo orthodoxy in clinical research. I: Empirical and methodological myths. J Law Med Ethics 1996; 24:243251.
  • 37Freedman B, Glass KC, Weijer C. Placebo orthodoxy in clinical research. II: Ethical, legal, and regulatory myths. J Law Med Ethics 1996; 24:252259.
  • 38Freedman B. Scientific value and validity as ethical requirements for research: a proposed explication. IRB: A Review of Human Subjects Research 1987; 9:710.
  • 39Lehrman S. Ashkenazi signals are timely. Nature 1997; 389:315316.
  • 40Weijer C. Protecting communities in research: philosophical and pragmatic challenges. Camb Q Healthc Ethics 1999; 8:501513.
  • 41Weijer C, Goldsand G, Emanuel EJ. Protecting communities in research: current guidelines and limits of extrapolation. Nat Genet 1999; 23:275280.
  • 42Weijer C, Emanuel EJ. Protecting communities in biomedical research. Science 2000; 289:11421144.
  • 43Australia National Health and Medical Research Council. National Statement on Ethical Conduct in Research Involving Humans. Canberra: NHMRC, 1999.
  • 44Medical Research Council of Canada, Natural Sciences and Engineering Research Council of Canada, Social Sciences and Humanities Research Council of Canada. Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans. Ottawa: Public Works and Government Services Canada, 1998.
  • 45Council for International Organizations of Medical Sciences (CIOMS). International Ethical Guidelines for Biomedical Research Involving Human Subjects. Geneva: CIOMS, 1993.
  • 46Council of Europe. Convention for the Protection of Human Rights and Dignity of the Human Being with Regard to the Application of Biology and Medicine: Convention on Human Rights and Biomedicine. Oviedo: European Treaty Series, 1997.
  • 47Freedman B, Fuks A, Weijer C. In loco parentis: Minimal risk as an ethical threshold for research upon children. Hastings Center Report 1993; 23:1319.
  • 48Kopelman L. Estimating risk in human research. Clinical Research 1981; 29:18.
  • 49Weijer C. Research involving the vulnerable sick. Accountability in Research 1999; 7:2136.
  • 50National Bioethics Advisory Commission. Research Involving Persons with Mental Disorders That May Affect Decisionmaking Capacity. Rockville, Maryland: NBAC, 1998.
Cite this page: "The Ethical Analysis of Risks and Potential Benefits in Human Subjects Research (Research Involving Human Participants V2)" Online Ethics Center for Engineering 5/29/2007 National Academy of Engineering Accessed: Wednesday, February 08, 2012 <www.onlineethics.org/Topics/RespResearch/ResResources/nbacindex/nbachindex/hweijer.aspx>