Listening to Prozac but Hearing Placebo:
A Meta-Analysis of Antidepressant Medication

Irving Kirsch, Ph.D.
University of Connecticut, Storrs, CT

Guy Sapirstein, Ph.D.
Westwood Lodge Hospital, Needham, MA

ABSTRACT
Mean effect sizes for changes in depression were calculated for 2,318 patients who had been randomly assigned to either antidepressant medication or placebo in 19 double-blind clinical trials. As a proportion of the drug response, the placebo response was constant across different types of medication (75%), and the correlation between placebo effect and drug effect was .90. These data indicate that virtually all of the variation in drug effect size was due to the placebo characteristics of the studies. The effect size for active medications that are not regarded to be antidepressants was as large as that for those classified as antidepressants, and in both cases, the inactive placebos produced improvement that was 75% of the effect of the active drug. These data raise the possibility that the apparent drug effect (25% of the drug response) is actually an active placebo effect. Examination of pre–post effect sizes among depressed individuals assigned to no-treatment or wait-list control groups suggest that approximately one quarter of the drug response is due to the administration of an active medication, one half is a placebo effect, and the remaining quarter is due to other nonspecific factors.

EDITORS' NOTE
The article that follows is a controversial one. It reaches a controversial conclusion—that much of the therapeutic benefit of antidepressant medications actually derives from placebo responding. The article reaches this conclusion by utilizing a controversial statistical approach—meta-analysis. And it employs meta-analysis controversially—by meta-analyzing studies that are very heterogeneous in subject selection criteria, treatments employed, and statistical methods used. Nonetheless, we have chosen to publish the article. We have done so because a number of the colleagues who originally reviewed the manuscript believed it had considerable merit, even while they recognized the clearly contentious conclusions it reached and the clearly arguable statistical methods it employed.

We are convinced that one of the principal aims of an electronic journal ought to be to bring our readers information on a variety of current topics in prevention and treatment, even though much of it will be subject to heated differences of opinion about worth and ultimate significance. This is to be expected, of course, when one is publishing material at the cutting-edge, in a cutting-edge medium.

We also believe, however, that soliciting expert commentary to accompany particularly controversial articles facilitates the fullest possible airing of the issues most germane to appreciating both the strengths and the weaknesses of target articles. In the same vein, we welcome comments on the article from readers as well, though for obvious reasons, we cannot promise to publish all of them.

Feel free to submit a comment by emailing admin@apa.org.

Peter Nathan, Associate Editor (Treatment)
Martin E. P. Seligman, Editor

We thank R. B. Lydiard and Smith-Kline Beecham Pharaceuticals for supplying additional data. We thank David Kenny for his assistance with the statistical analyses. We thank Roger P. Greenberg and Daniel E. Moerman for their helpful comments on earlier versions of this paper.

Correspondence concerning this article should be addressed to Irving Kirsch, Department of Psychology, U-20, University of Connecticut, 406 Babbidge Road Storrs, CT 06269-1020.
E-mail: Irvingk@uconnvm.uconn.edu

More placebos have been administered to research participants than any single experimental drug. Thus, one would expect sufficient data to have accumulated for the acquisition of substantial knowledge of the parameters of placebo effects. However, although almost everyone controls for placebo effects, almost no one evaluates them. With this in mind, we set about the task of using meta-analytic procedures for evaluating the magnitude of the placebo response to antidepressant medication.

Meta-analysis provides a means of mathematically combining results from different studies, even when these studies have used different measures to assess the dependent variable. Most often, this is done by using the statistic d, which is a standardized difference score. This effect size is generally calculated as the mean of the experimental group minus the mean of the control group, divided by the pooled standard deviation. Less frequently, the mean difference is divided by standard deviation of the control group (Smith, Glass, & Miller, 1980).

Ideally, to calculate the effect size of placebos, we would want to subtract the effects of a no-placebo control group. However, placebos are used as controls against which the effects of physical interventions can be gauged. It is rare for an experimental condition to be included against which the effects of the placebo can be evaluated. To circumvent this problem, we decided to calculate within-cell or pre–post effect sizes, which are the posttreatment mean depression score minus the pretreatment mean depression score, divided by the pooled standard deviation (cf. Smith et al., 1980). By doing this for both placebo groups and medication groups, we can estimate the proportion of the response to antidepressant medication that is duplicated by placebo administration, a response that would be due to such factors as expectancy for improvement and the natural course of the disorder (i.e., spontaneous remission). Later in this article, we also separate expectancy from natural history and provide estimates of each of these effects.

Although our approach is unusual, in most cases it should provide results that are comparable to conventional methods. If there are no significant pretreatment differences between the treatment and control groups, then the subtraction of mean standardized pre–post difference scores should result in a mean effect size that is just about the same as that produced by subtracting mean standardized posttreatment scores. Suppose, for example, we have a study with the data displayed in Table 1. The conventionally calculated effect size would be would be 1.00. The pre–post effect sizes would be 3.00 for the treatment group and 2.00 for the control group. The difference between them is 1.00, which is exactly the same effect calculated from posttreatment scores alone. However, calculating the effect size in this manner also provides us with the information that the effect of the control procedure was 2/3 that of the treatment procedure, information that we do not have when we only consider posttreatment scores. Of course, it is rare for two groups to have identical mean pretreatment scores, and to the extent that those scores are different, our two methods of calculation would provide different results. However, by controlling for baseline differences, our method should provide the more accurate estimate of differential outcome.

Table 1
Hypothetical Means and Standard Deviations for a Treatment Group and a Control Group

	Treatment		Control
	Pretreatment	Posttreatment	Pretreatment	Posttreatment
M	25.00	10.00	25.00	15.00
SD	5.50	4.50	4.50	5.50

 

Studies assessing the efficacy of antidepressant medication were obtained through previous reviews (Davis, Janicak, & Bruninga, 1987; Free & Oei, 1989; Greenberg & Fisher, 1989; Greenberg, Bornstein, Greenberg, & Fisher, 1992; Workman & Short, 1993), supplemented by a computer search of PsycLit and MEDLINE databases from 1974 to 1995 using the search terms drug-therapy or pharmacotherapy or psychotherapy or placebo and depression or affective disorders. Psychotherapy was included as a search term for the purpose of obtaining articles that would allow estimation of changes occurring in no-treatment and wait-list control groups, a topic to which we return later in this article. Approximately 1,500 publications were produced by this literature search. These were examined by the second author, and those meeting the following criteria were included in the meta-analysis:

1. The sample was restricted to patients with a primary diagnosis of depression. Studies were excluded if participants were selected because of other criteria (eating disorders, substance abuse, physical disabilities or chronic medical conditions), as were studies in which the description of the patient population was vague (e.g., "neurotic").
2. Sufficient data were reported or obtainable to calculate within-condition effect sizes. This resulted in the exclusion of studies for which neither pre–post statistical tests nor pretreatment means were available.
3. Data were reported for a placebo control group.
4. Participants were assigned to experimental conditions randomly.
5. Participants were between the ages of 18 and 75.

Of the approximately 1,500 studies examined, 20 met the inclusion criteria. Of these, all but one were studies of the acute phase of therapy, with treatment durations ranging from 1 to 20 weeks (M = 4.82). The one exception (Doogan & Caillard, 1992) was a maintenance study, with a duration of treatment of 44 weeks. Because of this difference, Doogan and Caillard's study was excluded from the meta-analysis. Thus, the analysis was conducted on 19 studies containing 2,318 participants, of whom 1,460 received medication and 858 received placebo. Medications studied were amitriptyline, amylobarbitone, fluoxetine, imipramine, paroxetine, isocarboxazid, trazodone, lithium, liothyronine, adinazolam, amoxapine, phenelzine, venlafaxine, maprotiline, tranylcypromine, and bupropion.

In most cases, effect sizes (d) were calculated for measures of depression as the mean posttreatment score minus the mean pretreatment score, divided by the pooled standard deviation (SD). Pretreatment SDs were used in place of pooled SDs in calculating effect sizes for four studies in which posttreatment SDs were not reported (Ravaris, Nies, Robinson, et al., 1976; Rickels & Case, 1982; Rickels, Case, Weberlowsky, et al., 1981; Robinson, Nies, & Ravaris, 1973). The methods described by Smith et al. (1980) were used to estimate effect sizes for two studies in which means and SDs were not reported. One of these studies (Goldberg, Rickels, & Finnerty, 1981) reported the t value for the pre–post comparisons. The effect size for this study was estimated using the formula:

d= t (2/n)^1/2

where t is the reported t value for the pre–post comparison, and n is the number of subjects in the condition. The other study (Kiev & Okerson, 1979) reported only that there was a significant difference between pre- and posttreatment scores. As suggested by Smith et al. (1980), the following formula for estimating the effect size was used:

d= 1.96 (2/n) ^1/2 ,

where 1.96 is used as the most conservative estimation of the t value at the .05 significance level used by Kiev and Okerson. These two two effect sizes were also corrected for pre–post correlation by multiplying the estimated effect size by (1 - r) ^1/2 , r being the estimate of the test–retest correlation (Hunter & Schmidt, 1990). Bailey and Coppen (1976) reported test–retest correlations of .65 for the Beck Depression Inventory (BDI; Beck, Ward, Mendelson, Mock, & Erbaugh, 1961) and .50 for the Hamilton Rating Scale for Depression (HRS-D; Hamilton, 1960) . Therefore, in order to arrive at an estimated effect size, corrected for the pre–post correlation, the estimated effect sizes of the HRS-D were multiplied by 0.707 and the effect sizes of the BDI were multiplied by 0.59.

In studies reporting multiple measures of depression, an effect size was calculated for each measure and these were then averaged. In studies reporting the effects of two drugs, a single mean effect size for both was calculated for the primary analysis. In a subsequent analysis, the effect for each drug was examined separately. In both analyses, we calculated mean effect sizes weighted for sample size (D; Hunter & Schmidt, 1990).

Sample sizes and effect sizes for patients receiving medication or placebo are presented in Table 2. Mean effect sizes, weighted for sample size, were 1.55 SDs for the medication response and 1.16 for the placebo response. Because effect sizes are obtained by dividing both treatment means by a constant (i.e., the pooled SD), they can be treated mathematically like the scores from which they are derived. ¹ In particular, we have shown that, barring pretreatment between-group differences, subtracting the mean pre–post effect size of the control groups from the mean pre–post effect size of the experimental groups is equivalent to calculating an effect size by conventional means. Subtracting mean placebo response rates from mean drug response rates reveals a mean medication effect of 0.39 SDs. This indicates that 75% of the response to the medications examined in these studies was a placebo response, and at most, 25% might be a true drug effect. This does not mean that only 25% of patients are likely to respond to the pharmacological properties of the drug. Rather, it means that for a typical patient, 75% of the benefit obtained from the active drug would also have obtained from an inactive placebo.

Table 2
Studies Including Placebo Control Groups

	Drug		Placebo
Study	n	d	n	d
Blashki et al. (1971)	43	1.75	18	1.02
Byerly et al. (1988)	44	2.30	16	1.37
Claghorn et al. (1992)	113	1.91	95	1.49
Davidson & Turnbull (1983)	11	4.77	8	2.28
Elkin et al. (1989)	36	2.35	34	2.01
Goldberg et al. (1981)	179	0.44	93	0.44
Joffe et al. (1993)	34	1.43	16	0.61
Kahn et al. (1991)	66	2.25	80	1.48
Kiev & Okerson (1979)	39	0.44	22	0.42
Lydiard (1989)	30	2.59	15	1.93
Ravaris et al. (1976)	14	1.42	19	0.91
Rickels et al. (1981)	75	1.86	23	1.45
Rickels & Case (1982)	100	1.71	54	1.17
Robinson et al. (1973)	33	1.13	27	0.76
Schweizer et al. (1994)	87	3.13	57	2.13
Stark & Hardison (1985)	370	1.40	169	1.03
van der Velde (1981)	52	0.66	27	0.10
White et al. (1984)	77	1.50	45	1.14
Zung (1983)	57	.88	40	0.95

 

Inspection of Table 2 reveals considerable variability in drug and placebo response effect sizes. As a first step toward clarifying the reason for this variability, we calculated the correlation between drug response and placebo response, which was found to be exceptionally high, r = .90, p < .001 (see Figure 1). This indicates that the placebo response was proportionate to the drug response, with remaining variability most likely due to measurement error.

Figure 1. The placebo response as a predictor of the drug response.

Our next question was the source of the common variability. One possibility is that the correlation between placebo and drug response rates are due to between-study differences in sample characteristics (e.g., inpatients vs. outpatients, volunteers vs. referrals, etc.). Our analysis of psychotherapy studies later in this article provides a test of this hypothesis. If the correlation is due to between-study differences in sample characteristics, a similar correlation should be found between the psychotherapy and no-treatment response rates. In fact, the correlation between the psychotherapy response and the no-treatment response was nonsignificant and in the opposite direction. This indicates that common sample characteristics account for little if any of the relation between treatment and control group response rates.

Another possibility is that the close correspondence between placebo and drug response is due to differences in so-called nonspecific variables (e.g., provision of a supportive relationship, color of the medication, patients' expectations for change, biases in clinician's ratings, etc.), which might vary from study to study, but which would be common to recipients of both treatments in a given study. Alternately, the correlation might be associated with differences in the effectiveness of the various medications included in the meta-analysis. This could happen if more effective medications inspired greater expectations of improvement among patients or prescribing physicians (Frank, 1973; Kirsch, 1990). Evans (1974), for example, reported that placebo morphine was substantially more effective than placebo aspirin. Finally, both factors might be operative.

We further investigated this issue by examining the magnitude of drug and placebo responses as a function of type of medication. We subdivided medication into four types: (a) tricyclics and tetracyclics, (b) selective serotonin reuptake inhibitors (SSRI), (c) other antidepressants, and (d) other medications. This last category consisted of four medications (amylobarbitone, lithium, liothyronine, and adinazolam) that are not considered antidepressants.

Weighted (for sample size) mean effect sizes of the drug response as a function of type of medication are shown in Table 3, along with corresponding effect sizes of the placebo response and the mean effect sizes of placebo responses as a proportion of drug responses. These data reveal relatively little variability in drug response and even less variability in the ratio of placebo response to drug response, as a function of drug type. For each type of medication, the effect size for the active drug response was between 1.43 and 1.69, and the inactive placebo response was between 74% and 76% of the active drug response. These data suggest that the between-drug variability in drug and placebo response was due entirely to differences in the placebo component of the studies.

Table 3
Effect Sizes as a Function of Drug Type

Statistic	Type of drug
	Antidepressant			Other drugs
	Tri- and tetracyclic	SSRI	Other
N	1,353	626	683	203
K	13	4	8	3
D—Drug	1.52	1.68	1.43	1.69
D—Placebo	1.15	1.24	1.08	1.29
Placebo/drug	.76	.74	.76	.76
N = number of subjects; K = number of studies; D = mean weighted effect size; placebo/drug = placebo response as a proportion of active drug response.

Differences between active drug responses and inactive placebo responses are typically interpreted as indications of specific pharmacologic effects for the condition being treated. However, this conclusion is thrown into question by the data derived from active medications that are not considered effective for depression. It is possible that these drugs affect depression indirectly, perhaps by improving sleep or lowering anxiety. But if this were the case and if antidepressants have a specific effect on depression, then the effect of these other medications ought to have been less than the effect of antidepressants, whereas our data indicate that the response to these nonantidepressant drugs is at least as great as that to conventional antidepressants.

A second possibility is that amylobarbitone, lithium, liothyronine, and adinazolam are in fact antidepressants. This conclusion is rendered plausible by the lack of understanding of the mechanism of clinical action of common antidepressants (e.g., tricyclics). If the classification of a drug as an antidepressant is established by its efficacy, rather than by knowledge of the mechanism underlying its effects, then amylobarbitone, lithium, liothyronine, and adinazolam might be considered specifics for depression.

A third possibility is that these medications function as active placebos (i.e., active medications without specific activity for the condition being treated). Greenberg and Fisher (1989) summarized data indicating that the effect of antidepressant medication is smaller when it is compared to an active placebo than when it is compared to an inert placebo (also see Greenberg & Fisher, 1997). By definition, the only difference between active and inactive placebos is the presence of pharmacologically induced side effects. Therefore, differences in responses to active and inert placebos could be due to the presence of those side effects. Data from other studies indicate that most participants in studies of antidepressant medication are able to deduce whether they have been assigned to the drug condition or the placebo condition (Blashki, Mowbray, & Davies, 1971; Margraf, Ehlers, Roth, Clark, Sheikh, Agras, & Taylor, 1991; Ney, Collins, & Spensor, 1986). This is likely to be associated with their previous experience with antidepressant medication and with differences between drug and placebo in the magnitude of side effects. Experiencing more side effects, patients in active drug conditions conclude that they are in the drug group; experiencing fewer side effects, patients in placebo groups conclude that they are in the placebo condition. This can be expected to produce an enhanced placebo effect in drug conditions and a diminished placebo effect in placebo groups. Thus, the apparent drug effect of antidepressants may in fact be a placebo effect, magnified by differences in experienced side effects and the patient's subsequent recognition of the condition to which he or she has been assigned. Support for this interpretation of data is provided by a meta-analysis of fluoxetine (Prozac), in which a correlation of .85 was reported between the therapeutic effect of the drug and the percentage of patients reporting side effects (Greenberg, Bornstein, Zborowski, Fisher, & Greenberg, 1994).

Natural History Effects

Just as it is important to distinguish between a drug response and a drug effect, so too is it worthwhile to distinguish between a placebo response and a placebo effect (Fisher, Lipman, Uhlenhuth, Rickels, & Park, 1965). A drug response is the change that occurs after administration of the drug. The effect of the drug is that portion of the response that is due to the drug's chemical composition; it is the difference between the drug response and the response to placebo administration. A similar distinction can be made between placebo responses and placebo effects. The placebo response is the change that occurs following administration of a placebo. However, change might also occur without administration of a placebo. It may be due to spontaneous remission, regression toward the mean, life changes, the passage of time, or other factors. The placebo effect is the difference between the placebo response and changes that occur without the administration of a placebo (Kirsch, 1985, 1997).

In the preceding section, we evaluated the placebo response as a proportion of the response to antidepressant medication. The data suggest that at least 75% of the drug response is a placebo response, but it does not tell us the magnitude of the placebo effect. What proportion of the placebo response is due to expectancies generated by placebo administration, and what proportion would have occurred even without placebo administration? That is a much more difficult question to answer. We have not been able to locate any studies in which pre- and posttreatment assessments of depression were reported for both a placebo group and a no-treatment or wait-list control group. For that reason, we turned to psychotherapy outcome studies, in which the inclusion of untreated control groups is much more common.

We acknowledge that the use of data from psychotherapy studies as a comparison with those from drug studies is far less than ideal. Participants in psychotherapy studies are likely to differ from those in drug studies on any number of variables. Furthermore, the assignment of participants to a no-treatment or wait-list control group might also effect the course of their disorder. For example, Frank (1973) has argued that the promise of future treatment is sufficient to trigger a placebo response, and a wait-list control group has been conceputalized as a placebo control group in at least one well-known outcome study (Sloane, Staples, Cristol, Yorkston, & Whipple, 1975). Conversely, one could argue that being assigned to a no-treatment control group might strengthen feelings of hopelessness and thereby increase depression. Despite these problems, the no-treatment and wait-list control data from psychotherapy outcome studies may be the best data currently available for estimating the natural course of untreated depression. Furthermore, the presence of both types of untreated control groups permits evaluation of Frank's (1973) hypothesis about the curative effects of the promise of treatment.

Study Characteristics

Studies assessing changes in depression among participants assigned to wait-list or no-treatment control groups were obtained from the computer search described earlier, supplemented by an examination of previous reviews (Dobson, 1989; Free, & Oei, 1989; Robinson, Berman, & Neimeyer, 1990). The publications that were produced by this literature search were examined by the second author, and those meeting the following criteria were included in the meta-analysis:

1. The sample was restricted to patients with a primary diagnosis of depression. Studies were excluded if participants were selected because of other criteria (eating disorders, substance abuse, physical disabilities or chronic medical conditions), as were studies in which the description of the patient population was vague (e.g., "neurotic").
2. Sufficient data were reported or obtainable to calculate within-condition effect sizes.
3. Data were reported for a wait-list or no-treatment control group.
4. Participants were assigned to experimental conditions randomly.
5. Participants were between the ages of 18 and 75.

Nineteen studies were found to meet these inclusion criteria, and in all cases, sufficient data had been reported to allow direct calculation of effect sizes as the mean posttreatment score minus the mean pretreatment score, divided by the pooled SD. Although they are incidental to the main purposes of this review, we examined effect sizes for psychotherapy as well as those for no-treatment and wait-list control groups.

Effect Sizes

Sample sizes and effect sizes for patients assigned to psychotherapy, wait-list, and no-treatment are presented in Table 4. Mean pre–post effect sizes, weighted for sample size, were 1.60 for the psychotherapy response and 0.37 for wait-list and no-treatment control groups. Participants given the promise of subsequent treatment (i.e., those in wait-list groups) did not improve more than those not promised treatment. Mean effect sizes for these two conditions were 0.36 and 0.39, respectively. The correlation between effect sizes (r = -.29) was not significant.

Table 4
Studies Including Wait-List or No-Treatment
Control Groups

Study	Psychotherapy		Control
	n	d	n	d
Beach & O'Leary (1992)	15	2.37	15	0.97
Beck & Strong (1982)	20	2.87	10	-0.28
Catanese et al. (1979)	99	1.39	21	0.16
Comas-Diaz (1981)	16	1.87	10	-0.12
Conoley & Garber (1985)	38	1.10	19	0.21
Feldman et al. (1982)	38	2.00	10	0.42
Graff et al. (1986)	24	2.03	11	-0.03
Jarvinen & Gold (1981)	46	0.76	18	0.34
Maynard (1993)	16	1.06	14	0.36
Nezu (1986)	23	2.39	9	0.16
Rehm et al. (1981)	42	1.23	15	0.48
Rude (1986)	8	1.75	16	0.74
Schmidt & Miller (1983)	34	1.25	10	0.11
Shaw (1977)	16	2.17	8	0.41
Shipley & Fazio (1973)	11	2.12	11	1.00
Taylor & Marshall (1977)	21	1.94	7	0.27
Tyson & Range (1981)	22	0.67	11	1.45
Wierzbicki & Bartlett (1987)	18	1.17	20	0.21
Wilson et al. (1983)	16	2.17	9	-0.02

Comparison of Participants in the Two Groups of Studies

Comparisons of effect sizes from different sets of studies is common in meta-analysis. Nevertheless, we examined the characteristics of the samples in the two types of studies to assess their comparability. Eighty-six percent of the participants in the psychotherapy studies were women, as were 65% of participants in the drug studies. The age range of participants was 18 to 75 years (M = 30.1) in the psychotherapy studies and 18 to 70 years (M = 40.6) in the drug studies. Duration of treatment ranged from 1 to 20 weeks (M = 4.82) in psychotherapy studies and from 2 to 15 weeks (M = 5.95) in pharmacotherapy studies. The HRS-D was used in 15 drug studies involving 2,016 patients and 5 psychotherapy studies with 191 participants. Analysis of variance weighted by sample size did not reveal any significant differences in pretreatment HRS-D scores between patients in the drug studies (M = 23.93, SD = 5.20) and participants in the psychotherapy studies (M = 21.34, SD = 5.03). The Beck Depression Inventory (BDI) was used in 4 drug studies involving 261 patients and in 17 psychotherapy studies with 677 participants. Analysis of variance weighted by sample size did not reveal any significant differences in pretreatment BDI scores between participants in drug studies (M = 21.58, SD = 8.23) and those in psychotherapy studies (M = 21.63, SD = 6.97). Thus, participants in the two types of studies were comparable in initial levels of depression. These analyses also failed to reveal any pretreatment differences as a function of group assignment (treatment or control) or the interaction between type of study and group assignment.

Estimating the Placebo Effect

Just as drug effects can be estimated as the drug response minus the placebo response, placebo effects can be estimated as the placebo response minus the no-treatment response. Using the effect sizes obtained from the two meta-analyses reported above, this would be 0.79 (1.16 - 0.37). Figure 2 displays the estimated drug, placebo, and no-treatment effect sizes as proportions of the drug response (i.e., 1.55 SDs). These data indicate that approximately one quarter of the drug response is due to the administration of an active medication, one half is a placebo effect, and the remaining quarter is due to other nonspecific factors.

Figure 2. Drug effect, placebo effect, and natural history effect as proportions of the response to antidepressant medication.

Discussion

No-treatment effect sizes and effect sizes for the placebo response were calculated from different sets of studies. Comparison across different samples is common in meta-analyses. For example, effect sizes derived from studies of psychodynamic therapy are often compared to those derived from studies of behavior therapy (e.g., Andrews & Harvey, 1981; Smith et al., 1980). Nevertheless, comparisons of this sort should be interpreted cautiously. Participants volunteering for different treatments might come from a different populations, and when data for different conditions are drawn from different sets of studies, participants have not been assigned randomly to these conditions. Also, assignment to a no-treatment or wait-list control group is not the same as no intervention at all. Therefore, our estimates of the placebo effect and natural history component of the response to antidepressant medication should be considered tentative. Nevertheless, when direct comparisons are not available, these comparisons provide the best available estimates of comparative effectiveness. Furthermore, in at least some cases, these estimates have been found to yield results that are comparable to those derived from direct comparisons of groups that have been randomly assigned to condition (Kirsch, 1990; Shapiro & Shapiro, 1982).

Unlike our estimate of the effect of natural history as a component of the drug response, our estimate of the placebo response as a proportion of the drug response was derived from studies in which participants from the same population were assigned randomly to drug and placebo conditions. Therefore, the estimate that only 25% of the drug response is due to the administration of an active medication can be considered reliable. Confidence in the reliability of this estimate is enhanced by the exceptionally high correlation between the drug response and the placebo response. This association is high enough to suggest that any remaining variance in drug response is error variance associated with imperfect reliability of measurement. Examining estimates of active drug and inactive placebo responses as a function of drug type further enhances confidence in the reliability of these estimates. Regardless of drug type, the inactive placebo response was approximately 75% of the active drug response.

We used very stringent criteria in selecting studies for inclusion in this meta-analysis, and it is possible that data from a broader range of studies would have produced a different outcome. However, the effect size we have calculated for the medication effect (D = .39) is comparable to those reported in other meta-analyses of antidepressant medication (e.g., Greenberg et al., 1992, 1994; Joffe, Sokolov, & Streiner, 1996; Quality Assurance Project, 1983; Smith et al., 1980; Steinbrueck, Maxwell, & Howard, 1983). Comparison with the Joffe et al. (1996) meta-analysis is particularly instructive, because that study, like ours, included estimates of pre–post effect sizes for both drug and placebo. Although only two studies were included in both of these meta-analyses and somewhat different calculation methods were used, ² their results were remarkably similar to ours. They reported mean pre–post effect sizes of 1.57 for medication and 1.02 for placebo and a medication versus placebo effect size of .50.

Our results are in agreement with those of other meta-analyses in revealing a substantial placebo effect in antidepressant medication and also a considerable benefit of medication over placebo. They also indicate that the placebo component of the response to medication is considerably greater than the pharmacological effect. However, there are two aspects of the data that have not been examined in other meta-analyses of antidepressant medication. These are (a) the exceptionally high correlation between the placebo response and the drug response and (b) the effect on depression of active drugs that are not antidepressants. Taken together, these two findings suggest the possibility that antidepressants might function as active placebos, in which the side-effects amplify the placebo effect by convincing patients of that they are receiving a potent drug.

In summary, the data reviewed in this meta-analysis lead to a confident estimate that the response to inert placebos is approximately 75% of the response to active antidepressant medication. Whether the remaining 25% of the drug response is a true pharmacologic effect or an enhanced placebo effect cannot yet be determined, because of the relatively small number of studies in which active and inactive placebos have been compared (Fisher & Greenberg, 1993). Definitive estimates of placebo component of antidepressant medication will require four arm studies, in which the effects of active placebos, inactive placebos, active medication, and natural history (e.g., wait-list controls) are examined. In addition, studies using the balanced placebo design would be of help, as these have been shown to diminish the ability of subjects to discover the condition to which they have been assigned (Kirsch & Rosadino, 1993).

¹ A reviewer suggested that because effect sizes are essentially z-scores in a hypothetically normal distribution, one might use percentile equivalents when examining the proportion of the drug response duplicated by the placebo response. As an example of why this should not be done, consider a treatment that improves intelligence by 1.55 SDs (which is approximately at the 6^th percentile) and another that improves it by 1.16 SDs (which is approximately at the 12^th percentile). Our method indicates that the second is 75% as effective as the first. The reviewer's method suggests that it is only 50% as effective. Now let's convert this to actual IQ changes and see what happens. If the IQ estimates were done on conventional scales (SD = 15), this would be equivalent to a change of 23.25 points by the first treatment and 17.4 points by the second. Note that the percentage relation is identical whether using z-scores or raw scores, because the z-score method simply divides both numbers by a constant.

² Instead of dividing mean differences by the pooled SDs, Joffe et al. (1996) used baseline SDs, when these were available, in calculating effect sizes. When baseline SDs were not available, which they reported to be the case for most of the studies they included, they used estimates taken from other studies. Also, they used a procedure derived from Hedges and Olkin (1995) to weight for differences in sample size, whereas we used the more straightforward method recommended by Hunter and Schmidt (1990).

Prevention & Treatment, Volume 1, Article 0002a, posted June 26, 1998
Copyright 1998 by the American Psychological Association