Ganzfeld experiment
A ganzfeld experiment (from the German word for “entire field”) is a pseudoscientific technique used in parapsychology to test individuals for extrasensory perception (ESP). The ganzfeld experiments are among the most recent in parapsychology for testing telepathy.[1] Consistent, independent replication of ganzfeld experiments has not been achieved.[2][3][4][5][6]
Historical context
The ganzfeld was originally introduced into experimental psychology due to the experiments of the German psychologist Wolfgang Metzger (1899–1979) on the perception of a homogenous visual field.[7] In the early 1970s, Charles Honorton had been investigating ESP and dreams at the Maimonides Medical Center and began using the ganzfeld technique to achieve a state of sensory deprivation in which he hypothesised that psi could work.[8] Honorton believed that by reducing the ordinary sensory input, psi conductive states may be enhanced and psi-mediated information could be transmitted.[7]
Since the first full experiment was published by Honorton and Sharon Harper in the Journal of the American Society for Psychical Research in 1974, the Ganzfeld has remained a mainstay of parapsychological research.
Experimental procedure
In a typical ganzfeld experiment, a "receiver" is placed in a room relaxing in a comfortable chair with halved ping-pong balls over the eyes, having a red light shone on them. The receiver also wears a set of headphones through which white or pink noise (static) is played. The receiver is in this state of mild sensory deprivation for half an hour. During this time, a "sender" observes a randomly chosen target and tries to mentally send this information to the receiver. The receiver speaks out loud during the 30 minutes, describing what he or she can "see". This is recorded by the experimenter (who is blind to the target) either by recording onto tape or by taking notes, and is used to help the receiver during the judging procedure.
In the judging procedure, the receiver is taken out of the Ganzfeld state and given a set of possible targets, from which they select one which most resembled the images they witnessed. Most commonly there are three decoys along with the target, giving an expected rate of 25%, by chance, over several dozens of trials.[9]
Analysis of results
Early experiments
Between 1974 and 1982, 42 ganzfeld experiments were performed.[10][11] In 1982, Charles Honorton presented a paper at the annual convention of the Parapsychological Association that summarized the results of the ganzfeld experiments up to that date, and concluded that they represented sufficient evidence to demonstrate the existence of psi. Ray Hyman, a psychologist, disagreed. The two men later independently analyzed the same studies, and both presented meta-analyses of them in 1985.[10][12]
Hyman criticized the ganzfeld papers for not describing optimal protocols, nor including the appropriate statistical analysis. He presented a factor analysis that he said demonstrated a link between success and three flaws, namely: flaws in randomization for choice of target; flaws in randomization in judging procedure; and insufficient documentation. Honorton asked a statistician, David Saunders, to look at Hyman's factor analysis and he concluded that the number of experiments was too small to complete a factor analysis.[13]
The ganzfeld studies examined by Hyman and Honorton had methodological problems that were well documented. Honorton reported only 36% of the studies used duplicate target sets of pictures to avoid handling cues.[14] Hyman discovered flaws in all of the 42 ganzfeld experiments and to assess each experiment, he devised a set of 12 categories of flaws. Six of these concerned statistical defects, the other six "covered procedural flaws such as inadequate randomization, inadequate security, possibilities of sensory leakage, and inadequate documentation."[15] Over half of the studies failed to safeguard against sensory leakage and all of the studies contained at least one of the 12 flaws. Because of the flaws, Honorton agreed with Hyman the 42 ganzfeld studies could not support the claim for the existence of psi.[15]
In 1986, Hyman and Honorton published A Joint Communiqué which agreed on the methodological problems and on ways to fix them. They suggested a computer-automated control, where randomization and the other methodological problems identified were eliminated. Hyman and Honorton agreed that replication of the studies was necessary before final conclusions could be drawn. They also agreed that more stringent standards were necessary for ganzfeld experiments, and they jointly specified what those standards should be.[16][17]
Autoganzfeld
In 1982 Honorton had started a series of autoganzfeld experiments at his Psychophysical Research Laboratories (PRL). These studies were specifically designed to avoid the same potential problems as those identified in the 1986 joint communiqué issued by Hyman and Honorton. The PRL trials continued until September 1989. In 1990 Honorton et al. published the results of 11 autoganzfeld experiments they claimed met the standards specified by Hyman and Honorton (1986).[18] In these experiments, 240 participants contributed 329 sessions.[19]
Hyman analyzed these experiments and wrote they met most, but not all of the "stringent standards" of the joint communiqué.[20] He expressed concerns with the randomization procedure, the reliability of which he was not able to confirm based on the data provided by Bem. Hyman further noted that although the overall hit rate of 32% was significant, the hit rate for static targets (pictures) was in fact insignificant (inconsistently with previous ganzfeld research). The overall significance of the experiments was solely due to dynamic targets (videos).[20] In the hit rates regarding these dynamic targets, however, some interesting patterns were found that implied visual cues may have been leaked:
The most suspicious pattern was the fact that the hit rate for a given target increased with the frequency of occurrence of that target in the experiment. The hit rate for the targets that occurred only once was right at the chance expectation of 25%. For targets that appeared twice the hit rate crept up to 28%. For those that occurred three times it was 38%, and for those targets that occurred six or more times, the hit rate was 52%. Each time a videotape is played its quality can degrade. It is plausible then, that when a frequently used clip is the target for a given session, it may be physically distinguishable from the other three decoy clips that are presented to the subject for judging.[15]
Hyman wrote these studies were an improvement over their older counterparts, but were not a successful replication of the ganzfeld experiments, nor a confirmation of psi.[20] He concluded the autoganzfeld experiments were flawed because they did not preclude the possibility of sensory leakage.[15]
Richard Wiseman published a paper discussing a non-psi hypothesis based on possible sender to experimenter acoustic leakage in the autoganzfeld to account for the results.[21][22] David Marks has written "Wiseman and his colleagues identified various different ways in which knowledge of the target could have been leaked to the experimenter. These included cues from the videocassette recorder and sounds from the sender who, of course, knew the target's identity... their conclusions provide little reassurance that sensory cueing of the experimenter was in any way substantially blocked."[6]
Milton and Wiseman (1999) carried out a meta-analysis of ganzfeld experiments in other laboratories. They found no psi effect; the results showed no effect greater than chance from a database of 30 experiments and a non-significant Stouffer Z of 0.70.[23]
Lance Storm and Suitbert Ertel (2001) published a meta-analysis of 79 studies published between 1974 and 1996 and concluded the positive statistically significant overall outcome indicates a psi effect.[24] In response, Milton and Wiseman (2001) wrote the meta-analysis of Storm and Ertel was not an accurate quantitative summary of ganzfeld research as they had included early studies which had been widely recognized as having methodological problems which make it impossible to interpret the results as evidence of a psi effect.[25]
Another meta-analysis was conducted by Daryl Bem, John Palmer, and Richard Broughton in which the experiments were sorted according to how closely they adhered to a pre-existing description of the ganzfeld procedure. Additionally, ten experiments that had been published in the time since Milton and Wiseman's deadline were introduced. Now the results were significant again with a Stouffer Z of 2.59.[26][27]
Contemporary research
The ganzfeld procedure has continued to be refined over the years. In its current incarnation, an automated computer system is used to select and display the targets ("digital autoganzfeld"). This overcomes many of the shortcomings of earlier experimental setups, such as randomization and experimenter blindness with respect to the targets.[28]
In 2010, Lance Storm, Patrizio Tressoldi, and Lorenzo Di Risio analyzed 29 ganzfeld studies from 1997 to 2008. Of the 1,498 trials, 483 produced hits, corresponding to a hit rate of 32.2%. This hit rate is statistically significant with p < .001. Participants selected for personality traits and personal characteristics thought to be psi-conducive were found to perform significantly better than unselected participants in the ganzfeld condition.[29] Hyman (2010) published a rebuttal to Storm et al. Hyman wrote the ganzfeld studies have not been independently replicated and have failed to produce evidence for psi.[30] According to Hyman, "reliance on meta-analysis as the sole basis for justifying the claim that an anomaly exists and that the evidence for it is consistent and replicable is fallacious. It distorts what scientists mean by confirmatory evidence." Storm et al. published a response to Hyman claiming the ganzfeld experimental design has proved to be consistent and reliable but parapsychology is a struggling discipline that has not received much attention so further research on the subject is necessary.[31] Rouder et al. in 2013 wrote that critical evaluation of Storm et al.'s meta-analysis reveals no evidence for psi, no plausible mechanism and omitted replication failures.[32]
A 2016 paper examined questionable research practices in the ganzfeld experiments.[33]
Psi-conducive variables
Bem and Honorton (1994) investigated certain personality traits and characteristics as potential psi-conducive variables which they suggested play an important role in claimed ESP performance.[19] According to parapsychologists these factors are thought to be positively correlated with increased scores in ganzfeld experiments, as compared to unselected participants.[29] Traits and characteristics of subjects thought to increase the chance of obtaining a successful hit rate in a psi experiment include:
- Positive belief in psi; ESP[34]
- Prior psi experiences[19]
- Practicing a mental discipline such as meditation[26]
- Creativity[19]
- Artistic ability[19]
- Emotional closeness with the sender[35]
While there are a number of reasons that researchers avoid special participants and sample only normal populations, these factors are important considerations in future replications of ganzfeld experiment, and may be useful in predicting the outcome of these studies.[29]
Criticism
There are several common criticisms of some or all of the ganzfeld experiments:
- Isolation – Richard Wiseman and others argue that not all of the studies used soundproof rooms, so it is possible that when videos were playing, the experimenter could have heard it, and later given involuntary cues to the receiver during the selection process. It could even have been possible that the receiver themselves could hear the video.[21]
- Randomization – When subjects are asked to choose from a variety of selections, there is an inherent bias to choose the first selection they are shown. If the order in which they are shown the selections is randomized each time, this bias will be averaged out. The randomization procedures used in the experiment have been criticized for not randomizing satisfactorily.[36]
- The psi assumption – The assumption that any statistical deviation from chance is evidence for telepathy is highly controversial. Strictly speaking, a deviation from chance is only evidence that either this was a rare, statistically unlikely occurrence that happened by chance, or something was causing a deviation from chance. Flaws in the experimental design are a common cause of this, and so the assumption that it must be telepathy is fallacious.[37]
Writing in 1985, C. E. M. Hansel discovered weaknesses in the design and possibilities of sensory leakage in the ganzfeld experiments reported by Carl Sargent and other parapsychologists. Hansel concluded the ganzfeld studies had not been independently replicated and that "ESP is no nearer to being established than it was a hundred years ago."[38]
David Marks in his book The Psychology of the Psychic (2000) has noted that during the autoganzfeld experiments the experimenter sat only fourteen feet from the sender's room. Soundproofing tiles were eventually added but they were designed to "absorb sound not to prevent transmission." According to Marks this was inadequate and no different than using any standard internal wall. The door and door frame were also a possible source of sensory leakage and none of these problems were ever eliminated.[6]
Terence Hines wrote in 2003 that the ganzfeld studies could not be said to provide evidence for psi as the alleged evidence disappears as the tightness of experimental controls is increased. As research progresses variables in science become clearer as more studies are published that describe under what specific condition the particular effect can be demonstrated. This is in opposition to the ganzfeld studies. According to Hines, there was "no clear way to obtain results showing any psychic phenomenon reliably" and that "the most reasonable conclusion" was that the effect did not exist and had never existed.[39]
In a 2007 review, Ray Hyman wrote that parapsychologists agree they have no positive theory of psi as it is negatively defined as any effect that cannot be currently explained in terms of chance or normal causes. Hyman saw this as a fallacy, as it encouraged parapsychologists to use any peculiarity in the data as a characteristic of psi. Hyman also wrote that parapsychologists have admitted it is impossible to eliminate the possibility of non-paranormal causes in the ganzfeld experiment. There is no independent method to indicate the presence or absence of psi.[15]
Until parapsychologists can provide a positive way to indicate the presence of psi, the different effect sizes that occur in experiments are just as likely to result from many different things rather than one thing called psi. Indeed given the obvious instability and elusiveness of the findings, the best guess might very well be that we are dealing with a variety of Murphy's Law rather than a revolutionary anomaly called psi.
— Ray Hyman, Evaluating Parapsychological Claims, 2007[15]
In their book 50 Great Myths of Popular Psychology (2011), Scott O. Lilienfeld and colleagues have written that the ganzfeld being a reliable technique is far from being resolved. They concluded that ESP has not been successfully demonstrated in experiments for over 150 years so this is hardly encouraging.[40]
In a 2013 podcast, Brian Dunning reviewed the flaws of the ganzfeld studies and came to the conclusion the technique had failed as evidence for psi and interest in ganzfeld has declined.[41]
Controversy
In 1979, Susan Blackmore visited the laboratories of Carl Sargent in Cambridge. She noticed a number of irregularities in the procedure and wrote about them for the Journal of the Society for Psychical Research.
It now appeared that in one session – number 9 – the following events had taken place.
- Sargent did the randomization when he should not have.
- A 'B' went missing from the drawer during the session, instead of afterwards.
- Sargent came into the judging and "pushed" the subject towards 'B'.
- An error of addition was made in favour of 'B' and 'B' was chosen.
- 'B' was the target and the session a direct hit.[42]
This article, along with further criticisms of Sargent's work from Adrian Parker and Nils Wiklund remained unpublished until 1987 but all were well known in parapsychological circles. Sargent wrote a rebuttal to these criticisms (also not published until 1987)[43] in which he did not deny what Blackmore had observed, but argued that her conclusions based on those observations were wrong and prejudiced. His co-workers also responded, saying that any deviation from protocol was the result of “random errors” rather than any concerted attempt at fraud.[44] Carl Sargent stopped working in parapsychology after this and did not respond "in a timely fashion" when the Council of the Parapsychological Association asked for his data, and so his membership of that organization was allowed to lapse.[45]
Writing for Skeptical Inquirer in 2018, Blackmore states that Sargent "deliberately violated his own protocols and in one trial had almost certainly cheated." Psychologists reading Daryl Bem's review in Psychological Bulletin would "not have a clue that serious doubt had been cast on more than a quarter of the studies involved" Sargent and Chuck Honortons. When Blackmore confronted Sargent, he told her "it wouldn't matter if some experiments were unreliable because, after all, we know that psi exists". Blackmore also recounts having a discussion with Bem at a consciousness conference where she challenged him on his support of Sargent and Honorton's research, he replied "it did not matter". Blackmore writes, "But it does matter. ... It matters because Bem's continued claims mislead a willing public into believing that there is reputable scientific evidence for ESP in the Ganzfeld when there is not".[46]
See also
- Remote viewing
- List of parapsychology topics
- List of topics characterized as pseudoscience
- Noumenon
- Sensory deprivation
- Zener cards
References
- "Parapsychological Association Glossary of Parapsychological terms". Archived from the original on September 27, 2006. Retrieved 2006-12-19.
- Frazier, Kendrick. (1991). The Hundredth Monkey: And Other Paradigms of the Paranormal. Prometheus Books. pp. 143–48. ISBN 978-0879756550
- Hyman, Ray. (1996). The Evidence for Psychic Functioning: Claims vs. Reality. The Skeptical Inquirer. pp. 24–26. Retrieved November 4, 2013.
- Smith, Jonathan. (2009). Pseudoscience and Extraordinary Claims of the Paranormal: A Critical Thinker's Toolkit. Wiley-Blackwell. ISBN 978-1405181228. Retrieved November 1, 2013.
- Rathus, Spencer. (2011). Psychology: Concepts and Connections. Cengage Learning. p. 143. ISBN 978-1111344856
- Marks, David; Kammann, Richard. (2000). The Psychology of the Psychic. Prometheus Books. pp. 97–106. ISBN 1-57392-798-8
- Philip John Tyson, Dai Jones, Jonathan Elcock. (2011). Psychology in Social Context: Issues and Debates. Wiley-Blackwell. pp. 199–200. ISBN 978-1405168236
- Honorton & Harper (1974). "Psi-mediated imagery and ideation in an experimental procedure for regulating perceptual input". Journal of the American Society for Psychical Research (68): 156–68.
- Palmer, John (2003). "ESP in the Ganzfeld". Journal of Consciousness Studies. 10 (6–7).
- Honorton, Charles (1985). "Meta-Analysis of Psi Ganzfeld Research: A Response to Hyman". Journal of Parapsychology (49).
- Radin, Dean (2006). Entangled Minds: Extrasensory Experiences in a Quantum Reality. Paraview Pocket Books. ISBN 978-1-4165-1677-4.
- Hyman, Ray (1985). "The Ganzfeld Psi Experiments: A Critical Appraisal". Journal of Parapsychology (49).
- Saunders, David (1985). "On Hyman's Factor Analysis". Journal of Parapsychology (49).
- Milton, Julie; Wiseman, Richard (2002). "A response to Storm and Ertel (2002)". The Journal of Parapsychology. 66: 183–86.
- Ray Hyman. Evaluating Parapsychological Claims in Robert J. Sternberg, Henry L. Roediger, Diane F. Halpern. (2007). Critical Thinking in Psychology. Cambridge University Press. pp. 216–31. ISBN 978-0521608343
- Hyman, R., and Honorton, C. (1986). A joint communiqué: the psi ganzfeld controversy. Journal of Parapsychology, 50. pp. 351–64.
- Neher, Andrew (2011). Paranormal and Transcendental Experience: A Psychological Examination. Dover Publications. p. 307. ISBN 978-0486261676
- Honorton, C., Berger, R.E., Varvoglis, M.P., Quant, M., Derr, P., Schechter, E.I., & Ferrari, D.C. (1990). Psi Communication in the Ganzfeld. Journal of Parapsychology, 54. pp. 99–139.
- Bem, Daryl J.; Honorton, Charles (1994). "Does psi exist? Replicable evidence for an anomalous process of information transfer" (PDF). Psychological Bulletin. 115 (1): 4–18. doi:10.1037/0033-2909.115.1.4. Archived from the original (PDF) on 2017-03-17. Retrieved 2007-07-31.
- Hyman Ray (1994). "Anomaly or artifact? Comments on Bem and Honorton". Psychological Bulletin. 115 (1): 19–24. doi:10.1037/0033-2909.115.1.19.
- Wiseman, R., Smith, M. and Kornbrot, D. (1996). Exploring possible sender-to-experimenter acoustic leakage in the PRL autoganzfeld experiments. Journal of Parapsychology, 60. pp. 97–128.
- Humphrey, Nicholas. (1996). Soul Searching: Human Nature and Supernatural Belief. Vintage. p. 136. ISBN 978-0099273417 "Richard Wiseman, a friend and former colleague of Honorton, has subsequently reanalysed the raw data trial by trial and shown that all the positive results can be attributed to those trials in which one or other of these sources of ‘sensory leakage’ was at least a possibility. In fact, in the relatively few trials (100 in all) where such leakage of information would not have been possible the receivers did no better than chance (26 per cent correct)."
- Milton, Julie; Wiseman, Richard (1999). "Does Psi Exist? Lack of Replication of an Anomalous Process of Information Transfer" (PDF). Psychological Bulletin. 125 (4): 387–91. doi:10.1037/0033-2909.125.4.387. PMID 10414223.
- Storm, L.. & Ertel, S. (2001). Does psi exist? Comments on Milton and Wiseman's (1999) meta-analysis of ganzfeld research. Psychological. Bulletin, 127. pp. 424–33.
- Milton, Julie; Wiseman, Richard (2001). "Does psi exist? Reply to Storm and Ertel (2001)". Psychological Bulletin. 127 (3): 434–38. CiteSeerX 10.1.1.377.1535. doi:10.1037/0033-2909.127.3.434.
- Bem DJ, Palmer J, Broughton RS (September 2001). "Updating the ganzfeld database: A victim of its own success?" (PDF). Journal of Parapsychology. 65 (3): 207–18. Archived from the original (PDF) on 2006-09-14.
- James Alcock, Jean Burns and Anthony Freeman. (2003). Psi Wars: Getting to Grips with the Paranormal. p. 59. Imprint Academic. ISBN 0907845487
- Goulding, A., Westerlund, J., Parker, A., & Wackermann, J. (2004). "The first Digital Autoganzfeld study using a real-time judging procedure". European Journal of Parapsychology. 19: 66–97.CS1 maint: multiple names: authors list (link)
- Storm, Tressoldi, Di Risio (July 2010). "Meta-Analysis of Free-Response Studies, 1992–2008: Assessing the Noise Reduction Model in Parapsychology" (PDF). Psychological Bulletin. 136 (4): 471–85. doi:10.1037/a0019457. PMID 20565164. Archived from the original (PDF) on 2011-01-24. Retrieved 2010-08-18.CS1 maint: multiple names: authors list (link)
- Hyman R. (2010). "Meta-analysis that conceals more than it reveals: Comment on Storm et al. (2010)". Psychological Bulletin. 136 (4): 486–90. doi:10.1037/a0019676. PMID 20565165.
- Storm L., Tressoldi P. E., Di Risio L. (2010). "A meta-analysis with nothing to hide: Reply to Hyman (2010)". Psychological Bulletin. 136 (4): 491–94. doi:10.1037/a0019840. PMID 20565166. S2CID 21103309.CS1 maint: multiple names: authors list (link)
- Rouder J. N., Morey R. D., Province J. M. (2013). "A Bayes factor meta-analysis of recent extrasensory perception experiments: Comment on Storm, Tressoldi, and Di Risio (2010)". Psychological Bulletin. 139 (1): 241–47. doi:10.1037/a0029008. PMID 23294092.CS1 maint: multiple names: authors list (link)
- Bierman, DJ; Spottiswoode, JP; Bijl, A (2016), "Testing for Questionable Research Practices in a Meta-Analysis: An Example from Experimental Parapsychology", PLOS ONE, 11 (5): 1, doi:10.1371/journal.pone.0153049, PMC 4856278, PMID 27144889,
We consider [questionable research practices] in the context of a meta-analysis database of Ganzfeld–telepathy experiments from the field of experimental parapsychology. The Ganzfeld database is particularly suitable for this study, because the parapsychological phenomenon it investigates is widely believed to be nonexistent.
- Lawrence, T.R. (1993). "Gathering in the sheep and goats: A meta-analysis of forced-choice sheep-goat ESP studies,1947–1993". In Proceedings of the 36th Annual Convention of the Parapsychological Association: 75–86.
- Broughton, R.S.; Alexander, C.H. (1997). "AutoganzfeldII: An attempted replication of the PRL ganzfeld research". Journal of Parapsychology. 61: 209–26.
- Hyman, Ray (1994). "Anomaly or Artifact? Comments on Bem and Honorton". Psychological Bulletin. 115 (1): 19–24. doi:10.1037/0033-2909.115.1.19.
- Carroll, Robert Todd (2005). "The Skeptic's Dictionary: Psi Assumption". Retrieved 2006-06-23.
- Hansel, C. E. M. The Search for a Demonstration of ESP in Kurtz, Paul. (1985). A Skeptic's Handbook of Parapsychology. Prometheus Books. pp. 97–127
- Hines, Terence. (2003). Pseudoscience and the Paranormal. Prometheus Books. pp. 137–38. ISBN 978-1573929790
- Lilienfeld, Scott O; Lynn, Steven Jay; Ruscio, John; Beyerstein, Barry L. (2009). Myth #3 Extrasensory Perception (ESP) Is a Well Established Scientific Phenomenon. In 50 Great Myths of Popular Psychology: Shattering Widespread Misconceptions About Human Behavior. Wiley-Blackwell. ISBN 978-1405131124
- Dunning, Brian. "Skeptoid #348: Ganzfeld Experiments". Skeptoid. Retrieved November 1, 2013.
- Blackmore, Susan (1987). "A Report of a Visit to Carl Sargent's Laboratory". Journal of the Society for Psychical Research. 54: 186–98.
- Sargent, Carl (1987). "Sceptical fairytales from Bristol". Journal of the Society for Psychical Research. 54: 208–18.
- Harley, Matthews (1987). "Cheating, psi, and the appliance of science: a reply to Blackmore". Journal of the Society for Psychical Research. 54: 199–207.
- Beloff, John (1997). Parapsychology: A Concise History. Palgrave MacMillan. pp. 283–84.
- Blackmore, Susan (2018). "Daryl Bem and Psi in the Ganzfeld". Skeptical Inquirer. 42 (3): 44–45.
Further reading
- "What's the story on "ganzfeld" experiments?". The Straight Dope, December 14, 2000.
- Andrew Colman. (1995). Controversies in Psychology. Longman Pub Group. ISBN 978-0582278035
- C. E. M. Hansel. (1989). The Search for Psychic Power: ESP and Parapsychology Revisited. Prometheus Books. ISBN 978-0879755331
- Terence Hines. (2003). Pseudoscience and the Paranormal. Prometheus Books. ISBN 978-1573929790
- Nicholas Humphrey. (1996). Soul Searching: Human Nature and Supernatural Belief. Vintage. ISBN 978-0099273417
- Ray Hyman (1995). "Evaluation of Program on Anomalous Mental Phenomena". The Journal of Parapsychology.
- Hyman Ray (2010). "Meta-analysis that conceals more than it reveals: Comment on Storm et al" (PDF). Psychological Bulletin. 136 (4): 486–90. doi:10.1037/a0019676. PMID 20565165. Archived from the original (PDF) on 2013-11-03.
- Paul Kurtz. (1985). A Skeptic's Handbook of Parapsychology. Prometheus Books. ISBN 978-0879753009
- Scott O. Lilienfeld (November–December 1999). "New Analyses Raise Doubts About Replicability of ESP Findings". Skeptical Inquirer.
- Scott O. Lilienfeld, Steven Jay Lynn, John Ruscio, Barry Beyerstein. (2009). 50 Great Myths of Popular Psychology. Wiley-Blackwell. ISBN 978-1405131124
- David Marks. (2000). The Psychology of the Psychic. Prometheus Books. ISBN 978-1573927987
- Julie Milton, Richard Wiseman (2001). "Does psi exist? Reply to Storm and Ertel (2001)" (PDF). Psychological Bulletin. 127 (3): 434–38. doi:10.1037/0033-2909.127.3.434. Archived from the original (PDF) on 2013-11-06. Retrieved 2013-11-01.
- Andrew Neher. (2011). Paranormal and Transcendental Experience: A Psychological Examination. Dover Publications. ISBN 978-0486261676
- Dean Radin (1997). The Conscious Universe: The Scientific Truth of Psychic Phenomena. HarperOne. ISBN 978-0-06-251502-5.
- Spencer Rathus. (2011). Psychology: Concepts and Connections. Cengage Learning. ISBN 978-1111344856
- Gordon Stein. (1996). The Encyclopedia of the Paranormal. Prometheus Books. ISBN 978-1573920216
- Victor Stenger. (1990). Physics and Psychics: The Search for a World Beyond the Senses. Prometheus Books. ISBN 978-0879755751
- Leonard Zusne and Warren Jones. (1989). Anomalistic Psychology: A Study of Magical Thinking. Psychology Press. ISBN 978-0805805086