Previous Next Index Thread
Previous Next Index Thread
Stewart Robert Hinsley wrote [of the dubious nature of meta-analysis]: > If one does enough experiments one will eventually get one with a result > a few standard deviations ought. Combining this with the rarity of the > publication of null results gives rise to a bias. > > For example, consider a system in which there is no correlation between > a postulated cause and effect. Say 100 experiments are done, of which 80 > give a null result, 10 give a borderline positive correlation, and 10 > a borderline negative correlation. [and so on, a crisp, clear exposition of the problems inherent in meta-analyses wrongly or stupidly used] All this is clearly true, and would be piercingly salient if that were the way parapsychologists dealt with their data these days, and if that were the kind of selected data sets they had available to them. Alas, all the skeptical rejoinders on this thread have had an eerie quality of arm-chair pontificating. Is nobody interested enough to fire up the web sites I cited and check for themselves the arguments and data deployed by skeptic Ray Hyman and pro-psi statistician Jessica Utts? I ask this, at the risk of being heavy-handed and boring, because that old red herring about *discarded null result data sets* has been exhaustively studied in parapsychology, and informed skeptics such as Hyman tend to agree that it's not a big problem. (They point to lots of *other* possible problems, often fatal in their estimation.) The `file-drawer' question, as it's known, implies (for a well-known Ganzfeld meta-analysis) that parapsychologist Charles Honorton's 42 studies - of which 55 percent showed `hitting' - would need to be balanced by some 440-580 dud experiments that are not known to anyone in the field. Such studies are extremely time-consuming, and the number of labs conducting Ganzfeld research is small. It is an absurd conjecture. Besides, for more than a decade the policy among parapsychology journals has been to report *all* results, null and otherwise, precisely to obviate this suspicion. If you're crazy enough to take up career parapsychology, you at least know that you earn as many brownie points for a well-conducted null experiment as for one corroborating a psi hypothesis. I invite skeptics to take a close look at the papers by Hyman and Utts, then come back and tear into the *real* targets... Damien Broderick