by Matthew Barsalou, guest blogger
In Jim Frost’s article “How to Be a Ghost Hunter with a Statistical Mindset,” he correctly pointed out the difficulties in distinguishing small effects from natural variation. However, he did not mention the benefits of doing measurement system analysis (MSA) in both ghost hunting as depicted by his example and in the statistical study using Minitab.
In industrial settings, testing equipment is evaluated to determine if the device used to assess the factor being studied is taking accurate measurements. In other words, are you collecting data that you can trust?
By doing a statistical study without assessing your measurement tools first, you risk using a measuring device that may not be sensitive enough to measure the phenomenon under investigation. For example, in industry this could mean measuring machined metal blocks with a variation of thousandths of a milliliter with calipers that can only measure hundredths of a millimeter. The resulting measurements might show the variation due to the calipers, but not the variation of the blocks being measured.
The same principle applies to ghost hunting with electromagnetic field (EMF) detectors. Are the EMF results due to a ghost or background EMF readings? As ghosts are not known to exist, it would be premature to conclude that they give off EMF readings.
Just as you’d want to assess the ability of the caliper to measure the metal block, an analysis should be performed on EMF detectors. A simple approach would be randomly selecting 10 houses not known to be haunted and 10 houses thought to be haunted. Naturally, each group of 10 houses should be approximately the same as each other in all factors, except that 10 are thought to be haunted. A statistical analysis could be performed using Minitab to determine if a statistically significant difference exists between EMF readings from suspected haunted houses and those from randomly selected houses.
The experimental conditions of a ghost hunt should also be considered. Ghost hunting is often done with “lights out” to increase the sensitivity of the ghost hunter's senses. But does it actually increase the sensitivity of the ghost hunter's senses? This should be empirically verified before the start of the ghost hunting. People may be more sensitive to unusual observations when in the dark in a place thought to be haunted. However, a person may just be subconsciously more susceptible to unusual observations because they believe they are in a haunted house.
As part of the measurement system analysis (MSA) often performed in industry, we try to ensure that there is no unacceptable operator error in our measurement results. The same principle could be applied to ghost hunting. Potential ghost hunters can be sent to investigate the 20 houses used to assess our EMF detectors.
Blinding and randomization should be used to increase accuracy. An experimenter should randomly select the order in which the 20 houses would be investigated and a second person who is unaware of the status of each house should give the ghost hunter the list of addresses and a check list to identify any unusual observations that are made in the houses. This ensures the second experimenter does not inadvertently give the ghost hunter clues about the status of the houses. The houses should all be investigated at the same time of night and in the dark.
At the end of the study the checklist from the ghost hunters would be analyzed using statistical software to determine if a statistically significant difference exists between unusual observations made in suspected haunted houses and those in randomly selected houses.
If the results show no difference between unusual observations from known haunted houses and randomly selected houses, it could be an indication that the methodology for ghost hunting should be revised. The results of EMF detectors in haunted houses should also be called into question if the analysis shows no difference in results between known haunted houses and randomly selected houses.
To sum up, just like as do in industry, ghost hunters should confirm their methodology works before gathering their data. Then they can improve their methodology if it is not sufficient, or be more open to Frost's null hypothesis: Ghosts do not exist.