believe in astrology.
Source: Correlation, Northern Winter 1996/97, 15(2), p. 14-20.
The Astrotest
A tough match for astrologers
by Rob Nanninga
--------------------------------------------------------------------------------
Introduction
Astrology text books contain many testable statements. To take an
example: it is said that people with the Sun in an Air-sign (Gemini,
Libra and Aquarius) are more thinking types than those with the Sun in
a Water-sign (Cancer, Scorpio, Pisces). Thus we might expect that
among skeptics Air signs are more numerous than Water signs. To my
knowledge this hypothesis has not yet been tested. Nevertheless, it
seems likely that it can be disproved because many similar tests have
failed to show any relationship between given Sun signs and certain
personality traits or vocation (e.g. Bastedo, R.W. 1978; Culver &
Ianna, 1988; Gauquelin, 1982, 1988; Hentschel, 1985; McGervey, 1977;
Startup, M. 1984; Tyson, 1980, 1984; Van Rooij, 1993).
Astrologers strongly object to these kind of tests. They emphasize
that one should always look at the whole chart. The Sun sign is only
one factor that is influenced by numerous other astrological factors.
The system as a whole does not equal the sum of it's parts, because
these parts are often in conflict with one another. The influence of
the Sun-sign can easily be cancelled out by the position of the Moon
or anything else. Therefore, one should never isolate one particular
factor, but try to interpret the whole chart.
Apparently, this implies that a group of a thousand Scorpions do not
have more in common then a random group with mixed Sun signs. If this
is true, we may wonder why astrological textbooks continue to tell us
that Scorpions are secretive, passionate, resentful and vindictive. We
may also wonder how astrologers have ever found out the meaning of the
different Sun signs.
Fortunately, we don't have to argue about this issue because there are
tests in which astrologers can use the whole chart. I am referring to
the so-called matching test. One of the best examples was conducted by
the Australian researcher Dr. Geoffrey Dean (1987). Using the Eysenck
Personality Inventory, Dean selected 60 people with a very high
introversion score and 60 people with a very high extraversion score.
Next, he supplied 45 astrologers with the birth charts of these 120
subjects. By analysing the charts the astrologers tried to identify
the extroverts from the introverts. The results were very
disappointing. It was as if the astrologers had tossed coins to
determine their choices. Their average success rate was only 50.2
percent.
As might be expected, astrologers do not like this type of test
either. They argue that they need more information on the subjects so
as to successfully accomplish their matching task. An extreme
extraversion score on the EPI is apparently not enough. Many
astrologers doubt the value of personality tests. How can we overcome
this hurdle? The best way to go about this is to ask astrologers what
kind of information they require. This strategy was used by John
McGrew and Richard McFall (1990), two psychologists of Indiana
University. The 'Astrotest' that I am about to discuss is similar to
their approach.
Devising the Astrotest
In May 1994 the Algemeen Dagblad (General Daily), a Dutch national
newspaper, published an article by Martin Boot, a former astrologer
who now works as a psychotherapist. Boot argued that astrologers
cannot predict. In response, the astrologer Rene Jelsma wrote a
rebuttal that was published under the heading "Astrologer can really
predict".
I decided to offer an experimental solution to this difference of
opinion by inviting astrologers to take part in the Astrotest of the
Skepsis Foundation. My proposal was as follows: All participants will
receive the birth data (date, time and place) of seven anonymous test
subjects. They will also receive seven questionnaires filled out by
these subjects. The questions will be devised by the participants.
They have to match each birth chart with the corresponding
questionnaire. To encourage participation, I offered 5000 guilders
(2500 dollars) to those who were able to match all seven charts.
The response was beyond expectation. More than 70 people showed
willingness to participate. I sent them a brief questionnaire to
determine how much experience they had in astrology. I also asked them
to supply me with a number of questions that I could put to my test
subjects. Later on, more astrologers volunteered. Some of them
replaced those who had not returned my questionnaire. It was decided,
however, that the maximum number of astrologers should be 50, because
we wanted to limit our risk of loosing the 5000 guilders. Even without
any astrological knowledge, each participant would have a chance of 1
in 5040 to obtain the money.
It was not easy to find suitable test subjects. Initially, I collected
people who had all been born in 1948. The president of the NGPA (Dutch
Society of Practising Astrologers), however, pointed out that in those
days the registered birthtimes were often rounded to the full or half
hour. Therefore, I decided to use subjects that were born around 1958.
After I had obtained their birth certificates, I used a computer
program to calculate their charts. Then I noticed that four of my
subjects had an ascendent that was near the cusp between two signs. If
they were born five minutes later or earlier than the time that was
registered in their birth certificate, they would have had a different
ascendending sign. Because I could not be sure that the birth times
were correct to the minute, I felt obliged to replaced these subjects
with others with a less ambiguous ascendent.
On the average the participants sent me ten questions to be answered
by the test subjects, leaving aside those that did not formulate any
questions. I synthesized the questions into a list of 25 that covered
several aspects of the subjects life: their education, vocation,
hobbies, interests, main goals, personality, relationships, health,
religion, etc. In addition, I asked them for the dates of some
important events in their life, because many participants had shown an
interest in specific dates. Finally, I added 24 multiple choice
questions that I had taken from the Berkeley Personality Profile.
Eight experienced astrologers were asked for their opinion about the
questionnaire. They had no major objections against it. At the
suggestion of one of them I added three multiple choice questions
covering the family background of the subjects. Finally, the
questionnaires and the list of birth data were sent to a few skeptics
who tried to find the matching pairs. Although one of them scored 3
hits, there was no reason to suspect that any of the pairs could be
identified by using hidden clues.
Expectations vs results
In the beginning of December 1994 all data were sent to the 50
participants of the Astrotest. They had ten weeks to complete the
test. I allowed them to formulate an additional question after
studying the charts, but only one person did.
Eventually, 44 astrologers completed the test. Many of them had much
experience. Half of the participants had read at least fifty books on
astrology. Three quarters had taken a course in astrology and one
quarter was training others. At least half of the participants had
analyzed over a hundred astrological charts and one-third was
frequently paid for it's services. One quarter was member of the Dutch
Society of Practising Astrologers (NGPA).
The astrologers were asked to indicate how many correct matches they
would have expected. There were 36 participants who revealed their
expectations. Half of them predicted that they had matched all
subjects with the correct charts. Only six astrologers expected less
than four hits.
In fact, the most successful astrologer achieved only three correct
matches, whereas half of the participants (22) did not score a single
hit. The average number of hits was 0.75. This is 0.25 below the mean
change expectation (MCE), a deviation that is not significant.
Moreover, there was no evidence that the most experienced astrologers
did any better than beginners.
It is interesting to compare the entries of the participants with each
other. Because they all had received the same information, one would
expect many similar responses. Actually, the lack of agreement was
striking. Each of the seven charts could be paired with seven
questionnaires. Of these 49 possible combinations, none was selected
more than twelve times. It was as if each astrologer had used a random
generator to determine the correct matches. There were only two
astrologers who had independently arrived at the same solution (p =
0.18). Two other entries were also identical, but in this case the
participants had joined forces.
Unconvincing excuses
Afterwards, I sent all participants a short questionnaire to find out
more about the impact of their negative results. Nine of the 22
respondents confessed that they were surprised by the lack of
agreement between the participants. Four admitted that the
possibilities of astrology were more limited then they had thought,
and seven came to the conclusion that astrology only works in actual
practice. Nevertheless, sixteen respondents still believed that
science can prove astrology right.
I asked the astrologers which factors might be responsible for the
disappointing results. Ten respondents found the horoscopes too much
alike. They pointed out that Saturn, Uranus, Neptune and Pluto were
nearly always located in the same astrological sign. Uranus, for
instance, was always in Leo. This is not surprising because Uranus
needs seven years to move from one sign to the next. The age
differences between the test subjects were necessarily much smaller,
because the questionnaires contained much information that could be
used to estimate their age. Anyhow, the horoscopes showed numerous
differences that were hard to overlook.
Ten respondents complained that the answers of the subjects were often
similar. For instance, in reply to a multiple choice question about
their job, all subjects claimed to be reliable workers. But again,
there were many more differences than similarities. Moreover, even in
those cases when most subjects expressed a similar opinion, there were
usually dissenters. For example, although four subjects had a vivid
imagination and were fond of aesthetic experiences, there was one
subject who showed not the slightest appreciation for such
experiences. Why didn't his chart stand out from the others?
Nine respondents suspected that the questions were not always answered
truthfully. Perhaps the subjects had a lack of self-knowledge or tried
to present a flattering image of themselves. However, in spite of this
potential bias, most questions concerned verifiable facts. There is no
reason to assume that the subjects lied about their hobbies or the
date of their wedding. Even if all facts were false (including the
birth dates), this would not explain why the astrologers failed to
demonstrate mutual agreement.
The majority of the respondents (13) asserted that the subjects had
not supplied enough information. We cannot rule out the possibility
that the chance results were due to a lack of relevant details. It
should be remarked, however, that nearly all participants had asked
for less information than they actually received. One might just as
well argue that they could not see the wood for the trees.
Is there a way out?
If astrology really works, it should be possible to design a test that
satisfies both scientists and astrologers. Unfortunately, astrologers
never specify what kind of test would be acceptable to them. They
argue that astrological research is very difficult because scientific
methods are too crude, intrusive or mechanical. Apparently, the
alleged astrological effects are so subtle and hard to detect that we
may wonder how astrologers had ever been able to identify them.
In order to explain the lack of objective evidence, some leading Dutch
astrologers (Van Assem, 1993) resorted to the paranormal. They believe
that astrologers can only arrive at the correct interpretation of a
chart by using their higher intuition or by tuning in with the cosmic
order. This intuition can solely be used during authentic
consultations. As soon as a scientist interferes by selecting clients
or posing questions, it disappears. Presumably, scientists are not
part of the cosmic order.
According to Geoffrey Dean and Arthur Mather (1994) scientists and
astrologers are in conflict over whether astrology works because they
tend to look at different things: scientists are concerned with
accuracy whereas astrologers look for client satisfaction. Astrologers
tend to believe that their clients are satisfied because astrology is
accurate. This implies that clients would be less satisfied if the
astrologer had inadvertently used an incorrect birth date. Astrology
is in conflict with science as long as it's claims are testable. An
increasing number of astrologers manage to escape from scientific
scrutiny by confining themselves to statements that can not be
falsified. They assert that a horoscope can only provide information
about our inner life, basic nature, true reality, hidden potential,
deepest aspirations, unconscious fears and motives, forgotten trauma's
and possibilities for future development. Astrology, they say, gives
insight into the meaning and quality of our subjective experiences by
locating them within a cosmic frame of reference. It tells us why we
are the way we are, it can clarify our problems and help us to find
solutions - but it can not predict our behaviour. Astrological
interpretations do not even necessarily fit with the introspection by
the person involved, so nothing can prove them wrong.
The most "progressive" astrologers regard astrology as a counselling
skill. To them the horoscope is only a therapeutic tool and not a
source of reliable information.
References
Bastedo, R.W. (1978). An emperical test of popular astrology.
Skeptical Inquirer, 3(1), 17-38.
Culver, R.B. & Ianna, P.A. (1984). Astrology: true or false? Buffalo:
Prometheus.
Dean, G. (1987). Does astrology need to be true? Scientific Inquirer,
11(3), 257-273.
Dean, G. & Mather, A. (1994). Is the scientific approach relevant to
astrology? Correlation, 13(1), 11-18.
Gauquelin, M. (1982). Zodiac and personality: an emperical study.
Skeptical Inquirer, 6(3), 57-65.
Gauquelin, M. (1988). Written in the stars. Wellingborough: Aquarian
Press, p. 252.
Hentschel, U. & Kiessling, M. (1985). Season of birth and personality:
another instance of noncorrespondence. Journal of Social Psychology,
125(5), 577-585.
McGervey, J.D. (1977). A statistical test of sun-sign astrology. The
Zetetic, 1(2), 49-54.
McGrew, J.H. & McFall, R.M. (1990). A scientific inquiry into the
validity of astrology. Journal of Scientific Exploration, 4(1), 75-83.
Startup, M. (1984). Personality and planetary positions at birth: an
attempted replication with ordinary people. Correlation, 4(2), 4-13.
Tyson, G.A. (1980). Occupation and astrology or season of birth: a
myth? The Journal of Social Psychology, 110, 73-78. Tyson, G.A.
(1984). An emperical test of the astrological theory of personality.
Personality and Individual Differences, 5(2), 247-250.
Van Assem, L. (1993). The astrologer's philosophy of life.
Correlation, 12(1), 52-54.
Van Rooij, J.J.F. (1993). Jungian typology and astrology: an emperical
test. Correlation, 12(1), 28-32.
No comments:
Post a Comment