----------------------------------------------------------------------

REPLY: By David R. Saunders

Last month, Steven Soter and Carl Sagan offered two counterarguments
relating to Terence Dickinson's article, "The Zeta Reticuli Incident"
(ASTRONOMY, December 1974).

Their first argument was to observe that the inclusion of connecting
lines in certain maps "is what a lawyer would call 'leading the
witness'." This was used as the minor premise in a syllogism for which
the major premise was never stated. Whether we should consider "leading
the witness" a sin or not will depend on how we conceive the purpose of
the original article. The implied analogy between ASTRONOMY magazine
and a court of law is tenuous at best; an expository article written
for a nonprofessional audience is entitled, in my opinion, to do all it
can to facilitate communication -- assuming that the underlying message
is honest. Much of what we call formal education is really little more
than "leading the witness", and no one who accepts the educational
goals objects very strongly to this process. In this context, we may
also observe that Soter's and Sagan's first argument provides another
illustrative example of "leading the witness"; the argument attacks
procedure, not substance -- and serves only to blunt the reader's
possible criticism of the forthcoming second argument. This paragraph
may also be construed as an effort to lead the witness. Once we have
been sensitized to the possibilities, none of us needs to be further
misled!

The second argument offered by Soter and Sagan does attack a
substance. Indeed, the editorial decision to publish the original
article was a responsible decision only if the issues raised by this
second line of possible argument were fully considered. Whenever a
statistical inference is made from selected data, it is crucial to
determine the strenuousness of that selection and then to appropriately
discount the apparent clarity of the inference. By raising the issue of
the possible effects of selection, Soter and Sagan are right on target.
However, by failing to treat the matter with quantitative objectivity (
by failing to weigh the evidence in each direction numerically, for
example), they might easily perform a net disservice.

In some situations, the weight of the appropriate discount will
suffice to cancel the clarity of a proposed inference -- and we will
properly dismiss the proposal as a mere capitalization on chance, or a
lucky outcome. (It is abundantly clear that Soter and Sagan regard the
star map results as just such a fortuitous outcome.) In some other
situations, the weight of the appropriate discount may be fully applied
without accounting for the clarity of the inference as a potentially
valid discovery. For example, if I proposed to infer from four
consecutive coin tosses observed as heads that the coin would always
yield heads, you would properly dismiss this proposal as unwarranted by
the data. However, if I proposed exactly the same inference based on 40
similar consecutive observations of heads, you would almost certainly
accept the inference and begin looking with me for a more systematic
explanation of the data. The crucial difference here is the purely
quantitative distinction between 4 and 40; the two situations are
otherwise identical and cannot be distinguished by any purely
qualitative argument.

When Soter and Sagan use phrases such as "some subset that
resembles", "free also to select the vantage point", "simple matter to
optimize", and "freedom to contrive a resemblance", they are speaking
qualitatively about matters that should (and can) be treated
quantitatively. Being based only on this level of argument, Soter's and
Sagan's conclusions can only be regarded as inconclusive.

A complete quantitative examination of this problem will require the
numerical estimation of at least three factors, and their expression in
a uniform metric so that wee can see which way the weight of the
evidence is leaning. The most convenient common metric will be that of
"bits of information", which is equivalent to counting consecutive
heads in the previous example.

One key factor is the degree of resemblance between the Hill map and
the optimally similar computer-drawn map. Precisely how many
consecutive heads is this resemblance equivalent to? A second key
factor is the precise size of the population of stars from which the
computer was allowed to make its selection. And a third key factor is
the precise dimensionality of the space in which the computer was free
to choose the best vantage point. If the first factor exceeds the sum
of the other two by a sufficient margin, we are justified in insisting
on a systematic explanation for the data.

The third factor is the easiest to deal with. The dimensionality of
the vantage-point space is not more than three. A property of the
metric system for weighing evidence is that each independent dimension
of freedom leads us to expect the equivalent of one more consecutive
head in the observed data. Three dimensions of freedom are worth
exactly 3.0 bits. In the end, even three bits will be seen as
relatively minor.

The second factor might be much larger than this, and deserve
relatively more discussion. The appropriate discount for this selection
will be log2C, where C is the number of distinct combinations of stars
"available" to the computer. If we were to agree that C must represent
the possible combinations of 46 stars taken 14 at a time, then log2C
would be 37.8 bits; this would be far more than enough to kill the
proposed inference. However, not all these combinations are equally
plausible. We really should consider only combinations that are
adjacent to one another and to the sun, but it is awkward to try to
specify exactly which combinations these are.

The really exciting moment in working with these data came with the
realization that in the real universe, our sun belongs to a closed
cluster together with just six of the other admissible stars -- Tau
Ceti, 82 Eridani, Zeta Tucanae, Alpha Mensae, and Zeta 1 and Zeta 2
Reticuli. The real configuration of interstellar distances is such that
an explorer starting from any of the seven should visit all of them
before venturing outside. If the Hill map is assumed to include the
sun, then it should include the other members of this cluster within an
unbroken network of connections, and the other connected stars should
be relatively adjacent in the real universe.

Zeta Reticuli occupies a central position in all of the relatively
few combinations that now remain plausible. However, in my opinion, the
adjacency criteria do leave some remnant ambiguity concerning the
combination of real stars to be matched against the Hill map -- but
only with respect to the region farthest from the sun. The stars in the
closed cluster and those in the chain leading to Gliese 67 must be
included, as well as Gliese 86 and two others from a set of five
candidates. Log2C for this remnant selection is 3.9 bits. we must also
notice that the constraint that Zeta Tucanae be occulted by Zeta
Reticuli reduces the dimensionality of the vantage-point space from 3.0
to 1.0. Thus, the sum of factors two and three is now estimated as only
4.9 bits.

The first factor is also awkward to evaluate -- simply because there
is no standard statistical technique for comparing points on two maps.
Using an approximation based on rank-order correlation, I've guessed
that the number we seek here is between 11 and 16. (This is the result
cited by Dickinson on page 15 of the original article.) Deducting the
second and third factors, this rough analysis leaves us with an
empirical result whose net meaning is equivalent to observing at least
6 to 11 consecutive heads. (I say "at least", because there are other
factors contributing to the total picture -- not discussed either by
Dickinson or by Soter and Sagan -- that could be adduced to enhance
this figure. For example, the computed vantage point is in good
agreement with Betty Hill's reported position when observing the map,
and the coordinate system implicit in the boundaries of the map is in
good agreement with a natural galactic coordinate system. Neither have
we discussed any quantitative use of the connections drawn on the Hill
map, which were put there in advance of any of these analyses.)

In the final interpretation, it will always be possible to argue that
5 or 10 or even 15 bits of remarkable information simply isn't enough.
However, this is a matter for each of us to decide independently. In
deciding this matter, it is more important that we be consistent with
ourselves (as we review a large number of uncertain interpretations of
data that we have made) than that we be in agreement with some external
authority. I do believe, though, that relatively few individuals will
continue a coin-tossing match in which their total experience is
equivalent to even six consecutive losses. In scientific matters, my
own standard is that I'm interested in any result that has five or more
bits of information supporting it -- though I prefer not to stick my
neck out publicly on the basis of less than 10. Adhering to this
standard, I continue to find the star map results exceedingly
interesting.

Dr. David R. Saunders is a Research Associate at the University of
Chicago's Industrial Relations Center.

Jerry
telnet://ricksbbs.synchro.net:23
http://ricksbbs.synchro.net:8080
---
þ Synchronet þ Rick's BBS telnet://ricksbbs.synchro.net:23