Homework 3 for Statistics 101-106 (Fall 98)

Due: Thursday 1 October

I received the following enquiry by email. How should I have responded? Is it possible to make assertions about the whole population (the constituency) on the basis of the information given?

Dear mavin of math,

I have a question regarding statistics which I am hoping someone in your
department could help me answer.  It has been some time since I had
statistics in college and to say that I am rusty is an understatement.
Before I ask the question you will need some background information.

I work for an organization that issues standards for use by public
entities.  We try to obtain input and feedback from our constituents to
ensure the standards meet their needs.  We recently put together a task
force of 30 people selected from among our constituents to help us
evaluate a proposed standard.  The 30 members of the task force were
judgmentally selected by my organization.  The emphasis in selecting
these task force members was on obtaining views from the many
organizations and groups that constitute our constituency.  Thus a
representative was selected from each organization and group.  We
selected multiple representatives from the larger of the organizations
and groups.

One member of our staff asked what use the task force would be in
helping us choose from among the many directions we could go with the
proposed standard.  I responded as follows, "If 28 out of the 30 members
agree on one option, that would be indicative to me that there may be a
similar trend in the population."  If there is a similar trend in the
population we could use the task force input to choose an option that
would best meet the needs of our constituency. 

In making the above statement I am not saying that we could extrapolate
the results to the population to say that 28 out of 30 members of the
population would also choose the same option.  The sample was not
randomly selected.  Not all organizations and groups that constitute our
constituency are equally represented on the task force based on
membership.  Further, although we know the size of the membership of
some of the organizations and groups that constitute our constituency,
we don't know the size of all of them.  However, together they compose a
sufficiently large number (over 5000) that the law of large numbers
applies to the population (our constituency).

What I am saying is that, if 28 out of 30 members choose the same
option, this must be significant and useful to our process.  If I
remember my statistics correctly, in order to say that something is
significant and useful one must also be able to say that confounding
factors do not account for the unanimity.  We are attempting to control
confounding factors at the task force meeting.  Thus in selecting the
task force members we attempted to find individuals who will represent
the views of the groups from which they are selected.  Further, all
options will be presented to the task force without bias.  The task
force is split into four small groups so that no one personality will
dominate and influence the choices of all others.  The moderators of
each sub-group will give each member equal time in the discussions.
Therefore, if 28 out of 30 task force members arrive at the same
conclusion, I believe it would be improbable that such unanimity would
be attributable to chance, some confounding factor, or anything other
than the merits of the options.  Thus, I believe such an outcome would
be significant no matter what applicable test for significance one used.

But what about useful?  I intuit that if 28 out of 30 task force members
reach the same conclusion it is not just significant, but it also has
predictive value.  Perhaps the question becomes, "just how
representative of the population is the sample?"  Since the sample was
not randomly selected I can't quantify what percentage of the population
will reach the same conclusions (plus or minus some range, with a given
degree of error).  Yet, for each member of the task force I believe
there is better than a 50/50 chance that they are representative of the
group they purport to represent.  If not, we have done a lousy job of
selecting the task force members.  We have, however, invested
significant time and effort in selecting the task force members to the
end that they would be representative of their various groups.  So, with
that background information, here is my question:  

Given a high degree of unanimity among the task force members (28 out of
30 in agreement) and further given that they are more likely than not to
represent (be representative of) their various groups, and assuming (or
not) that the various groups from which they were chosen are equal in
size, could I say with better than, say, 65% confidence that most
members of the population (our constituency) would reach the same
conclusion as the 28 reached?

That would be the ideal outcome for the task force; that they help us
select an option that most of our constituents would agree with.

If you or your colleagues don't have the time to answer this question(s)
would you please recommend some book(s) on the topic that I could slog
through without sinking over my head.  Your help is greatly appreciated.

[Name edited out]

Each Section will assign extra problems.