Q. How is the prediction accuracy
score calculated?
A. The primary accuracy score is simply the percentage of election
winners (offices and ballot measures) correctly predicted.
Ties are likely to be common, so ties will be split by a secondary
accuracy score, which is the average (mean) difference between
predicted and actual margins of victory (with lower scores indicating
greater accuracy).
Q. What should I do if I need to
close my web browser window before completing the survey? How can
I finish the survey at another time?
A. Before closing your browser window, make sure to copy the link or
URL for the current page of the survey. Save this link and return
to it when you are ready to complete the survey. If you weren't
able to save this link, please contact Devon Brewer by telephone at
206-985-2398
or by e-mail at
interscientific@yahoo.com
for instructions on how to continue.
Q. Can I complete the survey some other way
than online?
A. Yes, you can complete the survey by phone, fax, or e-mail if you
wish. These options might not allow you to respond anonymously,
although your responses and information will still be keep strictly
confidential (see questions about privacy, anonymity, and
confidentiality below). If you want to respond in one of these
other
ways, please contact Devon Brewer by telephone at 206-985-2398
or by e-mail at
interscientific@yahoo.com
to make the arrangements.
Q. What can I do if I want to update
my predictions after I complete the survey?
A. Once you complete the survey, you won't be able to re-enter
it. However, three weeks before the election, you will be invited
to participate in a very brief second survey. In this survey, you
can give your updated predictions. If you participate in the
second survey, your accuracy score for determining prizes will be based
on the survey in which your predictions are more accurate. In
other words, participating in the second survey gives you a chance to
improve your accuracy score.
Questions
about
privacy, anonymity, and confidentiality
Q. Are my responses and information
confidential?
A. All responses and other information you provide are kept strictly
confidential for the purposes of producing election forecasts only and
will not be shared with, sold to, or traded with any other organization
or individual (see the terms
and conditions for more information). Also, forecasts and
reports will based on aggregated responses only, and individual expert
respondents will never be identified, even indirectly.
Q. Can I participate
anonymously?
A. Yes, you can participate anonymously.
Q. If I participate anonymously, how
will I get my performance feedback and prize?
A. There are two ways you can receive feedback about your performance
and learn whether you won a prize. The first way is to give us an
e-mail address (for example, with Yahoo, Gmail, or Hotmail) for which
you have registered anonymously or with fictitious information.
Then we will send our correspondence to you at that e-mail
address.
Another way is to record the ID number we give you at the end of the
survey. You would then keep this number and return to our
election forecasting web site after the general election to query our
online database to receive your performance feedback and learn the
amount of your prize. Because the prizes are gift
certificates to online retailers, the only thing you need to redeem a
prize is the gift certificate number -- which you would find by
querying our online database with your ID number. If you choose
this option, however, you must remember to visit our site and enter the
correct ID number. Otherwise, we would have no way to contact
you. Please note if you choose to participate anonymously in this
way that you will not have an opportunity to improve your accuracy
score by completing the second brief survey.
Q. Do you track respondents'
IP addresses?
A. No, we do not track respondents' IP addresses. In fact, a few
days after sending our initial e-mail invitation message to experts, we
send another message to ask experts to complete the survey if they have
not already done so, because we have no way to know which respondents
have already completed the survey.
Q. To access the survey via the link
you sent me by e-mail, I also need to enter the unique one-time
password you sent. Can you connect that password to my e-mail
address and identity?
A. No, after we send experts the invitation to participate
by e-mail, we permanently delete from our records the password that had
been assigned to you. No connection remains between your password
and your e-mail address and identity. Furthermore, for added
protection, we delete all entered passwords from the survey data
file.
Q. How do respondents indicate their
informed consent to participate?
A. As noted on the first page of the online survey, continuing further
with the survey is how respondents indicate their informed
consent.
Q. What other terms and conditions
are there?
A. Click here
for other details on the terms and conditions of participation and
survey data.
Questions
about
respondents and sampling
Q. Are you surveying experts only?
A. Yes. Prior research indicates that non-experts tend to have
less accurate predictions than experts about election outcomes.
Q. How do you identify experts?
A. We invite to participate those individuals who have publicly
demonstrated experience in studying, reporting on, or participating in
statewide politics. Specifically, we recruit expert political
observers in academia, the media, the state legislature, and lobbyists
to state government.
Q. I would like to forward the
e-mail invitation to someone else so they can participate. Will
that work?
A. No, the link and unique password to access the survey that we sent
you will only work one time, and they are intended for your use
only. However, if there are types of people you think should be
surveyed, please let us know in the survey where you are asked for your
comments and suggestions. Depending on the advice we receive, we
can modify our selection of experts in the future.
Questions
about the
forecasting method
Q. How can you make accurate
forecasts if you don't have a random or probability sample of
respondents?
A. Our aim is not to sample voters to describe their opinions, but to
sample experts who have good knowledge of the election contests as well
as voters' opinions and voting tendencies. Knowledge of political
races and ballot measures, and of the electorate and its voting
patterns, is unevenly distributed in the community. Individuals
whose jobs involve, in part, assessing the political landscape tend to
be quite accurate in predicting election outcomes when their
predictions are aggregated. Collecting predictions from random
samples of voters in a state would needlessly increase the cost of
producing a forecast and reduce its accuracy.
Q. Aren't people's predictions of
election outcomes biased toward their own preferences? That is,
don't people's predictions reflect wishful thinking (also known as
desirability bias or the false consensus effect)?
A. There is a tendency for people to predict election outcomes
consistent with their own voting preferences. However, this bias
is less prominent in expert political observers. By including a
diverse sample of experts, who have different roles in observing
politics and different political orientations and values, we can
further minimize the influence of such biases, as they will cancel each
other. Also, we provide respondents monetary and other incentives
for accuracy. Moreover, with our analytic procedures, we can
detect when this bias is strong and adjust our forecast
accordingly.
Q. Why
don't you simply conduct a political poll instead?
A. There are two main reasons we don't conduct polls. First,
scientific polls, regardless of methodology, are much more expensive
than our forecasting approach. Second, our approach is as
accurate as polling carried out 7-8 weeks before an election.
Political
polls
provide snapshots of public opinion at varying points before an
election; our forecasts, however, are true projections of election
outcomes.
Q. Why don't you create an election
prediction market instead?
A. There are several reasons why prediction markets are not a good
option for forecasting most statewide election races and ballot
measures. First, prediction markets involving real money wagers
are, in most circumstances,
illegal in the U.S. Many knowledgeable experts might not
participate
in prediction markets without some sort of monetary incentive.
Second,
the prediction markets that do exist for statewide races (some governor
and US Senate races) are small and illiquid, meaning that they are
likely to be quite inaccurate and unattractive to most potential
bettors. States with smaller populations are particularly
unsuitable
for election prediction markets. Third, prediction markets'
forecasts
are effectively weighted by the size of bettors' wagers, not by their
knowledge or probable accuracy. Many different factors are
involved in
the size of bettors' wagers (such as amount available to wager,
aggressiveness, risk aversion, etc.). Fourth, bettors in
prediction
markets, like other markets, do not make their predictions
independently of each other, and are susceptible to a "herd" mentality,
diminishing the value of individual bettors' insights. Thus, even
if
prediction markets were a feasible alternative, it is likely that our
approach will still be more accurate.
Q. What techniques do you use
to produce election forecasts from experts' predictions?
A. Our proprietary analytic procedures are modified from standard
information aggregation techniques. In brief, our methods involve
synthesizing respondents' predictions into overall collective judgments
of predicted winners and margins of victory. To do so, we weight
respondents' predictions by individual respondents' estimated accuracy
(based on responses to a variety of questions in the survey and other
information). Our methods allow a statistical determination of
how reliable the forecasts are as representations of the aggregated
opinions of expert political observers in the state. Moreover,
with our procedures, we can assess different types of respondent bias
and make corresponding adjustments in forecasts, as necessary.
Q. Will
you report on the accuracy of your forecasts?
A. Yes. We have posted and distributed a report that includes a
full evaluation of the accuracy
of our forecasts for the 2008 general election in South Dakota, as
well as
a comparison with polls and other predictions.
Questions
about how the forecasts
will be used
Q. What will you do with my
predictions and other survey responses?
A. Your responses will be used to forecast election outcomes.
This information may be used in scientific research articles and
reports. Forecasts and reports will based on aggregated responses
only.
Q. When
will you make your forecasts available?
In 2008, all forecasts were made public for free 10 days before the
general
election.
Questions
about
Interdisciplinary Scientific Research
Q. What is Interdisciplinary
Scientific Research?
A. Interdisciplinary Scientific Research is a nonpartisan,
nonpolitical, scientific research firm.
Q. What is Interdisciplinary
Scientific Research's interest in forecasting elections in my state?
A. We intend to provide
forecasts for elections that have historically had few or no systematic
forecasts and to refine innovative election forecasting methods.
Q. What experience do you have in
maintaining privacy, confidentiality, and anonymity for research
participants?
A. We have conducted research on sensitive topics for federal agencies,
universities, and other research partners since 1988 without a
single breach of confidential information (see our Projects and Publications pages
for more information on our past work). Maintaining the
confidentiality of your information is our highest responsibility and
concern.
Q. I
would like you to make a forecast for an election.
Can you do it?
A. Please discuss your interests with us so we can determine
whether
our approach might be useful. Our approach works best for state
and local elections, particularly those with multiple races and/or
ballot measures. Contact Devon Brewer by telephone
at 206-985-2398 or
by e-mail at interscientific@yahoo.com.
Q. How much would it cost to
commission an election forecast?
The price for a comprehensive set of election forecasts, like those for
the 2008
general election in South Dakota is about 1/5th the cost of a
comparable interviewer-administered poll or about 1/3rd the cost of a
comparable automated (interactive voice response) poll. These
comparisons are based on the lowest price scientific polls of each type
in the industry.
Q. I have another question that
isn't listed here. How can I get an answer to my question?
A. Send us an e-mail message at interscientific@yahoo.com,
or call us
at 206-985-2398.
Copyright ©
2008 Interdisciplinary Scientific Research, LLC. All rights
reserved worldwide.
|
