Module 3

Formal fallacy - fallacy in form of argument

Affirming the consequent : if p, then q. q , therefore p
Denying the antecedent : if p, then q. Not p, therefore not q
^^ always invalid - only apply to deductive argument

Informal fallacy - fallacy in content of argument

Can occur in both deductive ad inductive arguments

Fallacies of Relevance: appeal to pity, appeal to the people, ad hominem abusive, ad hominem
circumstantial, tu quoque, fallacy of accident, straw man, & red herring.
Fallacies of Weak Induction: appeal to unqualified authority, hasty generalization, false cause,
slippery slope, & weak analogy.
Other Informal Fallacies: begging the question, false dichotomy, & equivocation.

Fallacies of relevance - when an argument contains one or more premises that are logically
irrelevant to the conclusion. Distinguish genuine evidence from what is emotionally or
psychologically persuasive but not logically relevant.
Appeal to pity - support conclusion by evoking pity “boss dont fire jerry for poor performance he
needs this job because xyz” - needs to be irrelevant to conclusion
Appeal to people - arguer attempts to support conclusion by exploiting desire to identify with or
be accepted, loved, and admired by others -- a number of different forms:
- Bandwagon fallacy - most people believe/use/do x therefore you should too
- Appeal to vanity - famous or admirable person p believes/does/uses x so you should too
- Appeal to snobbery - a select group of people believe/do/use x so you should too
Ad hominem fallacy - when person a’s argument relies on attacking person b directly rather than
their argument
- Ad hominem abusive - discrediting argment by attacking aspect of a persons character,
often characterized by verbal abuse
- Ad hominem circumstantial - attempting to discredit a person’s argument by pointing out
the circumstances of the arguer in an effort to expose biases
Tu quoque - attempting to discredit a person’s argument by pointing out some aspect of the
arguer’s life that suggests the arguer is hypocritical
Fallacy of accident - when a general rule is applied to a specific case it was not intended to
cover
Straw man fallacy - when one person misrepresents or distorts another person’s argument in
the process of attacking it
Red herring fallacy - when a person divers the attention of his or her audience by changing the
subject and then drawing a conclusion about the new subject

Fallacies of weak induction - when the connection between the premises and the conclusion is
not strong enough to support the conclusion
Appeal to unqualified authority - when the person presented as an authority lacks credibility
Hasty generalization fallacy - when there is a reasonable likelihood that the selected sample
that is drawn upon in order to draw a conclusion about a larger group is not representative of
the larger group
False cause fallacy - whenever the connection between the premises and the conclusion of the
argument depends on some causal connection that probably does not exist. 4 forms
- Post hoc ergo propter hoc - correlation does not imply causation
- Confusing the cause with the effect
- Oversimplifying the cause
- Gambler’s fallacy - assuming the previous wins have a causal effect on future wins
Slippery slope fallacy - when the conclusion of an argument rests on an alleged chain reaction
and there is not sufficient reason to think the chain reaction will actually take place. X will
inevitably lead to y
- A slippery slope argument becomes a slippery slope fallacy when it can be shown that
there are good reasons for objecting to y which do not apply to x
Weak analogy fallacy - when the analogy is not strong enough to support the conclusion that is
drawn

Other informal fallacies

Begging the question - when a person assumes what they are trying to prove
False dichotomy - when a statement erroneously limits the number of possible alternatives to
two, usually sharply opposed or extreme alternatives. Either or statements
Fallacy of equivocation - when the conclusion of an argument depends on the fact that a word
or phrase is used in two difference ways

Lesson 5
“To think incisively and to think for one’s self is very difficult. We are prone to let our mental life
become invaded by legions of half truths, prejudices, and propaganda… To save man from the
morass of propaganda, in my opinion, is one of the chief aims of education. Education must
enable one to sift and weigh evidence, to discern the true from the false, the real from the
unreal, and the facts from the fiction.” - Dr. Martin Luther King

Overview:

- Questions about the basis of knowledge prompt investigation into the conditions of knowledge,
including whether mere belief is sufficient for knowing.
- Justification is crucial in determining the validity of beliefs, but defining what constitutes
justification can be complex.
- The first part of the course focuses on epistemology, exploring issues such as justification and
belief.

Second Part: Scientific Reasoning:

- Introduction to the criteria of adequacy, a tool for evaluating competing theories beyond just
the "hard" sciences.
- Epistemological issues resurface in applying these criteria, leading to uncertainties despite the
quest for justified judgments.
- Foundational issues in scientific reasoning, including the logic of hypothesis testing, are
covered.
- Lewis Vaughn's explanation highlights the inductive nature of scientific reasoning and the
difference between confirming and disconfirming hypotheses.

Objectives:

- Understand the conditions of knowledge and contrast belief with justification.

- Grasp the concept of sufficient justification.
- Apply the TEST formula to assess scientific theories, including the criteria of adequacy.

Intro to Epistemology
Definition:
- Epistemology: Study of nature and scope of knowledge.
- Seeks to answer: What does it mean to know something? What do we know?

Developing a Conception of Knowledge:

- Initial claim: Knowledge = Belief.
- Flaw: Belief alone doesn't guarantee truth.
- Revised claim: Knowledge = (Belief + Truth).
- Problem: True belief may still lack justification.

Justification:
- Traditional account: Knowledge = (Belief + Truth + Justification).
- Issue: What constitutes justification?
- Some argue for certainty, but this limits knowledge severely.
- Descartes argued for certainty but ended with "I think, therefore I am."
- Certainty requirement leads to skepticism about reality.

Abandoning Certainty Requirement:

- Majority agrees to abandon certainty requirement.
- Importance of evidence and reasons.
- Justification comes in degrees; sufficient evidence is crucial.
- Not all evidence is sufficient; must be beyond reasonable doubt.

Challenges:
- Evaluating evidence constantly is crucial for rational thinking.
- Mistakes are inevitable due to cognitive limitations.
- Suspend judgment when evidence is insufficient.

Excessive Skepticism:
- Healthy skepticism: Analyze claims, subject to criticism.
- Excessive skepticism: Based on speculation, irrational doubt.
- Avoidance of belief can lead to irrationality; engagement with justification is necessary for
rationality.

Knowledge, Belief, & Evidence (Schick and Vaughn)

1. Importance of Knowledge:
- Knowledge empowers decision-making and problem-solving.
- It is valued for both practical applications and intrinsic satisfaction.
- The notion of "knowledge is power" underscores its significance in shaping outcomes.

2. Babylonian Knowledge Techniques:

- Babylonians employed elaborate methods like hepatoscopy (reading sheep livers) and
astrology for knowledge acquisition.
- Hepatoscopy was a revered technique, limited to use by kings and nobles for important
decisions.
- Astrology, though less messy, claimed causal links between celestial bodies and events.

3. Propositional Knowledge:
- Plato distinguishes between mere true beliefs and true beliefs grounded in reality, termed
propositional knowledge.
- True belief alone is insufficient for knowledge; justification through reasons is necessary.
- Plato emphasizes the importance of grounding beliefs in reality to qualify as knowledge.

4. Justification and Certainty:

- Certainty is not essential for knowledge, but evidence must put propositions beyond
reasonable doubt.
- The need for justification through evidence is highlighted to distinguish knowledge from mere
belief.
- Skepticism about requiring absolute certainty is addressed by emphasizing the role of
evidence in grounding beliefs.

5. Resolving Conflicting Beliefs:

- Conflicts among beliefs necessitate reassessment based on the credibility of new
information.
- The process involves evaluating the strength of evidence and adjusting beliefs accordingly.
- Proportioning belief to evidence is crucial for rational decision-making and avoiding errors.
6. Proportioning Belief to Evidence:
- Belief strength should correspond to the strength of evidence to minimize the risk of errors.
- The importance of aligning conviction with evidence is emphasized to maintain rationality.
- Overestimating belief without sufficient evidence can lead to unwarranted conclusions and
false beliefs.

Evaluating Scientific Theories: The Criteria of Adequacy

1. Introduction to Criteria of Adequacy:

- Criteria of adequacy provide guidelines for evaluating theories or possible explanations for
observed phenomena.
- They help in determining the credibility and reliability of theories by assessing their merits.

2. Testability:
- Testability refers to the ability of a theory to be subjected to empirical testing.
- A testable theory allows for the possibility of confirming or refuting its validity through
observation or experimentation.
- Untestable theories lack empirical support and are considered inadequate explanations.

3. Fruitfulness:
- Fruitfulness denotes the capacity of a theory to generate new insights and predict previously
unknown phenomena.
- A fruitful theory offers predictions that extend beyond existing knowledge, enhancing its
credibility.
- The ability to predict unexpected phenomena strengthens a theory's validity.

4. Scope:
- Scope refers to the extent of phenomena explained by a theory.
- A theory with broader scope explains a wider range of phenomena, increasing its
explanatory power.
- Comparing theories involves considering which theory accounts for more observations and
evidence.

5. Simplicity:
- Simplicity emphasizes the preference for theories with fewer assumptions or complexities.
- Simple theories, with minimal assumptions, are favored because they are less prone to error
and more elegant.
- Occam's Razor principle supports simplicity, suggesting that among competing hypotheses,
the simplest one is usually correct.

6. Conservatism:
- Conservatism involves the compatibility of a theory with established beliefs or knowledge.
 - A theory that aligns with existing justified beliefs is considered more credible.
- Skepticism towards theories conflicting with well-supported knowledge indicates rational
evaluation.

7. Application of Criteria:
- Evaluating theories involves weighing each criterion based on the specific context and
available evidence.
- There is no rigid formula for applying the criteria, and assessments require holistic
consideration.
- While some judgments may be clear-cut, others may involve subjective interpretation or
uncertainty.

8. Conclusion:
- Criteria of adequacy serve as essential tools for assessing the validity and reliability of
theories.
- Rational evaluation involves considering testability, fruitfulness, scope, simplicity, and
conservatism.
- Ultimately, theories meeting these criteria are deemed worthy of belief, but certainty is not
guaranteed, and assessments may evolve with new information.

Evaluating Scientific Theories: The TEST Formula

TEST formula.

Step 1: State the Theory and check for consistency.

Step 2: Assess the Evidence for the theory.

Step 3: Scrutinize alternative theories.

Step 4: Test the theories with the criteria of adequacy.

Step 1: State the Theory and Check for Consistency

- Internal Consistency: Ensures absence of contradictions within the theory.

- Example: Galileo disproved Aristotle's theory of motion due to internal inconsistency as it
implied a heavy object fell both faster and slower than a lighter one
- Schrödinger's cat experiment highlighted inconsistencies in quantum mechanics
interpretations.

- External Consistency: Aligns with observed data or phenomena.

- Example: Theory explaining all observed facts (e.g., Earth's shape) is externally consistent.
 - Theories failing to explain observed data are deemed externally inconsistent.

- Eligibility Criteria: Theories meeting both internal and external consistency are eligible.
- Vaughn suggests the best theory meets adequacy criteria better than competitors.

Step 2: Assess the Evidence for the Theory

- Importance: Evaluates empirical evidence supporting the theory.

- Lack of thorough assessment by the general public is problematic.
- Importance of scientific consensus based on qualified authority.

Step 3: Scrutinize the Alternative Theories

- Comprehensive Examination: Involves comparing competing theories to avoid confirmation

bias.
- Importance of keeping an open mind and reconsidering alternative explanations.
- Reapplication of Steps 1 and 2 to avoid errors and ensure objectivity.

Step 4: Test the Theories with the Criteria of Adequacy

- Purpose of Criteria: Provides standards for evaluating theories' plausibility.

- Differentiates between evidence assessment (Step 2) and theory evaluation (Step 4).
- Ensures fair comparison and elimination of inadequate theories.

Additional Considerations and Reflections

- Critical Reading Suggestions: Focus on distinguishing science from technology, ideology, and
scientism.
- Scientific Method: Understand the significance of each step, particularly Step 3.
- Criteria of Adequacy in Historical Context: Influence on scientific paradigm shifts (e.g.,
planetary models, evolution).
- Key Questions: Consider aspects of adequacy criteria driving scientific decisions and theory
rejections.

Judging Scientific Theories (Vaughn)

**Summary:**

- **Science and Explanations:**

- The world is filled with explanations or theories about why things happen.
- Science seeks knowledge and understanding through systematic testing and evaluation of
theories.
- Critical thinking involves evaluating theories using criteria of adequacy.
- **Science vs. Technology:**
- Science is a search for truth using the scientific method, while technology produces products.
- Technology applies knowledge from science to practical problems.
- Science seeks general principles to explain and predict phenomena.

- **Science vs. Ideology:**

- Science is not a worldview but a method of inquiry.
- Different worldviews may prevail in science, but the essence of scientific inquiry remains
constant.
- The mechanistic view of the universe has evolved over time with scientific discoveries.

- **Science vs. Scientism:**

- Scientism claims science as the only reliable path to truth.
- While science is a highly reliable method for understanding the empirical world, it's not the
sole path to knowledge.

- **The Scientific Method:**

- Involves several steps: identifying a problem, devising a hypothesis, deriving test
implications, performing tests, and accepting or rejecting the hypothesis.
- Hypotheses are formulated guided by criteria such as testability, fruitfulness, scope,
simplicity, and conservatism.
- Testing involves deriving implications from hypotheses and conducting experiments or
observations.

- **Testing Scientific Theories:**

- Scientists test hypotheses through controlled experiments or observations.
- Experiments should be well-designed, double-blind, and statistically analyzed.
- Results are evaluated, and hypotheses may be accepted, rejected, or modified based on
evidence and judgment.

- **Judging Scientific Theories:**

- The evaluation of theories involves considering criteria like testability, fruitfulness, scope,
simplicity, and conservatism.
- Historical examples like Copernicus vs. Ptolemy illustrate how theories are assessed based
on these criteria.

Evolution Versus Creationism:

- **Debate Overview**:
- Evolution and creationism vie for explaining the origin of biological life.
- Despite religious elements, science can adjudicate this debate using criteria of adequacy.

- **Evolution's Origin**:
 - Evolutionary concepts predate Darwin, with roots in ancient philosophy.
- Darwin's "On the Origin of Species" (1859) solidified modern evolutionary theory.

- **Evolution's Basics**:
- Organisms adapt via inherited traits leading to changes over generations.
- Natural selection, Darwin's concept, explains this process.

- **Creationism's Tenets**:
- Belief in abrupt creation, rejecting evolution's gradualism.
- Contradicts scientific evidence on Earth's age, fossil records, and geological events.

- **Testability**:
- Both theories are testable; evolution consistently confirmed, creationism refuted.
- Evolution predicts diverse phenomena across biology, validated by evidence.
- Creationism fails the test, contradicts established scientific knowledge.

- **Predictive Power**:
- Evolution predicts ongoing speciation, fossil record patterns, and genetic similarities.
- Creationism lacks predictive ability and struggles to explain various natural phenomena.

- **Simplicity and Scope**:

- Evolution maintains simplicity, explaining vast biological phenomena coherently.
- Creationism's complexity and limited explanatory power diminish its scientific merit.

- **Scientific Judgment**:
- Based on criteria of adequacy, evolution emerges as the superior theory.
- Scientists justify rejecting creationism in favor of evolution due to its consistency with
evidence and explanatory power.

Lesson 6
- Hostile debates on scientific claims leading to extreme viewpoints
- Some overly trusting of any scientific claim, labeling dissenters as "anti-science"
- Others deeply distrustful of science, due to ignorance or belief in political/economic
corruption
- Need to understand reasons for poor scientific outcomes and identify indicators of reliable
findings
- Directly engaging with scientific research advocated to mitigate misinformation and reach
genuine understanding

Objectives
Participants who complete this module will be able to:
Distinguish science from scientism.
Explain the nature of the replication crisis.
Identify the main reasons scientific studies cannot be replicated.

John Oliver Video

- P-hacking - playing with lots of variables and find statistical correlations about random
things
- Raw tomatoes and Judaism, ice tea and opinions on the movie “Crash”, cabbage
and belly button type
- Replication studies are rarely funded so nothing is concrete
- No Nobel prize for fact-checking
- Studies are twisted
- Study showing preeclampsia is not affected by chocolate reported by news
stories as good to eat 30g of chocolate a day
- Like playing telephone
- Times reporting on farts curing cancer - the study linked didn't mention either
- A study on rats and champagne reported as humans drinking champagne 1-3
times weekly prevents Alzheimer's
- Dehydrated drivers just as bad as drunk drivers study based on only 12 men and
funded by the European Hydration Institute and Coca-Cola
- Misreporting hurts our faith in science
- The View saying find the study that sounds best to you
- Companies use a lack of hard, confirmed science to take advantage
- Tobacco companies
- TODD talks where the TED format meets the intellectual rigor of morning shows

Scientism Revisited
- Science, while a valuable source of knowledge, is not immune to error or misuse
- Deliberate abuses and human error are inherent risks in scientific endeavors
- Despite its reliability, science can produce flawed studies and erroneous claims
- Critically assessing scientific claims is crucial
- Scientism, the excessive belief in the power of scientific knowledge, is a concern
- Uncritically accepting scientific claims is a form of scientism
- Skepticism should be applied to all claims, including scientific ones
- Comparing skepticism towards scientific claims to skepticism towards other types of claims
- Evaluation of scientific studies parallels evaluation of arguments
- Assessing the methodology and inferences drawn from data is essential
- Accepting scientific findings requires scrutiny of both methodology and logical inference
from data
Scientific Studies: The Replication Crisis
- Replication crisis in scientific research highlights the need for skepticism towards scientific
claims
- Studies in various disciplines, including psychology and cancer biology, have low replication
rates
- Factors contributing to irreproducibility include:
1. Deliberate manipulation of data (scientific misconduct)
2. Misuse of data (p-hacking)
3. Errors in methodology and analysis
4. Pressures of "publish or perish" culture leading to flawed research behavior
5. Biases affecting interpretation of data and conclusions
- Challenges in addressing bad science:
1. Cost and lack of incentives for replication
2. Flaws in peer-review system
3. Difficulty for non-scientists in evaluating studies
- Considerations for assessing scientific studies:
1. Reputation of the publishing venue
2. Funding sources and potential conflicts of interest
3. Replication of findings by independent teams
4. Importance of multiple studies supporting a claim
- Goal is not to reject all scientific claims, but to avoid undue reliance on unjustified or unreliable
ones, which can lead to harmful consequences

Vox Science is often wrong

https://www.vox.com/2015/5/13/8591837/how-science-is-broken

- Outright fraud, exemplified by Diederik Stapel's case, is just one way science can go wrong
- Stapel manipulated data systematically to produce desired results
- Many structural flaws contribute to science's imperfections
- Peer review inadequacies, publication bias, and irreproducibility issues are prevalent
- Challenges in study design and dissemination of research exacerbate these problems
- Overhyped press releases and broken hyperlinks hinder access to reliable information
- Addressing these flaws requires a shift in how we perceive and conduct science
- Embracing open-data initiatives, post-publication peer review, and reproducibility projects
- Sheila Jasanoff highlights science as a human construct prone to flaws, cautioning against
idealistic views
- Recognizing science's imperfections can foster a healthier appreciation and trust in its
methods, leading to improvements in research quality and public understanding
The Economist - How Science Goes Wrong
- Science relies on "trust, but verify" principle for robust results
- However, modern science faces challenges due to shoddy experiments and poor analysis
- Biotechnology VC's estimate half of published research cannot be replicated
- Research retraction due to errors or improprieties wastes time and resources
- Competition in academia undermines quality control and self-policing
- "Publish or perish" culture prioritizes quantity over quality
- Verification of results does little to advance researchers' careers
- Careerism leads to exaggeration, cherry-picking results, and exclusion of negative findings
- Leading journals' high rejection rates favor striking findings
- Failure to report negative results wastes resources and inhibits scientific progress
- Peer review is flawed, often failing to detect deliberate errors
- Solutions include improving statistical literacy, registering research protocols in advance, and
promoting open data
- Journals and funders should allocate space and funding for less sensational work
- Peer review may be replaced by post-publication evaluation with appended comments
- Policymakers should enforce rules for institutions using public funds
- Science's credibility rests on its ability to correct mistakes and pursue genuine mysteries

Wakefield Fraud 1
- The MMR vaccine-autism study, led by Andrew Wakefield, was published two decades ago
and was later debunked.
- Wakefield's study, involving only 12 children, suggested a link between the MMR vaccine and
autism, but it was based on dubious science.
- Wakefield manipulated data, had financial conflicts of interest, and failed to replicate his
findings.
- Subsequent large-scale studies have repeatedly debunked the vaccine-autism link.
- The media played a significant role in amplifying the study's influence, focusing on
sensationalism rather than scientific evidence.
- Journalists often report on single studies without considering methodological flaws or the
broader context of scientific progress.
- Vaccines have been crucial in reducing deaths and diseases worldwide, and progress against
vaccine-preventable diseases continues.
- To combat the spread of dubious science, efforts should include educating young people in
critical thinking skills and promoting rigorous scientific reporting.
Wakefield Fraud 2
- The 1998 Lancet paper suggesting a link between the MMR vaccine and autism has been
thoroughly discredited.
- Investigative journalist Brian Deer uncovered clear evidence of fraud committed by Andrew
Wakefield, the lead author of the paper.
- Wakefield manipulated data, altered facts about patients' medical histories, and had financial
conflicts of interest.
- Wakefield's coauthors were largely unaware of his misconduct, but they failed in their duties as
authors to ensure the integrity of the research.
- Wakefield's fraudulent study has had lasting damaging effects on public health, leading to
decreased vaccination rates and measles outbreaks.
- The research ethics committees, the Royal Free Hospital, and the Lancet also bear
responsibility for failing to prevent or address the fraud.
- Wakefield's other publications should also be scrutinized for potential misconduct, and further
investigations are warranted.
- The retraction of the Lancet paper should be expanded to acknowledge the fraudulent nature
of the study and close the door on any lingering doubts about its validity.

Lesson 7
- Overview:
- Ethics:
- Key issue: Moral truth.
- James Rachels emphasizes ethics relies on rational proofs.
- Many conditioned to believe ethics are subjective.
- Learn distinction between subjective and objective matters.
- Understand relation to truth.
- Recognize moral argument and naturalistic fallacy.
- Legal Ethics:
- Focus on three matters:
1. Standard of proof in criminal vs. civil cases.
2. Foundational role of analogical reasoning.
3. Distinction between ex ante and ex post reasoning.
- Learn how standards explain legal outcomes.
- Understand analogical reasoning.
- Use ex ante/ex post reasoning in legal contexts and beyond.

- Objectives:
- Identify value claims.
- Distinguish moral vs. non-moral value claims.
- Differentiate descriptive vs. moral statements.
 - Recognize a moral argument.
- Explain the Naturalistic Fallacy.
- Understand subjective-objective distinction.
- Contrast legal standard of proof in criminal vs. civil cases.
- Differentiate ex ante reasoning from ex post reasoning.

Ethics and Science

- One line of thought compares belief in "objective values" to belief in ghosts or witches,
questioning why science hasn't discovered them. David Hume argued that examining wicked
actions reveals no real existence corresponding to wickedness; it's subjective. John L. Mackie
stated that values aren't part of the fabric of the world.

- However, value isn't tangible like a planet; science won't discover it. This doesn't mean ethics
lacks an objective basis. A third possibility arises: moral truths are matters of reason, true if
backed by better reasons than alternatives. This suggests objectivity in moral truths,
independent of personal beliefs.

- Another perspective likens ethics to science, finding ethics lacking in proofs. Science offers
proofs, but ethics seems devoid. While moral judgments may seem unprovable, so do some
scientific matters. Complex issues in science also lack clear proofs, yet science progresses.

- Simplified moral matters can be proven. For instance, a student's claim of an unfair test can be
supported with evidence. Similarly, judgments about individuals' morality can be backed by
reasons, such as habitual lying or irresponsible behavior.

- Further justification can be provided by explaining why certain actions are morally wrong, like
lying, which harms trust and society. Constructing sound proofs in ethics involves giving
reasons, analyzing arguments, and justifying principles, different from scientific experimentation
but not deficient.

- Despite frustrations in proving ethical opinions, distinguishing between proving an opinion

correct and persuading someone of it is crucial. Philosophers focus on constructing sound
proofs, leaving persuasion to others. An argument may be valid even if it fails to persuade due
to biases or lack of receptiveness.

Moral Reasoning: Moral Statements & Arguments

- Morality Overview:
 - Concerned with how we should behave towards one another.
- Central idea: Avoid treating others in objectionable ways.
- Morality differs from social and legal norms.
- Historical examples show morally objectionable treatment once legally or culturally accepted.
- Morality is fundamental; societal and legal acceptance should align with moral norms.

- Moral Principles:
- Develop as guidelines for moral behavior.
- Actions are deemed morally right/wrong based on these principles.
- Morally wrong actions violate established moral principles, aiming to prevent harm to others.

- Value Claims:
- Value claims assert something as good, bad, right, wrong, better, or worse.
- Examples: Movie preferences, moral judgments.
- Moral value claims differ from non-moral ones by focusing on treatment of others.
- Not every value claim is a moral judgment; moral judgments involve treatment of beings with
interests.

- Moral Statements:
- Interchangeable with moral value claims.
- Assert actions as morally right or wrong.
- Distinguish from descriptive statements, which describe facts without moral judgment.

- Moral Arguments:
- Differ from non-moral arguments by having moral statements as conclusions.
- Can include non-moral premises.
- Example: Argument for voluntary active euthanasia.
- Avoid the naturalistic fallacy: don't infer moral conclusions from purely descriptive premises.

- The Naturalistic Fallacy:

- Attempting to derive a moral conclusion from solely descriptive premises.
- Fails due to the "Is-Ought distinction."
- A good moral argument requires at least one moral statement as a premise.

- Assessing Moral Arguments:

- Not just about validity or strength; premises must also be true.
- Moral premises evaluated based on moral principles, consistency with other moral
judgments, and avoidance of undesirable consequences.
- Requires extensive moral knowledge, including understanding of moral theories and
objections.
Moral Reasoning: Proof in Ethics
- Subjectivity vs. Objectivity in Moral Reasoning:
- Misconception: Some view morality as purely subjective, lacking objective truths.
- Importance of subjective-objective distinction: Understanding key to recognizing proofs in
ethics.
- Definition of "proof" in moral context: Not absolute certainty, but sufficient evidence beyond
reasonable doubt.

- Subjective Matters:
- Truth dependent on personal feelings, desires, preferences, values, etc.
- Examples: Enjoyment of music, color preferences, taste preferences, career choices,
recreational activities.
- Truth relative to individual perspectives, leading to varied judgments.

- Objective Matters:
- Truth independent of personal feelings, desires, preferences, values, etc.
- Examples: Mathematical principles, scientific laws, social science findings.
- Determined by axioms, observation, experimentation, data-driven studies, and reason.
- Standards of justification vary across disciplines due to differing modes of inquiry.

- Ethical Matters:
- Dispute over objectivity in ethics: Some assert subjectivity, challenging resolution of moral
disputes.
- Ethicists reject subjectivity, uphold rationality as primary tool for moral reasoning.
- Reason used to establish moral rules and address contentious issues.
- Objective truth in ethics not contingent on personal feelings, desires, or preferences.

- Proofs in Ethics:
- Distinct from scientific proofs but equally valid.
- Relies heavily on reason, similar to other disciplines in philosophy.
- Challenges arise from complex ethical issues, requiring ongoing rational discourse for
resolution.

- Objective Truth and Reasonable Disagreement:

- Objective truth in ethics doesn't guarantee immediate resolution.
 - Reasonable disagreement compatible with objective moral judgments.
- Rational discourse essential for advancing understanding of ethical issues.

- Cautionary Note on Non-Moral Value Claims:

- Not all non-moral value claims are subjective.
- Example: Assessment of job applicants based on qualifications.
- Objective criteria and rational assessment distinguish objective non-moral value claims from
subjective ones.

- Rhetorical Device and Endnotes:

- Beware of using subjectivity claim to divert discussion.
- Rejecting rational proofs in ethics implies rejection of proofs in science and mathematics.

Standards of Proof (Farnsworth)

by Ward Farnsworth

Excerpted from The Legal Analyst (The University of Chicago Press, 2007)

Chapter 27: Presumptions

Courts usually don’t declare things true or false. That’s too difficult. Usually they just try to
estimate likelihoods; more precisely, they decide whether the evidence that a claim is true gets
over a high enough bar to justify a decision—whether the decision is a criminal conviction, a
verdict in a civil lawsuit, an opinion striking down a statute or regulation, or the reversal of a trial
court’s ruling on appeal.

This way of thinking helps explain the law’s treatment of O. J. Simpson. He was prosecuted for
murdering his ex-wife and a friend of hers, and he was acquitted. Then he was sued for
damages by the families of the murder victims—and they won. How can that be? Is there not
some contradiction between the first court’s declaring him innocent and the second court’s
declaring him guilty? Not really, no; for that isn’t quite what either of the courts really said.
Simpson wasn’t “found innocent” in the criminal court. His victory in that case was just a
declaration that the evidence against him didn’t satisfy the standard of proof used in criminal
cases: proof beyond a reasonable doubt. Nor was he “found guilty” in the civil suit. The court
only concluded that the evidence against him got over the lower bar used in civil lawsuits in the
United States: proof by a preponderance of the evidence. We will talk more about those different
standards of proof in the next chapter, but you already can see the point. If the legal system
concluded that Simpson was, say, 70 percent likely to have committed the murders, then the
results in both cases were right. The evidence was enough to support a judgment that took
away his wealth but not enough to support a judgment that took away his liberty. More generally
the two cases illustrate our initial claim. The courts don’t decide whether a defendant did a
killing or didn’t do it; they aren’t in the business of declaring certainties. They decide whether the
evidence in a case gets over the bar. Sometimes the evidence gets over one bar but not
another…

Chapter 27: Standards of Proof

It’s rare for people making decisions to be perfectly sure they are right, and this is as true in a
courthouse as elsewhere. But how much confidence should we require in a decision? We find a
lively set of studies in the problem in decisions courts make of the what if variety. A sailor falls
over the side of a ship and is dragged to his death by a squid some moments later. He might
have been saved if he had been thrown a rope, but the captain had mistakenly left all the rescue
gear behind on shore. In a lawsuit resulting from this sort of case, it isn’t enough for the sailor’s
kin to say it was blameworthy in the captain to have gone to sea ill-equipped (let’s assume that it
was). It also has to be shown that the rope would have made a difference—that it would have
saved the sailor. Maybe it wouldn’t have; maybe the squid was too quick. The normal thing
would be to put together a jury and ask them to decide the question. But now suppose the jurors
aren’t entirely sure. They think a rope could well have saved him, but perhaps not. How sure
must they be before they find against the captain?

It might seem a natural solution to have the jury assign a figure to its best guess and then have
the captain pay over to the plaintiffs a share of money that corresponds to it. So suppose the
sailor’s death deprived his kin of $100,000 in earnings they otherwise had coming; if the jury
thinks there is a 30 percent chance that a rope would have saved him, the captain would be
obliged to pay the kin 30 percent of the earnings: $30,000. If the rope was 80 percent likely to
have made a difference, then $80,000 would be due from the captain. The award would
increase with our certainty of its justice.

Yet usually we do nothing of the kind. We tell the jury instead to answer a yes-or-no question:
whether the plaintiff has shown by a preponderance of the evidence that a rope would have
saved the sailor. Most courts regard this as equivalent to asking if the chance the rope mattered
was better than 50 percent. If it wasn’t—if the jury’s best guess is that the rope was, say, 30
percent likely to have mattered—the plaintiff hasn’t met the standard of proof and the captain
pays nothing. But if there’s a 60 percent chance that the sailor would have been rescued, the
captain is held wholly responsible and pays the entire $100,000. The captain pays everything or
nothing, though whether the rope would have helped may forever be a question to which our
answer lies somewhere between those poles, and maybe near the middle.

Why this discrepancy; why does the law turn such modest confidence one way or the other into
all-or-nothing results? The most likely reasons are practical. It might be hard for the parties to
supply any decider—whether jury or judge—with the knowledge they would need to make more
precise estimates. The trier of the facts would end up making guesses based on trivial
foundations; the apparent precision of a finding that the rope was 70 percent likely to have
saved the sailor from the squid would be an illusion. The judge or jury is more likely to be equal
to the task if called on just to sort the case onto one side or another of the “more likely than not”
threshold. Most of the time this allows the jury merely to consider the stories told by the two
sides and compare their plausibility. The uncertainties about whether the rope was 20 percent or
30 percent or 45 percent likely to matter can be left unresolved, for they all mean the same thing
in the end; likewise for distinctions between 60 percent and 80 percent. The general idea, which
comes up a lot in law, is that sometimes a bad question that a court can answer relatively well is
preferable to a better question that is more likely to be answered badly…

In criminal cases the standard of proof is highest of all: the familiar proof beyond a reasonable
doubt. The reason, evidently enough, is that mistakenly convicting someone is thought worse
than mistakenly letting him go. Thus Blackstone’s claim that it’s “better that ten guilty persons
escape, than that one innocent suffer”—a fine example of comparing false positives and false
negatives, though the right numbers that ought to be plugged into the maxim are open to
debate. But the high standard of proof also does something else. It helps offset other
advantages that prosecutors enjoy against the defendant. The state has extensive powers to
spy, search, and subpoena; it has the police and the warden; it has ambitious prosecutors with
lots of resources. The defendant normally has little money, so he is supplied with other armor:
the right to a free lawyer, a right to appeal, the right to have evidence suppressed if it was
obtained wrongfully—and a presumption of innocence until it is overcome beyond a reasonable
doubt. Try thinking of these points as packages that might be balanced against one another,
with lots of edges where marginal adjustments can be made. When courts worry that the
balance goes too far in favor of the state, they expand the rights of defendants; when
legislatures think the balance tips too far toward the criminal, they make inroads along other
margins, as we saw in chapter 3—cutting back on appeals, lengthening sentences, hiring police,
and otherwise adding weights in the state’s pan. The standard of proof may thus be viewed as
one part in an ongoing contest of inequalities. The substance of this point is more familiar than it
may sound. We saw earlier in this chapter that civil standards of proof make up one piece of a
system that regulates the flow of lawsuits and the incentives that result from them. The same
goes for the standard on the criminal side, though this time the goals are different. The point for
now is just to see again that standards of proof do more than set the bar for a jury’s decision in a
case. They have side effects.

The most interesting question about reasonable doubt is what it means to speak of it. Earlier we
spoke interchangeably of a preponderance of the evidence, a more-likely-than-not standard,
and confidence better than 50 percent, which most courts say all mean the same thing.
Reasonable doubt has not lent itself so well to such restatement. Some say the phrase has a
meaning too self-evident to bear elaboration, but this is bluff; it does not survive the ample
evidence that people—judges as well as laymen—have quite different ideas in mind when they
speak and hear of reasonable doubt. The mighty Learned Hand didn’t think much of the
distinction between a preponderance of the evidence and reasonable doubt; he declined “to
distinguish between the evidence which should satisfy reasonable men, and the evidence which
should satisfy reasonable men beyond a reasonable doubt. While at times it may be practicable
to deal with these as separate without unreal refinements, in the long run the line between them
is too thin for day to day use.” This sounds startling to the modern ear. Is there not a massive
difference between 51 percent confidence and, say, 90 percent confidence? Yet Judge Hand
never should be dismissed lightly; perhaps the difference between the rules is indeed
exaggerated. A juror’s experience of believing something is more likely than not may be similar
to thinking it true beyond reasonable doubt: as to any point offered in court, either you think it’s
true or you don’t. Still, empirical studies do show that jurors (or at least mock jurors) are more
likely to convict under lower standards of proof, and later courts have all rejected Hand’s
view—but have left the difference between the standards imprecise. A survey was made of 171
federal judges, asking them to assign percentages to the certainty they associate with proof
beyond reasonable doubt. Twenty-one said 100 percent, 25 said 80 percent or less, and the rest
were scattered over various points in between, with the largest number—56—answering 90
percent.

The range of numbers that judges attach to reasonable doubt may seem to cry out for a formal
settlement of the matter: what is the right number? Nobody will say. A few years ago a judge in
Nevada told a jury that reasonable doubt was “about seven and a half, if you had to put it on a
scale” of zero to ten. The defendant’s conviction was swiftly reversed, both because the number
seemed wrong and because it was held wrong to speak in numbers at all. No better, and
reversed with as much dispatch, was the Connecticut judge who told the jury that “[y]ou don’t
have to go a hundred yards for a guilty finding. You got to go somewhere, I suppose, beyond the
fifty yard line; where it is in there is up to you to decide.” There are appellate judges who have
said in their written opinions that confidence beyond a reasonable doubt means “0.9 or so, with
adjustments depending on the gravity of the offense,” but there are no known cases of any court
approving such a statement to a jury. What we want behind a conviction, it is said, is a felt sense
of great confidence on the jury’s part, not a belief that the probability of guilt has passed a high
numerical threshold.

Might this give up a little too fast on the prospect of thinking in numbers? We already ask jurors
to estimate various probabilities, especially in civil cases. They have to decide the chance that
some wrong by the defendant caused an injury; or they have to figure the plaintiff’s damages,
which involves estimates of future wages and life expectancies—all matters of odds. People
elsewhere lay down numerical gambles all the time when they make decisions about
investments, formal or informal. The trouble is that inviting jurors to bet on the defendant’s
likelihood of guilt would be unappealing. To continue the analogy, perhaps the trouble is that
they’re playing with house money; they are deciding whether the terrible coercive power of the
government should be unleashed on someone else. We can move away from all thoughts of
betting on guilt or innocence, but it’s surprisingly hard to come up with other ways of expressing
the issue in numbers. For example, you wouldn’t want to think in frequencies—to say an 80
percent likelihood of guilt means that if the facts were repeated 100 times, the defendant would
be guilty in 80 of them. The real facts of criminal cases don’t usually get repeated more than
once, and asking jurors to imagine 100 of them invites errors of various sorts. One is a kind of
hindsight bias, which we discussed in chapter 23: now that the jurors are presented with an
outcome and a particular defendant, they might imagine him guilty more often than really would
occur in a large number of rehearsals. It’s all hard to say, precisely because the whole exercise
is hypothetical. Indeed, in most cases—all those that don’t hang on scientific evidence—a
number is likely to be just a metaphor for a feeling. To say you are 90 percent sure of something
usually reports a mental sensation of certainty rather than a real estimate of odds.

Here is a final conundrum. Suppose a defendant has been arrested for trying to buy cocaine. He
had found a dealer and offered to trade a gun for a couple of ounces of the drug; the dealer
accepted, then announced that he was an undercover FBI agent and led the defendant off to
jail; now he is being prosecuted under a statute that gives many years of prison to anyone who
“uses a firearm” in relation to a drug trafficking offense. This raises a nice question: do you use
a gun if you try to barter it for drugs, or do you only “use” a gun under the statute by putting it to
work as a weapon? There are arguments to make either way; assume it’s close but that the
judge thinks the statute does cover the case. Well, but wait—how sure must the judge be? We
know that in criminal cases the law throws almost the entire risk of error onto the government,
so intolerable is the thought of convicting an innocent; that is what it means to require that the
government prove guilt beyond a reasonable doubt or else lose the case. Presumably the same
goes here. The defendant should get the long, hard sentence only if the judge is sure beyond a
reasonable doubt that “use” in the statute covers the trade of a gun as well as the wielding of it.
Except that isn’t what judges do; they’re not required to be convinced beyond a reasonable
doubt before they say what the law is. Indeed, they don’t confess to using any standard of proof
at all when they interpret the law. They just say what they think the law means and that’s the
end of the case. If there are a few possible meanings of a statute, and the best of them seems
just 40 percent likely to be right, that is the one the judges will adopt, however brutal the result
may be for the defendant.

At first it seems puzzling that the legal system would be so protective of defendants when juries
find facts against them but so indifferent when judges find law against them. To grasp why the
standard of proof for decisions of law is so weak, pause and imagine how the world would work
if it were otherwise (this is frequently a sound strategy for making sense of strange rules). Let’s
go back to an example from early in this book. I spend all afternoon chasing a buffalo and at last
am closing in on the animal as it starts to tire. At the last minute you emerge from behind a rock
and deliver the lethal blow. I say it belongs to me. You say it belongs to you. I grab it and run;
you follow and forcibly take the animal back from me. Alas, in our jurisdiction the rule governing
this situation isn’t written as clearly as we would like; its meaning certainly isn’t clear beyond a
reasonable doubt. So it follows (on our current assumption) that the law has nothing to say. I
can’t be prosecuted for theft because my guilt isn’t clear beyond a reasonable doubt. (I confess
to all the facts, but the law isn’t clear enough.) But this is good news for you as well, for it also
follows that you weren’t committing a theft when you took the animal back. Any force you used
might itself have been unlawful—unless, of course, it was just the force needed to stop me from
stealing your animal, as you claim—a point on which the law (to repeat) is too unclear for a
court to enforce. We’re well on our way to a blood feud.
A lot of law consists of rules like ones saying who gets the buffalo: law settles disputes; it
answers questions; it tells where our rights start and end. But those settlements and answers
often aren’t written well or with any great foresight; when invoked they may be found not to have
meaning, or application to the facts, that can be called clear beyond a reasonable doubt or even
by a preponderance of the evidence. To thus conclude that those laws don’t count for anything
would leave large tracts of conduct outside law altogether and regulated only by self-help. That
undoubtedly is one way to run the world, but it fast can turn ugly—with results that explain the
lack of popularity that anarchy historically has enjoyed.

It would be too much to say that claims of law never are subject to standards of proof.
Sometimes courts create so-called clear statement principles holding that unless statutes say so
clearly, they will not be read to create some given result—retroactive application, for instance, or
preemption of state authority, or the raising of hard constitutional questions. These principles
serve various purposes; they may force lawmakers to deliberate and be clear where those
things are most wanted, or they can give courts a way not to deal with thorny problems unless
they absolutely must. Anyhow, notice that any such rules require judges to say in a rough way
not only what a law seems to mean but how likely that reading is to be right. It shows that courts
do know how to apply standards of proof not only to questions of fact but to questions of
law—when it suits them. At present the confidence of judges in their legal decisions almost
always goes unremarked. They typically just pick whatever reading they think is the best
available and then announce it with a strong but vague show of certitude. One could argue that
judges should pay more attention to the point; for isn’t there something bothersome about a
defendant going to prison under a statute that has a questionable fit to the case?

As it happens there is a clear statement principle that bears on this last concern: the rule of
lenity, a judge-made principle saying that ambiguities in penal statutes ought to be resolved in
favor of the criminal defendant. The roots of the rule lie in the concerns similar to those that
produce the high standard of proof for the factual side of a criminal charge. It’s best for courts
not to send people to prison, or worse, unless they are sure they should. This may appear to be
the answer—at last—to your worry about being imprisoned for trying to trade a gun to a drug
dealer. Whether you “used” the gun is open to two readings, so the rule of lenity requires that
you should have the benefit of the doubt. Or so it might seem. But in the actual case of the
traded gun the members of the Supreme Court argued at some length, disagreed, and then a
majority of them said the statute did apply to the defendant. They denied that the statute was
ambiguous and so had no use for the rule of lenity. The defendant got a thirty-year sentence.
The case ended as it did because the rule of lenity isn’t a clear standard of proof after all. It’s a
rule of thumb that has been repeated in so many different words, some strong and some weak,
that judges usually are free as a practical matter to invoke or ignore it at their pleasure. We saw
earlier that imprecision has its advantages; it has its drawbacks, too.
Legal Reasoning: Two Standards of Proof
The legal system in the United States is often referred to as an adversarial system. Very
roughly, this is a system in which two opposing attorneys (adversaries) represent their clients
before an impartial judge or jury. In an adversarial system, with respect to criminal cases, the
prosecuting attorney presents a case on behalf of the government, the defense attorney
presents a case on behalf of his or her client (the accused), and an impartial party or body – the
judge or jury – adjudicates the matter after careful consideration of the competing cases. In the
U.S., the same system applies to both criminal and civil cases. However, in civil cases, a person
or entity (such as a corporation) – known as the plaintiff – accuses another person or entity of
breach of a legal duty (for example, the breaking of a contract, personal injury, etc.). Here again,
competing cases are made: the attorney (commonly called a litigator in civil cases) representing
the plaintiff presents a case on behalf of his or her client, the attorney representing the
defendant presents a case on behalf of his or her client, and either a judge or jury decides the
case.

Importantly, this adversarial system is characterized by a presumption of innocence. What this

means is that the burden of proof falls on the prosecution in criminal cases and on the plaintiff in
civil cases. But, as elaborated in the assigned reading, one standard of proof applies to criminal
cases, while another standard of proof applies to civil cases. In criminal cases, the burden of
proof entails proving guilt beyond a reasonable doubt. However, the burden of proof is lower for
civil cases in which the plaintiff has the burden of proving his or her case by a “preponderance
of the evidence,” which means that the allegations against the defendant are more likely to be
true than not true. The threshold of proof for civil cases is thus 50.1%. Farnsworth refers to this
as the “more likely than not” threshold. But, as Farnworth stresses, no number value can reliably
be associated with the notion of proving guilt beyond a reasonable doubt. Some claim that 75%
should be the threshold, while others say it should be 80%, 90%, or even 100%. It is arguably
the case that expecting certainty (100%) is unreasonable. In criminal cases, one doesn’t have to
be certain that the defendant is guilty; one just needs enough evidence to say that the
defendant is guilty beyond a reasonable doubt. But this is quite different than the threshold of
guilty beyond any doubt.

The O.J. Simpson Case

In 1994, O.J. Simpson, former NFL running back, was arrested in connection with the deaths of
Nicole Brown Simpson (O.J.’s wife) and Ron Goldman – both were found stabbed to death. Five
days after the bodies were found, O.J. and a friend (Al Cowlings) led police on a now infamous
slow-speed chase. Simpson was eventually charged with two counts of murder, but was found
“not guilty” after a highly-publicized court case. Not too long after the criminal case, O.J.
Simpson was sued in a civil case by Ron Goldman’s family. In that case, Simpson was
unanimously found liable for the wrongful death of Goldman and was ordered to pay
$33,500,000 in damages. Those unfamiliar with the American legal system may be perplexed
by this. How could Simpson be found not guilty in the first case, but liable in the second case?
The answer lies in the different standards of proof. In the criminal case, the prosecutor needed
to prove guilt beyond a reasonable doubt. However, in the civil case, the plaintiff only needed to
prove their case by a preponderance of the evidence. As a result, while Simpson was acquitted
of murder (a criminal action), he was found liable for a death (a civil action).

Legal Reasoning: The Common Law Tradition &

Analogical Reasoning
The legal system of the United States was largely “derived many years ago from the English
system, and an essential feature of the English system is its dependence on precedent” (Hurley,
A Concise Introduction to Logic, p. 543). This is referred to as “common law.” In the common
law tradition, significant emphasis is placed on court precedent[1] – the ruling or decision in one
case creates a “precedent” that may be appealed to in future (similar) cases. Common law is
also referred to as “case law” or “judge-made law.”[2] In such a system, lawyers will often find
themselves arguing that there are crucial relevant similarities between a present case and a
past case such that the present case should be decided in the same way the past case was.
But, of course, a lawyer may also need to argue that there are crucial relevant differences
between a present case and past case such that the two cases should not be decided similarly.
When one attorney is arguing for similar treatment, while an opposing attorney is arguing for
dissimilar treatment, judges are often forced to either decide which attorney provided the best
argument or to themselves offer a competing argument for placing the present case in the
context of past cases.[3]

This feature of our legal system (dependence on precedent) makes analogical reasoning (i.e.,
reasoning by analogy) fundamental to legal reasoning.

When legal educators claim that the basic mission of the first year of law school is to train the
student to “think like a lawyer,” it is this sort of analogical reasoning they generally have in mind
– the ability to spot the factual and legal similarities and (more important) differences between
the case under study and related previous cases and to recognize which similarities and
differences are relevant… and which are not. (Phoebe Ellsworth, Legal Reasoning, p. 687).

In light of this, let’s us dig a little deeper into reasoning by analogy. You may remember that
arguments by analogy are inductive arguments. As such, the goal is for our arguments to be
"strong" rather than "weak." Given that analogical reasoning is reasoning that depends on a
comparison of instances, we can judge an analogical argument to be strong only if the instances
are sufficiently similar. If the instances are not sufficiently similar, then the argument is
considered weak. Let’s look again at the basic structure of an argument by analogy that was
provided in a previous lesson on deduction and induction.

Thing A has properties P1, P2, P3, P4…

Thing B has properties P1, P2, P3…
Thus, thing B probably has property P4.
To say that the instances are sufficiently similar is to say that the things being compared share
certain relevant properties or characteristics and also do not diverge in any relevant or important
ways. In the abstract argument above, we need it to be the case that the properties P1, P2, and
P3 are connected in some relevant or important way to P4. If this is the case, and it is also the
case that things A and B are not relevantly dissimilar in any unspecified ways, then we’d
consider the argument strong. However, if either of these necessary conditions fails to be
satisfied, then the argument would be considered weak. Let’s consider a case:

Little Johnny likes to run in the rain, jump in puddles, play in the bathtub, and let the ocean’s
water run over his feet at the beach.
Little Bobby likes to run in the rain, jump in puddles, and play in the bathtub.
Therefore, Little Bobby probably likes to let the ocean’s water run over his feet at the beach.
Is this argument strong or weak? To answer, you’ll need to consider a couple of things. First, are
the shared properties (running in the rain, jumping in puddles, and playing in the bathtub)
relevant to the property that is being inferred in the conclusion (letting the ocean’s water run
over one’s feet)? It is true that they are all water-related activities. But can you generally say that
if one enjoys the first three activities, he or she is also likely to enjoy the fourth? If yes, then the
argument would be on track towards being strong. But we must also consider whether Little
Johnny and Little Bobby are dissimilar in any relevant (but presently unstated) ways. If they are
not, then we might conclude that the argument by analogy is strong. However, let’s imagine that
there is a relevant dissimilarity – namely, Little Bobby is terrified of large bodies of water. If true,
this gives us good reason to doubt that the conclusion probably follows and we’d consider the
argument weak.

All other things equal, the more relevant similarities there are between two things being
compared, the stronger the argument by analogy will be. If the similarities are not relevant, then
they do not matter. For example, let’s imagine that Lucy wants a car that handles very similarly
to Tom’s. The problem is that she doesn’t want Tom’s car because it’s a Toyota and she’s
always been a Honda person. So Lucy’s goal is to identify a car manufactured by Honda that
has similar handling characteristics to Tom’s Toyota. Lucy would need to focus on features of a
car that are relevant to its handling. The type of upholstery (fabric or leather), the gauge cluster,
and the “infotainment” features wouldn’t be relevant or important to the handling of the car. Of
course, these things may be used to break a tie if she identifies two Honda vehicles that handle
like Tom’s, but they don’t matter in terms of the handling. Instead, Lucy should compare things
like the suspension setup, tire setup, vehicle weight, steering characteristics, etc. These are
items that would appear to be relevant to the overall handling characteristic of a car.

All other things equal, the identification of relevant dissimilarities serve to weaken an argument
by analogy. Take the case of Little Johnny and Little Bobby above. If Little Bobby is terrified of
large bodies of water while Little Johnny is not, then that is a relevant point of disanalogy that
weakens the argument. In fact, if it is true that Little Bobby is terrified of large bodies of water,
then that point of disanalogy is so crucial that it alone arguably makes the argument weak. Of
course, if additional relevant dissimilarities were identified, such a discovery would further
weaken the argument.
The Famous Violinist: An Argument by Analogy

Let us now consider a classic case of analogical reasoning found in moral philosophy. I’ll
present the case to you first, then explain afterwards what other sort of case the author believes
is analogous. The case was presented in the 1970's by the philosopher Judith Jarvis Thomson.
Here it is:

You wake up in the morning and find yourself back to back in bed with an unconscious violinist.
A famous unconscious violinist. He has been found to have a fatal kidney ailment, and the
Society of Music Lovers has canvassed all the available medical records and found that you
alone have the right blood type to help. They have therefore kidnapped you, and last night the
violinist’s circulatory system was plugged into yours, so that your kidneys can be used to extract
poisons from his blood as well as your own. The director of the hospital now tells you, ‘Look,
we’re sorry the Society of Music Lovers did this to you – we would never have permitted it if we
had known. But still, they did it, and the violinist is now plugged into you. To unplug you would
be to kill him. But never mind, it’s only for nine months. By then he will have recovered from his
ailment, and can safely be unplugged from you.’ Is it morally incumbent on you to accede to this
situation? No doubt it would be very nice of you if you did, a great kindness. But do you have to
accede to it?

What is your answer? Are you morally obligated to stay hooked up to the famous violinist? Why
or why not? Nearly everyone quickly proclaims that they are not obligated to stay hooked up to
the famous violinist. After all, in this case you were abducted and hooked up to the famous
violinist against your will. As a result, this case would then appear to be a clear violation of your
right over your own body.

So what sort of situation is this case supposed to be analogous to? Thomson utilizes this case
in the abortion debate and her claim is that this case is analogous to cases of pregnancy due to
rape. Let’s consider the relevant similarities. In both the case of famous violinist and the case of
pregnancy due to rape, there is an obvious absence of consent or permission. In both cases,
the survival of one being depends on using the body of another. In both cases, there are major
inconveniences involved – 9 months of having one’s body used. In both cases, the entity using
the body of another, whether the fetus or the unconscious violinist, is innocent. Finally, Thomson
– simply for sake of argument – is willing to assume that fetuses can be said to be persons and,
as a result, have a right to life.[4] Under this assumption, both cases exhibit a conflict between a
person’s right to life (the violinist’s and the fetus’) and another person’s right over their body
(your body and the pregnant woman’s body). Thomson argues that the similarities between the
two cases are so strong that if one claims it is morally permissible to detach oneself from the
from the famous violinist, then one is logically committed to claiming that it is also morally
permissible for a woman to obtain an abortion in a case of pregnancy due to rape.

As you might imagine, Thomson’s argument has caused radical anti-abortionists a lot of grief.
After all, radical anti-abortionists will typically claim that abortion is immoral even in cases of
pregnancy due to rape. But Thomson has argued that the right we have over our own bodies
will sometimes take priority over another being’s right to life. Those who think Thomson’s
argument is mistaken have the burden of explaining why it would be morally permissible for us
to detach ourselves from the famous violinist, but not morally permissible for a woman to obtain
an abortion in a case of pregnancy due to rape. And doing this depends on identifying
morally-relevant differences between the two types of cases. That means identifying some
morally-relevant characteristic present in cases of abortion, where the pregnancy was due to
rape, that is not captured by (and cannot be captured by) Thomson’s famous violinist case.
Absent the identification of such morally-relevant differences, Thomson’s argument would
appear to be strong.

The reasoning process in law is similar. A present case is compared to one or more cases that
have been decided in the past. Below is the abstract form of an argument by analogy that is
common in law. Once the structure of the argument is in place, lawyers may then argue over the
relevant points of analogy as well as any relevant points of disanalogy.

The present case X is similar to past case Y: the cases share relevant features A, B, and C and,
importantly, don’t appear to be dissimilar in any legally-relevant ways.
In past case Y, it was decided that Z.
Therefore, in present case X, it should be decided that Z.
The analogical reasoning process is crucial for it is sometimes said that, in law, we are bound by
the requirement of precedent – similar cases must be decided similarly. If similar cases were not
decided similarly, the legal system would lack continuity and consistency, which would serve as
a source of unfairness.[5] As such, if cases X and Y from above really are sufficiently similar,
then they should be decided similarly (which is precisely what the conclusion is asserting). But
our ability to satisfy the requirement of precedent depends on our ability to identify when two or
more cases are sufficiently similar and, equally important, to accurately determine when
different cases ought to be decided differently. In other words, one’s ability to uphold the
requirement of precedent is underwritten by one’s skill at analogical reasoning.

[1] In our legal system, there are three main sources of the law: statutes (those rules of law
passed by local, state, and federal legislatures), the Constitution (statutes that conflict with the
Constitution, or its amendments, have no legal force), and precedents (judge-made law).

[2] Louisiana is an outlier. While criminal cases in Louisiana are largely treated much like
criminal cases in the other 49 states, civil cases are treated differently. The civil code in
Louisiana has strong Napoleonic (French) roots as well as a notable Spanish influence. One
unique attribute of the Napoleonic tradition is that it doesn’t place the same emphasis on legal
precedent or case law that the other 49 states do. But we shouldn’t overstate the difference:
Louisiana judges do benefit from a couple centuries of case history, even if such case history
doesn’t serve as the fundamental basis for their rulings. Nevertheless, as a result of its French
and Spanish influences, some civil laws in Louisiana are unique and not prominent in the other
49 common law states (e.g., laws governing how estates are handled, how property is divided
up in a divorce, how contractual disputes are managed, etc.).

[3] What if there is no precedent? A case lacking a clear precedent is known as a case of first
impression.

[4] Thomson does not think this assumption is warranted, at least not in the early stages of
pregnancy. Again, she’s only assuming this for the sake of argument.

[5] It does not follow from this that past decisions are always treated as correct and just.
Precedent can be overturned if there is good reason to do so. When this happens, an appellate
court (i.e., a court of appeals), for example, may take-up and review a decision from a lower
court. At any rate, “our legal system assumes that, if there is no adequate reason to overturn a
precedent or to distinguish the precedent from the present case, then the precedent provides
some reason to decide the present case in the same way as the precedent” (Sinnot-Armstrong
& Fogelin, “Components of Legal Reasoning”).

Legal Reasoning: Ex Ante and Ex Post Reasoning

Another important tool in the legal reasoner’s toolkit is the distinction between ex ante reasoning
and ex post reasoning. This isn’t a tool useful for only those who practice law on a day-to-day
basis; it is something that can help to illuminate the reasoning behind laws that we might think
are, at first pass, odd or unusual. Consider the following case:

A thief walks into a bank, puts a gun to the head of one of the customers, and announces that
he will shoot unless the teller hands over all the money in the drawer. The teller does nothing.
The thief shoots the customer, runs off, and is never seen again. The customer dies of his
injuries, and his estate brings a lawsuit against the bank, complaining that the teller should have
given the money – it was only $5,000, let us imagine – to the thief. What should the court say?
(Farnsworth, The Legal Analyst)

There are two really different vantage points from which you might approach this case. On the
one hand, you might view this as merely a legal dispute between the bank and the estate of one
of its customers. On this approach, one attempts to adjudicate this case by merely analyzing the
past facts of the case – the details of what has already happened. One then asks, does justice
require the bank to pay damages to the plaintiff? “If the teller’s refusal saved the bank money,
would it be fair for the bank to pay nothing to the party who was injured as a result (or, more
precisely, to his estate)” (Farnsworth, p. 3)?

But now let’s consider a second way of analyzing the case. On the second approach, what
happened that day at the bank is actually of secondary interest. Instead, one looks ahead to the
future and asks whether a particular ruling would reduce the likelihood of a similar event
occurring again. The concern here is with incentives. Will certain rulings incentivize certain
unwanted actions? Let us imagine here that the court rules in favor of the plaintiff and the bank
then has to pay damages to the estate of the deceased customer. What sort of incentives will
this create? Here’s one possibility: if the bank is ordered to pay damages, then in the future
banks will be incentivized to hand the money over when thieves take hostages (assuming the
damages are greater than what the thief would have gotten). But the worry is that this will in turn
incentivize thieves to take hostages. “The trouble becomes apparent. Letting the customer’s
estate win might cause more hostages to be taken in the future” (Farnsworth, p. 5). On this
view, one may be inclined to side with the bank as a means to avoiding such perverse
incentives. The reasoning here is similar to the reasoning behind national policies of not
bargaining with terrorists. It is often claimed that by bargaining with terrorists, nations will
inadvertently incentivize future terrorist threats against them.

We are now confronted with two potentially competing considerations. First, what would be a fair
judgment given the past facts of the case? Second, what is the right rule or precedent for the
future? The first consideration characterizes the ex post perspective. It entails looking back at
an event that has already occurred and deciding what to do about it. The second consideration
characterizes the ex ante perspective. It entails looking forward and asking what effects a
judgment in this case will have on similar situations in the future. At the center of ex ante
thinking is the consideration for incentives. Certain ways of dealing with injustices or wrongs
may encourage future acts of a similar type. In a nutshell, ex post reasoning is backward looking
while ex ante reasoning is forward looking.

In a legal context, one is not relegated to thinking in terms of just one perspective or the other.
Legal professionals think in both ways when they analyze cases. Moreover, while it is
unfortunate, our ex ante reasoning and ex post reasoning may independently yield conflicting
judgments. That is, how we’d judge a case based on ex ante thinking may not agree with how
we’d judge a case based on ex post thinking. “Does a court’s decision settle the dispute
between these parties or does it make a rule for others in the future? It almost always does
both” (Farnsworth, p. 5). Moreover, legal professionals may find themselves having to argue
over which type of reasoning – ex ante or ex post – should take priority in a given case.

Ex post thinking comes pretty naturally to us. As a result, decisions based on ex post reasoning
may be easier for us to comprehend. It is important to see though that the courts aren’t
concerned solely with ex post reasoning. So, the next time you encounter a legal judgement or
law that seems unusual from the ex post perspective, ask whether it makes any sense from the
ex ante perspective.

Outside of legal contexts, ex ante reasoning has its place as well. As such, if you're familiar with
both types of reasoning, you're less likely to be caught off guard when someone counters your
ex post considerations with ex ante considerations. If you have the foresight to see that coming,
then you’ll be better prepared to offer a rejoinder. Also, when parents set rules for their children,
they have to think carefully about the ways in which those rules might incentivize unwanted or
undesirable behavior. This is also true of employer-employee relationships.
An Encroachment Case

Imagine that you’ve built a house on a piece of land that you own. The problem is that you were
mistaken about where your property lines lie and it turns out that your house extends two feet
over your property line onto your neighbor’s property. Your neighbor is not too happy about this,
and it seems that there is no cheap way to solve the problem. Let’s imagine that your house
either has to stay where it is or it has to be torn down (it can’t be relocated intact). What should
be done?

Consider that someone has proposed, from the ex post perspective, that you pay your neighbor
for that two-foot strip of land that your house now occupies. You won’t pay just the “fair market
value” though; you’ll have to pay a slight premium due to the inconvenience you’ve caused your
neighbor. Does this work? Before you answer, take a moment to consider the ex ante
perspective.

From the ex ante perspective, undesirable incentives will be created. Let’s imagine that I am
aware of your case, and we’ll assume momentarily that you were simply forced to purchase that
strip of land from your neighbor at a slight premium to fair-market value. Like you, I want to build
a house, but I don’t have the land space I need to build the house I want. I really need to use
some of my neighbor’s land – but I don’t think they’d be willing to sell me what I need. Never
mind, I’ll just go ahead and build my house, even if it means encroaching on my neighbor’s land.
They can then sue me for the fair-market value of the land plus a slight premium consistent with
the decision in your case.

The problem is that the judgment in your case failed to create any incentive for me to be more
careful about not transgressing my property lines. Instead, the opposite happened – I was
incentivized to not care too much about property lines. It is for this reason that, in such
encroachment cases, the plaintiff (the neighbor) is usually entitled to demanding that the house
be removed. Now, if that happens, think about the incentives: if you had to remove your house
at a great cost to yourself that is certainly going to disincentivize me from building first and
apologizing later.

Lesson 8
Overview
Statistics are used to persuade and are frequently incorporated into arguments. Statistics are
subject to abuse. We need to be alert to the possibility that others intend to utilize statistics in
misleading and deceptive ways. The best defense is to be capable of evaluating a statistic. This
requires that we evaluate how the statistic was compiled.
The fundamental lesson about statistical reasoning is situated in the context of public opinion
polls. They are pervasive and frequently presented in ways that are highly misleading. Once
one is familiar with some basic issues in sampling, it is much easier to anticipate and identify
compilation errors which lead to inaccurate statistics.

Objectives
You will be able to:
-demonstrate understanding of margin of error, sample size, and confidence level
-identify the main barriers to random sampling
- distinguish a biased sample from a representative sample

Samples (Hurley)
Most statistical evidence presented in support of inductively drawn conclusions is gathered from
analyzing samples. A sample that is found to possess a certain characteristic is used to argue
that the group as a whole possesses that characteristic.
Ex: we want to know what the student body at a certain university thought about a proposal to
adopt an academic honor code. We can take a poll of 10% or students, and if the results of the
poll show that 80% of those sampled favored the code, we might draw the conclusion that 80%
of the entire student body favored it. This is classified as an inductive generalization.
- The problem is whether the sample is representative of the population
- Samples that are not representative are biased
Determining whether the sample is biased
1. Whether the sample is randomly selected
2. The size of the sample
3. Psychological factors
Random sample - one in which every member of the population has an equal chance of being
selected. The requirement that a sample be randomly selected applies to practically all samples,
but it sometimes can be taken for granted.
Ex: a physician draws a blood sample to test for blood sugar. There is no need to take a little bit
from the finger, arm, and leg because blood is a circulatory fluid and you can assume that it is
homogenous in regard to blood sugar.
Randomness demands more attention when the population consists of discrete units. A quality
control engineer for a manufacturing firm needs to determine whether the components on a
certain conveyor belt are within specifications. The engineer removes every tenth component for
measurement. The sample obtained would not be random if the components were not randomly
arranged on the conveyor belt. A result of some malfunction in the manufacturing process could
have impacted every tenth component and if the engineer only selected the perfect ones, the
sample would be biased. A selection procedure more likely to ensure a random sample would
be to roll a pair of dice and remove every component corresponding to a roll of ten. Since the
outcome of a roll of dice is a random event, the selection would also be random. This would be
more likely to include defective components that turn up at regular intervals.
Randomness is even more important for humans.
Ex: a poll conducted on excessive corporate profits is conducted on Wall Street in New York
City. Clearly biased in favor of corporations.
- A less biased sampling would be random phone numbers from the telephone directory,
but even this would not yield a completely random sample. Among other things, the time
of day in which a call is placed influences the kind of responses obtained. Most people
who are employed full time are not available during the day, and even if calls are made
at night, a large percentage of the population have unlisted landline and cell phone
numbers.
- A poll conducted by mail would also yield a fairy random sample, but this method also
has its shortcomings. Many apartment dwellers are not listed, and others move before
the directory is printed. Furthermore, none who live in rural areas are listed.
It is both difficult and expensive to conduct a large scale public opinion poll that succeeds in
obtaining responses from anything approximating a random sample of individuals.
Ex of biased poll: Literary Digest magazine poll to predict the outcome of the 1936 presidential
election. Sample consisted of a large number of the magazine’s subscribers together with other
people selected form the telephone directory. Four similar polls had picked the winner in
previous years so the results were highly respected. However, the Republican candidate Alf
Landon got a significant majority in the poll but Frankling D Rooselevent won the election by a
landslide. The incorrect prediction is explained by the fact that the 1936 election occurred in the
middle of the Depresson, when many people could afford neither a telephone nor a subscription
to the Digest. These people were overlooked in the poll and voted for Roosevelt.
Size is also an important factor. The larger the sample, the more closely it replicates the
population. In statistics, this degree of closeness is expressed in terms of sampling error.
- Sampling error is the difference between the relative frequency with which some
characteristic occurs in the sample and the relative frequency with which the same
characteristic occurs in the population
- Ex: poll taken of a labor union and 60% of members sampled expressed their
intended vote for Smith but only 55% of the union intended to vote. Sampling
error would be 5%
How large a sample should be is a function of the size of the population and the degree of
sampling error that can be tolerated.
- Ex sampling error of 5% in a population of 10,000 would require a larger sample than a
population of 100. However, the ratio is not linear.
- To obtain precision, the sample for the larger population need not be 100 times as large
as the one for the smaller population. When a population is very large, the size of the
sample needed to ensure a certain precision level off at a constant figure.
- Ex: a random sample of 500 million will yield results that are just as accurate
whether the population is 100,000 or 100 million.
When population is very large and the sample is random and less than 5%, sampling error can
be expressed in terms of a mathematical margin of error as per Table 12.1:
These figures are based on a 95% level of confidence, which means that we can be 95%
certain that a sample will accurately reflect the whole population within the margin of error for
that sample. Thus, if in a random poll of 9,604 people 53% say they prefer Jones for governor,
we can be 95% certain that about 52-54% of the whole population prefers Jones.
- Figures are based on mathematica and are not dependent on any empirical
measurement. For a 95% confidence level, the margin of error is approximately equal to
.98/Ön where n is the size of the sample. For a 99% confidence level, the margin of error
is approximately equal to 1.29/Ön. Comparing these two expressions shows that as
the confidence level increases, so does the margin of error.
Most polls base their margin of error on a 95% or higher confidence level. If a much
lower confidence level is selected and this is not disclosed, then the result of the poll
could be deceptive, even if the margin of error is stated.
- Any poll that fails to disclose the margin of error can be deceptive.
Even when the margin of error is stated, incorrect conclusions are often drawn.
- Ex: a poll shows Adams leading Baker 55 to 45. An article is published criticizing
Adams and a second poll is taken showing Adams trailing Baker by 49 to 51. The
margin of error is 4% and the confidence level is 95%. People often draw the
conclusion that Adam’s lead disappeared. No such conclusion is warranted. The
margin of error of 4% means it could be the case that Adam’s lead has remained
constant, say 53 to 47.
Advertisers and marketers of products synch as patent medicines have been known to
take rather small samples until they obtain one that gives the “right” result.
 - Ex: several polls of 25 people might be taken inquiring about the preferred brand
of aspirin. Even though the samples may be random, one will eventually be found
in which 20 of the 25 respondents indicate their preference for Alpha brand
aspirin. The marketing firm can then promote the brand as the one preferred by
four our of five of those sampled.
Psychological factors can also have a bearing on whether the sample is representative.
When the population consists of inanimate objects, such as cans of soup or machine
parts, psychological factors are usually irrelevant, but they can play a significant role
when the population consists of human beings.
- If the people composing the sample think they will gain or lose something by the
kind of answer they give, their involvement will likely affect the outcome.
- Ex: if residents of a neighborhood were to be surveyed for annual income with
the purpose of determining whether the neighborhood should be ranked among
the fashionable areas of the city, we would expect the residents to exaggerate
their answers. But if the purpose was to determine whether the neighborhood
could afford a special levy that would increase property taxes, we might expect
the incomes to be underestimated.
The kind of question may have psychological bearing.
- How often do you brush your teeth? And how many books do you read in a year?
Can be expected to generate responses which overestimate he truth
- How many times have you been intoxicated? And how many extramarital affairs
have you had? Would probably receive answers that underestimate the truth
Similar exaggerations can result from the way a question is phrased.
- Do you favor a reduction in welfare benefits as a response to rampant cheating?
Would be expected to receive more affirmative answers than simply do you favor
a reduction in welfare benefits?
Another source is the personal interaction between the surveyor and the respondent.
- Ex: door to door survey to determine how many people believe in God or attend
church on Sunday. If the survey were conducted by priests and ministers dressed
in clerical garb, one might expect a larger number of affirmative answers than if
the survey were taken by nonclerics.
The simple fact is that many people like to give answers they think will please the
questioner.
To prevent this interaction, studies are often conducted under “double blind” conditions
in which neither the surveyor nor the respondent knows what is the “right” answer.
- Ex: a double blind study to determine the effectiveness of a drug may use bottles
containing the drug mixed with other bottles containing a placebo. The contents
of each bottle would be matched with a code on the label, and neither the person
distributing the bottles nor the person recording the responses would know what
 the code is. Under these conditions, the people conducting the study would not
be able to influence the response of the people to whom the drugs are given.
Most statistical evidence encountered in ordinary experience contains no reference to
such factors as randomness, sampling error, or the conditions under which the sample
was taken. In the absence of this information, the person faced with evaluating the
evidence must use their best judgement.
If either the organization conducting the sample, or the people composing it have
something to gain by the answers given, the results of the survey should be regarded as
suspect.
If the questions that are asked concern topics that would naturally elicit distorted
answers, the results should be rejected.
In either event, the mere fact that a study appears scientific or is expressed in
mathematical language should never intimidate a person into accepting the results.
Numbers and scientific terminology are no substitute for an unbiased sample.

Statistical Reasoning: Public Opinion Polls

statistic - quantitative piece of information derived from the study of a sample (a subset of some
population) or from the study of a whole population
- Why do we care about this in a logic & critical thinking class?
- We care because arguments often incorporate statistics (and statistics sometimes
influence our decisions)
Consider the following inductive generalization which relies on statistics:
- 60% of the 500 students surveyed at this college identify as pacifists.
- Therefore, 60 percent of all students at this college probably identify as pacifists.
And here’s the abstract form of the argument.
- X percent of the observed members of group G have property P.
- Therefore, X percent of all members of G probably have P.
The strength of such an argument depends on an evaluation of the relied upon statistic
- Evaluating the statistic means evaluating how the statistic was compiled.
-
Issues in sampling as they relate to public opinion polls
Public opinion polls (or at least the math behind them) can be highly sophisticated.
- “They are used to arrive at generalizations about everything from the outcome of
presidential elections to public sentiments about cloning babies to the consumer’s
appetite for tacos. But as complex as they are, opinion polls are still essentially inductive
arguments (or the basis of inductive arguments) and must be judged accordingly”
(Vaughn, The Power of Critical Thinking, p. 291).
In light of this, here are some examples of statistic-based claims which are the result of
inductive inferences and which are typically made as a result of a public opinion poll.[1]
 - 16% of American adults say that abortion should be “illegal in all circumstances.”
(3/2016)
- 53% of American adults support Obama’s proposal to provide “free tuition to attend
community colleges at a cost to the federal government of sixty billion dollars over 10
years.” (1/2015)
- 60% of registered voters support “stricter guns laws” in the United States. (11/2017)
We want to have a clearer picture of exactly how these sorts of claims are arrived at and what
factors affect the reliability of such claims.

Distinguishing between a sample and a census.

- sample is a set of data (observations) drawn from a portion of a population
- The goal is to use the sample data to make an inductive inference about the
whole population from which the sample data was drawn.
- Ex: you are trying to determine which political party is most endorsed by CSN students.
One way of making this determination would be to poll every single student at CSN. To
do this would be to take a census.
- census is a study of every unit, everyone or everything, in a population.
- known as a complete enumeration, which means a complete count (Australian
Bureau of Statistics).
- Completing a census, however, can be extremely costly in terms of time and
money.
- Ex the 2010 U.S. Census, which attempted to contact every household in
the U.S., cost 13 billion dollars and utilized 635,000 temporary workers (a
lot of people were needed to go door to door)
- In other situations, it may be nearly impossible to enumerate all units of
the population without unreasonable costs.
So, utilizing a sample (a partial enumeration) often makes the most practical sense.
Two things are important here.
1. It is when we survey only a sample, but then go on to draw a conclusion about the whole
population that we make an inductive inference. No such inference exists when one
actually surveys every member of a population.
2. As soon as one diverts away from a census, a number of potential issues arise with
respect to our ability to create a sample that reasonably models (i.e., that accurately
reflects) the whole population.
The primary sampling issues that arise revolve around the following three questions.
1. Is the sample size adequate?
2. Was the sample randomly selected?
3. Are there any psychological factors that may distort the data sample?
Statistical Reasoning: Sample Size, Margin of Error,
& Confidence Level
In order to evaluate the adequacy of a sample size, we have to discuss three interrelated
concepts: sample size, margin of error, and confidence level. The sample size is the total
number of “units” (persons or things) that you have “observed” (analyzed, polled, etc.).[1] But
let’s again suppose that you are looking to conduct a poll to determine which political party is
most endorsed by CSN students. So you ask, what sample size do I need? Answering that
question turns on what margin of error and confidence level you are comfortable with. As such,
these three concepts are interrelated.

The margin of error is a measure of the expected accuracy of a statistic. In sampling, it

expresses the maximum expected difference between the values derived from a sample and the
values that are likely if a census were actually performed. Let’s simplify this and say that the
margin of error is a measure of the expected accuracy of a statistic expressed as a range. So,
let’s imagine that you complete your poll and find that 60% of those CSN students polled
endorse the Republican Party. That is, 60% of your sample endorses the Republican Party. You
don’t have the time or resources to perform a census – to poll every CSN student – so you must
make an inference based on the sample data. But since you are making an inference about the
whole population of CSN students based on a survey of just a portion of the CSN student body,
your inference cannot be assumed to be precise. That is, some error is likely to be introduced.
Let’s assume that, via the appropriate mathematical calculations, it is determined that the
margin of error for your poll is 5%. Remember, in your poll, 60% of CSN students endorsed the
Republican Party. So how do we get our margin of error expressed as a range? It’s simple,
subtract 5% from your statistic to derive the low end of the range and add 5% to the statistic to
derive the high end of the range. As such, if your margin of error is 5%, this means the statistic
in question – 60% – is most likely somewhere between 55% and 65%. That is, if you were able
to perform a census, you’d expect the percentage of people who actually endorse the
Republican Party to be as low as 55%, but as high as 65%. Importantly, then, the margin of
error is applied both above and below the statistic. Finally, a 5% margin of error is often written
as ±5.0.

But now imagine that you perform a census – polling every single student – and discover that, in
fact, only 57% of students endorse the Republican Party. This means you were off by 3%. In
other words, you had a sampling error of 3%. That’s a good thing as your sampling error fell
within the expected margin of error. To summarize the distinction between margin of error and
sampling error, we can say this: the margin of error is the expected (or theoretical) range of error
while the sampling error is the actual (or empirical) error.[2] Sometimes the sampling error falls
outside the margin of error. For example, you might conduct a poll with a margin of error of ±5.0
but have an actual error (i.e., sampling error) of 8%. This indicates that some aspect of your
polling methodology may have been corrupted – perhaps by factors that we’ll discuss below. At
any rate, the important thing to remember is that “[e]very instance of sampling is only an
approximation of the results that you would get if you polled every single individual in a target
group [i.e., in the whole population]” (Vaughn, p. 294).

Finally, the confidence level is the level of confidence you can have that a statistic derived from
a sample will accurately reflect the whole population within the margin of error for that sample.
As Lewis Vaughn puts it, “in statistical theory, the confidence level is the probability that the
sample will accurately reflect the target group within the margin of error. A confidence level of 95
percent (the usual value) means that there is a 95 percent chance that the results from polling
the sample (taking into account the margin of error) will accurately reflect the results that we
would get if we polled the entire target population.” (Vaughn, p. 295)

Take for example a poll regarding a presidential election. Let’s imagine that the poll concludes
that candidate X will receive 60 percent of the popular vote. Let’s further imagine that the poll
has a margin of error of ±3.0 and a confidence level of 95%. In light of the margin of error, we’d
expect candidate X to actually receive between 57% and 63% of the popular vote. However, the
confidence level introduces even more uncertainty. What a 95% confidence level tells us is that
there is a 95% chance that that candidate X will actually receive between 57% and 63% of the
popular vote. In other words, there is a 5% chance that the actual results will fall outside of this
range – i.e., there is a 5% chance that candidate X will receive either less than 57% of the
popular vote or more than 63% of the popular vote.[3] Here’s the takeaway: all opinion polls that
are samples entail uncertainties.

Ok, so now you’ve got a basic understanding of sample size, margin of error, and confidence
level. But you still want to answer the following question: what is the appropriate sample size for
my study? In order to get that, some calculations need to be performed. Given a desired margin
of error and confidence level, we can calculate the required sample size. But this isn’t a
statistics class and I’m not going to expect you to hand-calculate anything. Fortunately, there are
enough tables and sample size calculators available online so that we don’t have to manually
perform these calculations. Here’s a table that is useful for determining what sample size you’ll
need (in order to achieve a particular margin of error and confidence level) given the population
size.
Let’s return to our poll of CSN students. Assume there are 25,000 CSN students spread across
the three main campuses. Also assume that you’d like a 5% margin of error as well as a 95%
confidence level. According to the table above, you’d need to poll at least 378 students to get
the desired margin of error and confidence level. An important warning is in order here. In
theory, you can get a 5% margin of error and a 95% confidence level out of a population of
25,000 by polling just 378 “units.” However, in order to achieve these results, everything else
about your poll has to be perfect. But as we’ll see below, that is usually not the case. As a result,
polling organizations often have to adjust the sample size upwards to account for various
imperfections in opinion polls.
With regard to the question of whether one’s sample size is adequate, this is mostly an issue
when population sizes are really small (say, under 1,000).[4] Once your sample size reaches
384, the population size doesn’t matter. If the goal is to achieve a 5% margin of error and a 95%
confidence level, then a sample size of 384 is in theory sufficient regardless of whether the
population size is 100,000 or one billion.[5] Still, the larger the sample, the more closely it will
reflect the population. In other words, the lower the margin of error (ME) and the higher the
confidence level (CL) you desire, the larger your sample needs to be. For our hypothetical CSN
poll, if we desired 1% ME and a 95% CL, we’d need a sample size of 6,939 students. If we
desired a 1% ME and a 99% CL, we’d need a sample size of 9,972 students. Of course, the
time and resources to survey that many students will be higher as well, so one has to balance
accuracy with time and resource constraints.

[1] Each item in a sample size is a “sample unit.” The sample unit can be either a living being or
an inanimate object. For example, you might want to know the average height of kids at a local
elementary school. Alternatively, you might want to know what percentage of cars in Las Vegas
is yellow. If you have to physically take the height of the children, then you analyze the sample
units. However, if you wanted to know what percentage of those children preferred chocolate
milk over white milk, then you’d poll or survey them.

[2] You might wonder how it is the case that one can know the sampling error if one didn’t have
the time and resources to perform a census. This really comes into play when conducting polls
on matters that will later be voted on. Polling organizations often try to predict at various
moments in time whether a candidate will win or whether a policy proposal will pass when put to
public vote. While voting results don’t count as a census unless every qualified voter actually
votes, the actual vote typically captures input from a much larger portion of the population than
an opinion poll does. The goal for the polling organization is to predict – with as little error as
feasible – the actual outcome.

[3] Here’s another way to think about confidence level. “Our ‘95 percent confidence’ means that
if we conducted 100 different polls on samples drawn from the same population, we would
expect the answers we get from our sample in 95 of those polls to be within 3 percentage points
[if margin of error is 3%] in one direction or the other of the population’s true sentiment”
(Wheelan, Naked Statistics).

[4] This is why sample size calculators often don’t ask for the population size unless it is
relatively small. An alternative formula is then used to adjust sample sizes in light of a small
population.

[5] According to the sample size calculator at checkmarket.com

(https://www.checkmarket.com/market-research-resources/sample-size-calculator/), you’d need
a sample size of 385 if you wanted to make an inductive inference about all 7 billion people on
the planet with a 5% margin of error and a 95% confidence level.
Statistical Reasoning: Random Sampling &
Psychological Factors
Random Sampling

For a sample to be randomly selected, it must be the case that every member of the population
has an equal (non-zero) chance of being selected. Imagine that we are again trying to
determine which political party is most endorsed by CSN students. For your findings to have a
chance of being representative of the whole population, it must be the case that students are
randomly selected. Would your sample be randomly selected if you polled only students in the
CSN Student Republicans Club? Nope. What if you started by polling all the students presently
enrolled in macroeconomics or sociology or environmental ethics? Again, no – members of the
green party may be more inclined to take environmental ethics…. so our sample would be
skewed. So, how could we get a truly random sample? Maybe we take a list of all student IDs,
then use a random generator to start selecting IDs. This is probably the only way to guarantee
that every student has an equal non-zero chance of getting selected. But notice, this really only
works if everyone who is randomly selected actually completes the survey. As you can probably
imagine, the non-response rate is going to be high.

So what happens if only 15% of those contacted actually complete the survey? We’d then need
to contact many more students than we’ll get a response from. But this may create a new
problem: our sample might be skewed if, for example, republicans are more likely than
democrats to complete surveys. To see the problem, consider an example involving phone
surveys. A polling organization might make phone calls to the public in an effort to determine the
approval rating of a policy that provides a helpful service to the unemployed. They expect the
non-response rate to be high (as many people just won’t answer or will hang up when they
realize it’s a survey), so they have to make a large number of calls to get the required number of
completed surveys. All calls are made between 9am and 6pm. Will this skew the sample? It
depends… if the unemployed are more likely to be available at those times, then the worry is
that those who answer will be more likely to approve of a policy that benefits the unemployed. If
this happens, the sample won’t be representative. So, one significant challenge for pollsters is
to find ways to try to get as representative of a sample as possible.[1]

Psychological Factors

In conducting an opinion poll, we also want to avoid having the results heavily skewed by
psychological factors. It is probably impossible to avoid having the results skewed by any
psychological factors whatsoever. That said, there are some steps we can take to limit this
skewing. Here are some questions you need to ask:

1. Is the question asked in a biased (non-neutral) way?

An example of a biased (non-neutral) question: “Do you approve of the murder of fetuses?” This
question simply assumes that abortion constitutes an unjust killing. But whether abortion is
murder (an unjust killing) is precisely the point up for discussion. So, perhaps the question
asked should be: “Do you approve of the killing of fetuses?” Although abortion is technically the
killing of a fetus, the term “killing” might be intended to emotionally direct the respondent. So,
perhaps the better question is: “Do you approve of abortion?” The point is that we have to
consider whether the wording of our inquiries exhibits biases that might influence respondents
one way or the other.
2. Will social pressures/expectations influence the answer?

Examples of questions in which social pressures or expectations may cause the respondent to
either exaggerate or lie: “How many times a month do you have sex?” “How many times have
you snorted cocaine in your life?” “How many times have you got so drunk that you wet the
bed?” “How many books do you read in a year?” First, respondents may be hesitant to give
accurate answers about very personal matters (e.g., sexual practices). Second, respondents
may purposely exaggerate/de-exaggerate or lie if they feel they are being judged or they are
embarrassed by the truth.
3. Do the respondents stand to gain or lose something if they answer in a particular way?

An example of a question in which respondents might be seen as standing to gain or lose

something: “How would you rank the level of safety in the neighborhood in which you live?” A
respondent might worry about their property values when answering this (especially if they know
the result will be publicized).

[1] Because of these sorts of challenges, you’ rarely see national polls of American adults (more
than 187 million) conducted with a sample size of 384. Sample sizes in national polls are often
in the range of 1,000 to 1,500 even for a 5% ME and 95% CL. The greater sample size is
needed to reflect the many challenges that come with trying to get a truly representative sample.

Statistical Reasoning: Representative Samples &

Media Deception
Representative (Non-Biased) Samples

A biased sample does not accurately reflect the characteristics of the whole population, usually
because the sample is: (1) too small; (2) not random; and/or (3) skewed by psychological
factors. A representative sample is non-biased. Hence, a representative sample avoids the
problems that are associated with biased samples and, as such, accurately reflects the
characteristics of the whole population. Whenever sampling is performed, the goal is to get a
non-biased sample.
Why is this important? Once again, people may incorporate statistics from sampling into
inductive arguments. Whether such an argument is strong or weak will depend on the
evaluation of the statistic(s) in question. But that means having a basic understanding of sample
size, margin of error, confidence level, and representative samples.

Check Your Understanding: For each of the following, determine whether the sample appears to
be biased? If you think it is biased, try to explain why.

1. Ben wants to determine the average height of UNLV students. Since he is an athlete, he
begins and ends his sample with students involved in athletics.

2. Julie wants to know what proportion of Las Vegas residents approve or disapprove of ISPs
(Internet Service Providers) tracking and preserving customer internet activity. She randomly
polls 1,000 residents with the question “Do you approve or disapprove of ISPs violating your
right to privacy by tracking and preserving your internet activity?”

3. Jen wants to know what proportion of CSN students favor or oppose the death penalty. She
randomly polls 50 students with the question “Do you strongly favor, favor, oppose, or strongly
oppose the death penalty?”

4. Frank wants to know how frequently CSN students cheat on exams. So he randomly polls
500 students with the question: “How often do you cheat on exams at CSN: Never, rarely,
sometimes, often, almost always?”

Answers

1. Biased. The sample is not going to be random if athletes are likely to be taller than
non-athletes.

2. Biased. There is a psychological factor here since the question is asked in a non-neutral way
by implying that such tracking amounts to violating one’s right to privacy.

3. Biased. The sample size is too small.

4. Biased. There is a psychological factor here since responses may be affected by societal
pressures or expectations.

Deceiving With Public Opinion Polls

Consider the following hypothetical scenario: One week ago Gallup reported that the President’s
approval rating was at 57%. Today, Bloomberg reports that the approval rating is at 47%. In light
of this, a news organization runs the following headline:

Presidential approval rating plummets. Presidential rating now at 47%.

What’s the problem? First, we don’t know the margin of error. If these opinion polls had a 5%
margin of error or greater, then the data doesn’t tell us much. The effect may not be due to an
actual drop in support at all, but rather due to comparing two different samples that may have
been derived in different ways. If we assume a 5% margin of error, then in the first poll, the
actual approval rating might have been 52% (the lower end of the range). Yet, in the second
poll, the actual approval rating might also have been 52% (the upper end of the range). Maybe
there is likely some drop here, but the headline attempts to exaggerate what the data tells us.[1]
Second, we have no idea whether both polls asked the same question in the same way. If the
questions were different, perhaps that explains the variation in results.

The lesson here is to be cautious: news commentators may interpret opinion polls in
inappropriate ways in order to facilitate preconceived narratives or to create new narratives. To
show such a slide in the approval rating, we’d really need more data points with a focus on
general trends.[2]

Consider another case: a study finds that 52% of Americans disapprove of the recreational use
of marijuana. +/- 5.0, 95% CL. In light of this, a news organization runs the following headline:
More than half of all Americans disapprove of the recreational use of marijuana.

The problem here is that the newspaper is not at all justified in publishing this headline. After all,
in light of the margin of error, all that we can say is that, if a census were performed, then the
percentage of Americans who disapprove of the recreational use of marijuana is most likely
between 47% and 57%. In other words, according to the poll, it may actually be the case that
less than half of Americans disapprove of the recreational use of marijuana.

[1] It would be a much more complicated to figure out the statistical likelihood of the actual error
of one poll being at its far lower range while another poll (conducted about the same matter)
simultaneously has an actual error at its far upper range. In a best case scenario, this might
warrant one to conclude that at least some drop in the approval rating is likely. But whether this
best case scenario holds depends on how well the two polls were conducted and how similar
the polling questions were.

[2] Drawing an inference from data aggregated from a larger number of polls would be
preferable to drawing an inference on the basis of the results of just two different polls.