The Nature of Evidence

Introduction

In scientific research, evidence is crucial for supporting or refuting a hypothesis, model, or theory. The quality and type of evidence determine the strength of the conclusions that can be drawn. Understanding the nature of evidence is essential for designing sound investigations and interpreting research findings.

Types of Evidence

Quantitative Data

• Definition: Numerical data that can be measured and statistically analyzed.
• Examples: Scores on a mental wellbeing scale, reaction times, frequency of specific behaviors.
• Characteristics:
  • Objective and measurable.
  • Allows for statistical analysis to determine significance.
  • Can be displayed in graphs, charts, and tables.
• VCE Relevance: Student-designed investigations in VCE Psychology typically involve the generation of primary quantitative data.

Qualitative Data

• Definition: Non-numerical data that describes qualities or characteristics.
• Examples: Interview transcripts, observational notes, open-ended survey responses.
• Characteristics:
  • Subjective and descriptive.
  • Provides rich, detailed insights.
  • Analyzed through thematic analysis or content analysis.
• Note: While not the primary focus of VCE student investigations, qualitative data can provide valuable context and depth to research.

Primary vs. Secondary Data

• Primary Data: Data collected directly by the researcher for a specific purpose.
  • Example: Data from your own experiment.
• Secondary Data: Data that has already been collected by someone else.
  • Example: Data from published research articles, government statistics.

KEY TAKEAWAY: Quantitative data, the primary focus of VCE Psychology student investigations, provides objective and measurable evidence that can be statistically analyzed to support or refute a hypothesis.

What Constitutes Good Evidence?

Reliability

• Definition: The consistency and stability of a measurement. A reliable measure produces similar results under similar conditions.
• Types:
  • Test-retest reliability: Consistency of results when the same test is administered to the same person at different times.
  • Internal consistency: Consistency of results across different items within the same test.
  • Inter-rater reliability: Consistency of results between different raters or observers.
• Importance: Reliable evidence is essential for drawing valid conclusions.

Validity

• Definition: The extent to which a measurement tool accurately measures what it is intended to measure.
• Types:
  • Face validity: The extent to which a test appears to measure what it is supposed to measure.
  • Content validity: The extent to which a test covers all aspects of the concept being measured.
  • Construct validity: The extent to which a test measures the theoretical construct it is designed to measure.
  • Criterion validity: The extent to which a test predicts performance on a related criterion.
• Importance: Valid evidence ensures that the research is measuring the intended variables.

Objectivity

• Definition: Minimizing personal bias in the collection and interpretation of data.
• Strategies:
  • Using standardized procedures.
  • Employing objective measurement tools.
  • Blinding participants or researchers to the study conditions.
• Importance: Objectivity increases the credibility and trustworthiness of the evidence.

Sample Size

• Impact: A larger sample size generally provides more reliable and representative data.
• Considerations:
  • Larger samples reduce the likelihood of random error.
  • Sample size should be appropriate for the research question and design.
• VCE Relevance: In student investigations, sample sizes are often limited, which should be acknowledged as a limitation.

Control of Extraneous Variables

• Definition: Identifying and controlling variables that could influence the results, other than the independent variable.
• Importance: Controlling extraneous variables helps to establish a cause-and-effect relationship between the independent and dependent variables.
• Examples: Standardized instructions, random assignment of participants to conditions.

EXAM TIP: When evaluating research, always consider the reliability and validity of the evidence. Discuss how these factors might affect the conclusions drawn from the study.

Using Evidence to Support or Refute a Hypothesis

Supporting Evidence

• Definition: Data that is consistent with the predictions of the hypothesis.
• Example: If a hypothesis predicts that increased sleep will improve mental wellbeing, supporting evidence would be data showing a positive correlation between sleep duration and mental wellbeing scores.
• Note: Supporting evidence does not prove a hypothesis, but it increases confidence in its validity.

Refuting Evidence

• Definition: Data that contradicts the predictions of the hypothesis.
• Example: If a hypothesis predicts that increased screen time will decrease mental wellbeing, refuting evidence would be data showing no correlation or a positive correlation between screen time and mental wellbeing scores.
• Note: Refuting evidence suggests that the hypothesis may be incorrect and needs to be revised or rejected.

Statistical Significance

• Definition: A statistical measure of the probability that the results of a study occurred by chance.
• P-value: The probability of obtaining the observed results (or more extreme results) if the null hypothesis is true.
  • A p-value of less than 0.05 (p < 0.05) is typically considered statistically significant, meaning there is less than a 5% chance that the results occurred by chance.
• Importance: Statistical significance provides evidence that the results are not due to random variation.

Correlation vs. Causation

• Correlation: A statistical relationship between two variables.
• Causation: A relationship in which one variable directly causes a change in another variable.
• Important Note: Correlation does not equal causation. Just because two variables are correlated does not mean that one causes the other. There may be other factors involved (confounding variables).

COMMON MISTAKE: Students often confuse correlation with causation. Be careful to avoid making causal claims based solely on correlational data.

Models and Theories

Models

• Definition: Simplified representations of complex phenomena.
• Purpose: To explain and predict behavior or mental processes.
• Examples:
  • The biopsychosocial model of mental wellbeing.
  • Models of memory.

Theories

• Definition: A comprehensive explanation of a phenomenon that is supported by a large body of evidence.
• Purpose: To provide a framework for understanding and predicting behavior or mental processes.
• Examples:
  • Attachment theory.
  • Cognitive behavioral therapy (CBT).

Evaluating Models and Theories

• Consistency with Evidence: Does the model or theory accurately explain existing data?
• Predictive Validity: Does the model or theory make accurate predictions about future behavior?
• Parsimony: Is the model or theory as simple as possible while still providing an adequate explanation?
• Testability: Can the model or theory be tested through research?

Limitations of Evidence

• Sample Bias: The sample may not be representative of the population, limiting the generalizability of the findings.
• Experimenter Bias: The researcher’s expectations can influence the results of the study.
• Participant Bias: Participants may alter their behavior because they know they are being observed (Hawthorne effect) or try to give socially desirable responses.
• Measurement Error: Errors in the measurement tools or procedures can reduce the reliability and validity of the data.
• Confounding Variables: Uncontrolled variables may influence the results, making it difficult to determine the true relationship between the independent and dependent variables.
• Ethical Considerations: Ethical constraints may limit the types of research that can be conducted.

STUDY HINT: When reviewing research studies, make a checklist of potential limitations to help you critically evaluate the evidence.

The Role of Evidence in Psychological Research

Hypothesis Formation

• Evidence from previous research and observations informs the development of hypotheses.

Data Collection

• Evidence is gathered through carefully designed experiments, surveys, or observations.

Data Analysis

• Evidence is analyzed using statistical techniques to determine whether it supports or refutes the hypothesis.

Interpretation of Results

• Evidence is interpreted in the context of existing theories and models.

Revision of Theories and Models

• Evidence may lead to revisions or refinements of existing theories and models.

VCAA FOCUS: Understand how the nature of evidence (reliability, validity, objectivity) impacts the conclusions you can draw from psychological research.

Example: Evaluating Evidence Related to Specific Phobias

Hypothesis: Cognitive Behavioral Therapy (CBT) is an effective treatment for specific phobias.

Supporting Evidence:

• Quantitative Data: Studies showing a significant reduction in phobia symptoms (e.g., anxiety scores) after CBT compared to a control group.
• Qualitative Data: Interview transcripts from individuals with phobias reporting a decrease in fear and avoidance behaviors after CBT.
• Statistical Significance: P-value < 0.05, indicating that the results are unlikely to be due to chance.

Refuting Evidence:

• Quantitative Data: Studies showing no significant difference in phobia symptoms between CBT and a placebo treatment.
• Qualitative Data: Interview transcripts from individuals with phobias reporting no improvement in their symptoms after CBT.

Evaluating the Evidence:

• Reliability: Are the measures of phobia symptoms reliable (e.g., consistent scores on anxiety scales)?
• Validity: Are the measures of phobia symptoms valid (e.g., do they accurately measure the specific phobia being studied)?
• Objectivity: Were the therapists delivering CBT trained to follow a standardized protocol?
• Sample Size: Was the sample size large enough to detect a meaningful effect?
• Control of Extraneous Variables: Were other factors that could influence phobia symptoms (e.g., medication use) controlled for?

Conclusion

The nature of evidence is fundamental to psychological research. By understanding the different types of evidence, the criteria for good evidence, and how to use evidence to support or refute hypotheses, models, and theories, students can critically evaluate research findings and contribute to the advancement of psychological knowledge.

APPLICATION: When designing your own student investigation, carefully consider the type of data you will collect, how you will ensure its reliability and validity, and how you will analyze it to answer your research question.