ScholaRAG
This chapter corresponds to code in the researcherRAG repository
View CLI Source
🧪

Hypothesis Validation

Test your research hypotheses by systematically identifying supporting and refuting evidence from your literature database. Perfect for building rigorous arguments backed by empirical data.

📋 When to Use This Prompt

  • ✓ Testing specific research hypotheses or claims
  • ✓ Building evidence-based arguments for papers
  • ✓ Identifying contradictory findings
  • ✓ Validating theoretical predictions

Prompt Template

Copy this prompt and replace YOUR_HYPOTHESIS_HERE with your hypothesis:

My hypothesis: "YOUR_HYPOTHESIS_HERE"

Please:
1. List evidence SUPPORTING this hypothesis
2. List evidence REFUTING or contradicting this hypothesis
3. Provide reasoning for each piece of evidence
4. Include effect sizes, statistical values, and page numbers

Example Usage

Here's how you might customize this prompt:

My hypothesis: "AI chatbots improve speaking proficiency more effectively than traditional language labs"

Please:
1. List evidence SUPPORTING this hypothesis
2. List evidence REFUTING or contradicting this hypothesis
3. Provide reasoning for each piece of evidence
4. Include effect sizes, statistical values, and page numbers

Expected Output Structure

## Hypothesis Validation: AI Chatbots vs Traditional Language Labs

### Supporting Evidence (5 papers)

**Strong Support**
1. **[Smith et al., 2023, p.145]** - RCT with 120 students
   - AI chatbot group: +12.3 points on speaking test (Cohen's d = 0.87)
   - Language lab group: +6.1 points (Cohen's d = 0.42)
   - Difference significant (p < .001)
   - Reasoning: Personalized feedback and adaptive difficulty

2. **[Lee & Kim, 2022, p.67]** - 8-week intervention study
   - AI group showed 2.3x more improvement (ES = 0.91)
   - Students practiced 40% more with AI (engagement factor)

**Moderate Support**
3. **[Martinez, 2024, p.201]** - Qualitative study
   - Students reported lower anxiety with AI (no human judgment)
   - More willingness to practice (self-reported)
   - No quantitative comparison to traditional methods

### Refuting/Contradictory Evidence (3 papers)

**Direct Contradiction**
1. **[Johnson & Davis, 2021, p.89]** - No significant difference
   - AI chatbot: +7.2 points (d = 0.54)
   - Language lab: +6.8 points (d = 0.51)
   - Difference NOT significant (p = .23)
   - Reasoning: Both groups received equal practice time

**Partial Contradiction**
2. **[Park et al., 2023, p.134]** - Mixed results by proficiency level
   - Beginners: AI better (+8.5 vs +5.1, p < .05)
   - Advanced: Language lab better (+9.2 vs +7.8, p < .05)
   - Reasoning: Advanced learners benefit from peer interaction

**Methodological Concerns**
3. **[Chen, 2022, p.178]** - Questions measurement validity
   - Current speaking tests may not capture AI-trained skills
   - AI students excel in fluency but not pragmatic competence

### Summary
- **Balance of Evidence**: 5 supporting vs 3 refuting (63% support)
- **Effect Size Range**: d = 0.42 to 0.91 (moderate to large)
- **Confidence Level**: Moderate (heterogeneous findings)
- **Key Moderators**: Student proficiency level, measurement tools, practice time

Customization Examples

Multiple Hypotheses
I have three related hypotheses:
1. "AI chatbots improve speaking proficiency"
2. "AI chatbots reduce language anxiety"
3. "AI chatbots increase practice frequency"

For EACH hypothesis:
1. List supporting evidence
2. List refuting evidence
3. Rate strength of evidence (Strong/Moderate/Weak)
4. Include effect sizes and citations
Focus on Specific Study Types
My hypothesis: "[your hypothesis]"

Consider ONLY experimental studies (RCT or quasi-experimental).

For each supporting/refuting paper:
1. Specify experimental design
2. Report control condition
3. Note sample size and power
4. Include effect sizes with 95% CI
Meta-Analysis Format
My hypothesis: "[your hypothesis]"

Create a table for meta-analysis:
| Study | Year | N | Design | Effect Size | 95% CI | p-value | Direction |
(For each paper reporting quantitative outcomes)

Then summarize:
- Weighted average effect size
- Heterogeneity (I²)
- Publication bias indicators

Common Follow-up Questions

  • Q: "Why do Smith (2023) and Johnson (2021) report conflicting results?"
  • Q: "Are there moderating variables that explain the contradictions?"
  • Q: "Which study has the strongest methodology for testing this hypothesis?"
  • Q: "What sample sizes were used in supporting vs refuting studies?"
  • Q: "Show me the exact quotes from papers refuting my hypothesis"

Pro Tips

⚖️ Seek Disconfirmation

Actively look for refuting evidence. Confirmation bias is real—don't ignore contradictory findings.

📊 Compare Effect Sizes

Don't just count papers. Weight by effect size and study quality. One rigorous RCT may outweigh three weak correlational studies.

🔍 Check Moderators

If results are mixed, ask "What factors differ between supporting and refuting studies?" (sample, context, measurement)

✅ Verify Claims

Read the original papers for critical claims. AI may misinterpret nuanced findings. Always spot-check.

⚡

Contradiction Detection

Deep dive into conflicting results after identifying contradictions

📊

Statistical Extraction

Extract detailed quantitative data for meta-analysis

← Back to Prompt Library
Next
Introduction