Reasoning Verifier
Verify reasoning using the Reverse Chain-of-Thought (RCoT) methodology to detect and correct errors in LLM-generated solutions by systematically comparing what the problem stated versus what the solution assumed.
When to Use
- After complex multi-step reasoning or analysis tasks
- When solution correctness is critical
- To catch overlooked conditions from the original problem
- To identify hallucinated assumptions not present in the original problem
- Before finalizing recommendations or decisions based on LLM analysis
RCoT Methodology
Follow these steps in order:
Step 1: Problem Decomposition
Extract all conditions from the original problem:
- Read the original problem statement carefully
- List every explicit condition, constraint, and requirement
- Note any implicit conditions that can be logically inferred
- Number each condition for reference
Format:
## Original Conditions
1. [Condition from original problem]
2. [Condition from original problem]
3. [Condition from original problem]
...Step 2: Solution Reconstruction
Analyze the provided solution and reconstruct what problem it appears to solve:
- Read through the solution's reasoning steps
- Identify every assumption the solution made
- List conditions the solution used in its reasoning
- Note the final answer/conclusion
Format:
## Reconstructed Conditions (from solution)
1. [Condition the solution assumed]
2. [Condition the solution assumed]
3. [Condition the solution assumed]
...
## Solution's Answer: [final answer]Step 3: Condition Comparison
Compare the two condition lists to find discrepancies:
Overlooked Conditions: Present in original but NOT used in solution
- These are facts the solution ignored
- May cause incorrect answers if they affected the logic
Hallucinated Conditions: Present in solution but NOT in original
- These are assumptions the solution invented
- Invalid if they cannot be logically deduced from original conditions
For each candidate condition, ask: "Can this be logically deduced from the other condition list?"
Format:
## Comparison Results
### Overlooked Conditions
- [Condition X] from original was not addressed
- Impact: [How this affects the solution]
### Hallucinated Conditions
- [Condition Y] was assumed but not stated
- Validity: [Can this be deduced? YES/NO]
- Impact: [How this affects the solution]Step 4: Error Classification
Classify the severity of each discrepancy:
- Critical: Changes the final answer
- Major: Affects intermediate reasoning significantly
- Minor: Technical oversight with no impact on answer
- None: Solution is correct
Step 5: Revision Prompt Generation
If errors are found, generate targeted correction prompts:
For overlooked conditions:
You have ignored some real conditions:
1. [overlooked condition]
Here are detailed reasons:
1. [Why this condition matters and how it affects the answer]For hallucinated conditions:
You have assumed conditions not in the problem:
1. [hallucinated condition]
This cannot be logically derived because:
1. [Why this assumption is invalid]Step 6: Revised Solution
If errors were found, provide the corrected solution:
- Acknowledge the specific mistakes
- Incorporate all overlooked conditions
- Remove invalid hallucinated assumptions
- Show corrected reasoning step-by-step
- Provide the revised answer
Output Format
# RCoT Verification Report
## Problem Summary
[Brief description of the problem being verified]
## Original Conditions
1. [condition]
2. [condition]
...
## Reconstructed Conditions (from solution)
1. [condition]
2. [condition]
...
## Comparison Results
### Overlooked Conditions
[List or "None found"]
### Hallucinated Conditions
[List or "None found"]
## Verdict: [CORRECT / NEEDS REVISION]
## Severity: [Critical / Major / Minor / None]
---
[If NEEDS REVISION:]
## Revision Prompt
[Generated prompt to correct the solution]
## Revised Solution
[Corrected answer with proper reasoning]Example
Original Problem: "Mary has 40 window ledges. She has 2 potted plants on each ledge. Yesterday, she received 18 new potted plants from her favorite nursery. She decided to give away 1 potted plant from each ledge. How many potted plants will Mary remain with?"
Original Solution: "Mary has 2 x 40 = 80 plants. After giving away 1 from each ledge, she has 80 - 40 = 40 plants."
RCoT Analysis:
- Overlooked: "She received 18 new potted plants" - this condition was completely ignored
- The solution is INCORRECT
Revised Solution: "Mary starts with 2 x 40 = 80 plants. She receives 18 new plants, giving her 80 + 18 = 98 plants. After giving away 1 from each of 40 ledges, she has 98 - 40 = 58 plants."