shanraisshan/ralph-wiggum-self-evolving-loop

Ralph Wiggum Self Evolving Loop - Novel LLM-Breaking Questions

a ralph wiggum self evolving loop that generates questions designed to break modern LLMs. The goal is to find simple, generic questions that any human can easily answer, but cause AI models to disagree.

The Goal

Create questions like the famous "strawberry problem" — simple one-liners that:

• A 5-year-old can answer correctly
• Frontier LLMs (GPT-5, Claude, Gemini) consistently fail or disagree on
• Achieve < 10% consensus among 5 independent AI researchers

The Winning Question

After 103 research iterations, we found it — a simple riddle that achieved 0% consensus among 5 independent LLM researchers:

"I follow you everywhere but I get LONGER the closer you get to the sun. What am I?"

Why it breaks LLMs:

• LLMs pattern-match to the classic "shadow" riddle
• But shadows get shorter when closer to light sources
• The correct answer: Your trail/path/journey — the closer you travel toward the sun, the longer your trail becomes

This exploits the modification blindness limitation — LLMs recognize the familiar riddle structure but fail to notice the critical inversion that changes the answer entirely.

⚠️ WARNING

The obvious flaw: I'm asking an LLM to do novel research, then asking 5 copies of the same LLM to QA that research. It's pure Ralph Wiggum energy - "I'm helping!" They share the same knowledge cutoff, same

Progress So Far

104 questions tested — LLM-breaking question found at #103, continuing research.

Notable Results

Previously Rejected Questions (0% score but rejected by Owner)

How It Works

Run the infinite research loop:

Method 1: Claude Code Skill

/execute-workflow

Runs a single iteration of the workflow inside Claude Code.

Method 2: Ralph Wiggum Loop

./ralph.sh 100

Uses the Ralph Wiggum bash loop pattern to run the workflow autonomously for N iterations. The script:

• Spawns fresh Claude instances for each iteration
• Automatically detects <promise>COMPLETE</promise> to stop when a novel question is found
• Continues until max iterations or success

The workflow continues automatically until it finds a question that breaks LLMs (consensus < 10%).

Workflow Diagram

Workflow Steps

MCP Tools Integration

The opus-researcher agent uses these MCP tools:

Agents Summary

State Files

research-workflow-state.yaml

Tracks workflow execution state:

workflow:
  status: idle | running | error
  current_phase: generate | verify | evaluate | update_agent | commit | report
  current_iteration: <number>
  last_completed_phase: <phase_name>

research-questions.yaml

Stores all questions tested:

questions:
  - id: 103
    question: "I follow you everywhere but I get LONGER the closer you get to the sun. What am I?"
    answer: "Your trail/footprints/path (NOT a shadow - shadows get shorter near light sources)"
    status: completed
    folder: research103
    score: 0
    research_status: complete
    started_at: "02/02/2026 05:50 AM PST"
    completed_at: "02/02/2026 05:56 AM PST"

research-status.json

Quick-access status:

{
  "status": "complete"
}

Why This Works

LLMs have fundamental limitations:

Result: Found the next "strawberry problem" — Research #103's riddle exposes modification blindness in frontier LLMs.

Questions

All 104 questions tested in this research, with their consensus scores and rejection status.

Other Repos

Claude Code Best Practice

Claude Code Hooks

Codex CLI Best Practice

Codex CLI Hooks

Gemini CLI Best Practice

Gemini CLI Hooks

Models going HIGH

Flagship models tripping over the exact failure modes this repo hunts, captured live in Claude Code.

Opus 4.7

Tokenization-blind letter counting and reflexive negation.

• how many p in the word strawperrry
• is 2027 next year?