autoresearch-dspy implements a generalized AutoResearch ratchet loop using DSPy 3.1.3 and dspy.RLM. It applies Karpathy’s autonomous experiment pattern — propose, execute, evaluate, keep or revert — generalized beyond ML to any optimizable system. Includes confidence-aware classification scoring via the GEPA ConfidenceAdapter.
When to use
- Autonomous optimization loops
- Iterative experiments that keep only measured improvements
- RLM recursive context exploration
- Overnight system optimization
- The Karpathy Loop pattern
- Classification prompt optimization with confidence scoring
Trigger phrases: autoresearch, ratchet loop, experiment loop, optimize overnight, autonomous research, Karpathy loop, iterative optimization, self-improving code, experiment automation, confidence adapter, classification optimization.
Core concept
The fundamental ratchet loop:
- Propose a change.
- Execute the experiment.
- Evaluate against the chosen metric.
- Keep if it measurably improves; otherwise revert.
Only measured improvements survive, so progress is monotonic by construction.
Install
The plugin ships under plugins/autoresearch-dspy/ with a single skill at skills/autoresearch-dspy/SKILL.md plus supporting scripts. Last modified on May 19, 2026
