RuhrohSee what coding agents actually deliver.

Run realistic software tasks, inspect the finished work and the agent journey, compare repeated outcomes, and improve what happens next.

Run The Example

Inspect The Evidence

npm

Loop engineering

Build better coding-agent loops from evidence

Ruhroh makes agent improvement repeatable. Give agents realistic work, preserve what happened, judge the finished outcome, compare like with like, and use the findings to decide what to change next.

1
Run
Execute the same realistic task across agents, prompts, or configurations.
2
Inspect
Review the finished workspace, journey, transcripts, and evaluator evidence.
3
Compare
Check repeated outcomes, failure patterns, uncertainty, and cohort consistency.
4
Improve
Change the agent, prompt, connector, task, or reviewer and run the next cohort.

What Ruhroh reveals

A 50% pass rate that should not be published

The checked-in sample runs one newsletter task twice. The number is easy to calculate. The evidence explains why it is not yet a conclusion.

PassedRun 1

Delivered the required newsletter with three stories.

Goal mismatchRun 2

Created a page, but included only one story.

BlockedClaim status

Only 2 of 5 required runs exist, an evaluator warning remains, and one run needs review.

Open the comparison Inspect the review queue See the blockers

Questions Ruhroh Helps You Answer

Did the agent deliver?

Judge the finished software outcome against user-facing success criteria.

Inspect one run

Why did it fail?

Follow the implementation journey, commands, transcripts, reviewer evidence, and final workspace.

Follow the evidence

Is the comparison fair?

Lock task versions, plan samples, preserve cohort metadata, and surface mismatches.

Review the methodology

Can the reviewer be trusted?

Calibrate known outcomes and flag missing evidence, weak judgment, or disagreement.

Build a reviewer

What should improve next?

Compare failure modes, duration, iterations, cost, and tokens across repeated runs.

Explore the compare report

Can others verify the result?

Package the claim, reports, hashes, review queue, and source evidence together.

Publish or block a claim

Run The Credential-Free Example

bash

pnpm add -D @kestrel-agents/ruhroh
pnpm exec ruhroh init
export RUHROH_RUN_AGENT_COMMAND="$PWD/ruhroh/adapters/fixture-newsletter/run.sh"
export RUHROH_EVAL_COMMAND="$PWD/ruhroh/evaluators/fixture-newsletter/run.sh"
pnpm exec ruhroh first-run

first-run checks the local task, suite, agent command, reviewer command, and Harbor installation, then prints the exact next command. It does not modify the project or mistake a dry-run preview for completed evidence.

Choose Your Path

Improve an agentAdd Ruhroh to a real project, connect an agent, and inspect where its loop breaks down.Build an evaluationAuthor a task and reviewer, freeze a suite, and collect comparable runs.Publish evidenceValidate the result, resolve review items, and create a portable evidence packet.

For the product model, read Core Concepts. For generated output, open the Report Gallery. For every command and option, use the CLI Reference.