Start with the work, then choose the abstraction
Complex tasks with clear acceptance
Refactors, migrations, and protocol implementations. The destination is fairly clear even when the execution path is not.
Preserve: acceptance gaps, versions, repair evidenceMeasure: completion rate and cost
Open-ended exploration
Research, algorithm experiments, and system optimization. New evidence continuously changes the next route.
Preserve: hypotheses, counterevidence, candidate directionsMeasure: useful discoveries and validation
Digital workers for continuous delivery
Take an issue, fix it, follow review, handle new feedback, and remain accountable for the result.
Preserve: responsibility, external state, waiting conditionsMeasure: delivery quality and human effort
These categories can nest. An agent maintaining a repository over time may first explore a performance problem, then complete a fix with clear acceptance, and finally follow the pull request through review. The first two categories mainly describe how work is solved; the third adds an ongoing responsibility and a stream of newly arriving work.
What should persist is the work—and the reasoning behind its decisions.
01 / Finish a complex task
“Implement a decoder that conforms to the specification” sounds definite. Yet after the visible examples pass, edge inputs, memory safety, or compatibility may still be missing. Start a new session, and previously confirmed failure conditions may disappear from context.
LoopX keeps the goal and its acceptance gaps continuous across turns, then carries new evidence into the next decision. The model implements and judges; domain validators check the result; the control plane records which results were accepted and which work remains.
The best-fitting tasks usually span repeated validation, waiting, or handoffs. Existing agents may already be enough for small work that can be accepted in one session; an added control plane must justify its overhead.
Benchmarks: start with LHTB, then three supporting signals
Start with recovery and regression across long-horizon domains, then use three software-engineering studies to examine continuation, delivery, and validation. The four studies differ in tasks, model settings, and measurement rules. Each needs to be read on its own terms; the current evidence does not establish a universal LoopX gain.
LHTB: what does LoopX accomplish beyond Plain and native Goal?
GPT-5.6 Sol / max · 46 matched tasks · one effective run per task-arm cell
What LHTB tests: Long-Horizon Terminal-Bench asks an agent to complete hundreds of dependent terminal actions in a stateful container. Its 46 tasks cover nine categories: software and reverse engineering; scientific computing; earth, climate and energy; multimodal work; research reproduction; systems, performance and security; games; APEX professional workflows; and logic puzzles. Examples include framework migration, paper reproduction, investment-banking deliverables and playing 2048. Categories and representative tasks →
How it scores: hidden verifiers grade final artifacts or replayable outcomes on a 0–1 reward scale. This study counts ≥ 0.95 as solved, reporting average progress and full acceptance separately.
| Execution | Mean reward | Solved / 46 |
|---|---|---|
| Plain | 0.4218 | 7 |
| Native Goal | 0.4475 | 4 |
| LoopX Heartbeat (1.0.3) | 0.4948 | 7 |
Observation: against Plain, LoopX gains 0.0731 mean reward (17.3%), with 17 wins, 13 ties and 16 losses, and the same solve count. Against native Goal, it gains 0.0473 (10.6%), with 23 wins, 13 ties and 10 losses, and 7 solves versus 4. Counts compare published unrounded scores strictly; higher mean reward does not imply better results on every task.
Where gains cluster: in post-hoc groups based on task demands, research/modeling (4 tasks) gains +0.2802 / +0.2179 mean reward over Plain / Goal; logic puzzles (4 tasks) gain +0.2739 / +0.1374. Research stays positive without Tabular. Sokoban retains solved levels, Rush Hour requires a route ledger and replay checks, and Tabular requires experiments and validation—promising conditions to test.
Where the advantage is unclear: multimodal analysis (6 tasks) yields −0.0034 / +0.0058; science and simulation (7 tasks) yield +0.0161 / +0.0099. Games are mixed: 2048 improves, but Snake trails Plain. APEX law trails Goal, and DuckDB scores 0 versus about 0.770. Continuation cannot substitute for perception, domain judgment or correctness checks.
Insight: prioritize tasks where the next step can be checked and useful progress retained. The median paired delta across all tasks is 0 against Plain and about 0.0003 against Goal: mean gains are concentrated. Groups contain only 4–7 tasks and are not an official task mapping. Prompt structure suggests explanations, but does not establish the mechanism behind a trial’s score. Nine groups: counts, wins/ties/losses, sensitivity and task instructions →
Boundary: LoopX here is the study’s 1.0.3 fresh-exec Heartbeat. Execution settings differ, replacement trials include longer limits, and there are no repeated seeds. LoopX records $551.73, above the Plain / Goal estimates of $212.97 / $344.25. Accounting also differs; equal-budget efficiency gains are not established.
Full study: two baselines, gains and losses, 46 tasks, and historical context →
SWE-Marathon: continuation must close real gaps
GPT-5.6 Sol / high · 15 matched tasks · 3 retained modes, one run per cell · timeout factor 0.3
Observation: native Goal completed 4/15; Heartbeat completed 5/15. Total cost for the two groups rose from $533 to $830, about 56%.
Insight: in the zstd case, continuation addressed acceptance beyond the visible tests; in the Excel case, repeated continuations still did not finish. The questions worth pursuing are which gap a next run closes and whether that gain justifies the added cost.
Boundary: each cell has one run and several mechanisms change together. There are both useful cases and ineffective continuation; stable gains or cost advantages have not been established.
Full study: results, cost, and positive and negative cases →
DeepSWE × Sol: decompose the completion difference
Script default GPT-5.6 Sol / xhigh · 113 tasks · historical best-valid aggregation
Observation: bare Codex, native Goal, and Heartbeat completed 54, 60, and 70 tasks respectively. Heartbeat completed ten more than Goal, but its default time window was also longer.
Insight: the source separates “continue execution” from “deliver a valid result”: even when a worktree contains code, the collector may still receive an empty patch. Recovery, patch delivery, and independent acceptance should be examined separately before attributing their contribution to the completion difference.
Boundary: this is the best valid result per task, not a single-run pass rate; budgets differ, and attempts and cost are not fully disclosed. Unreverified merged scores cannot be treated as a net gain at equal budget.
Full study (Chinese): historical results, recovery, and delivery mechanics →
DeepSWE × V4 Flash max: let counterexamples change the implementation
DeepSeek V4 Flash / max + Codex · frozen 113 tasks · Goal and LoopX compared under the same hint condition
Observation: LoopX had about two percentage points higher feature coverage per task on average. In a post-hoc long-duration slice where both groups had hints, completions rose from 12/29 to 14/29 while cumulative duration fell 16.4%.
Insight: the selected cases differ in whether failure counterexamples are retained and whether external contracts can overturn implementation assumptions. Validation matters when it changes the fix and triggers revalidation—not merely when it adds more checks.
Boundary: coverage is not success rate; the long-duration slice is post-hoc, and elapsed time includes successful and failed runs. Local findings cannot be generalized into an overall gain or equal-quality speedup.
Full study: requirement coverage, behavior cases, and the long-duration slice →
Recover progress, preserve working results, and check whether continuation closes acceptance gaps.
The four studies offer local positive observations while exposing regressions, cost, and attribution problems. Matched budgets and repeated experiments should examine recovery, rollback, delivery, and counterexample-driven repair separately.
Withdrawn SSH Goal and Codex CLI scores and conclusions from SWE-Marathon and DeepSWE × Sol are not used in these comparisons.
02 / Find a path through open-ended exploration
“Find a better algorithm” does not come with a fully specified task chain. Real progress may validate a hypothesis—or eliminate a promising-looking route. If only successful conclusions survive, the next run can easily repeat ideas that were already disproved.
Explore provides an optional evidence graph and bounded branch planning: nodes represent questions and findings, while relationships express supports, refutes, and leads_to. Planning suggestions still pass through normal execution boundaries; the graph neither launches workers nor grants spending authority.
Auto Research organizes this pattern as research work: select a topic, propose hypotheses, run experiments, evaluate independently, and produce a report. Existing protocols and commands provide a foundation, while the public showcase still contains blueprints and items awaiting validation. Real effectiveness must be demonstrated on specific tasks.
Place RSI here, but keep the claim precise
When the object of improvement is the agent’s own tools, strategies, or harness, the work becomes self-improvement research. Changing its own code only creates a candidate; sustained improvement still requires independent evaluation, cross-task generalization, regression checks, and rollback.
Applying past experience to a later decision, automatically generating experiments, and modifying the harness are different levels. We can discuss an experimental path toward recursive self-improvement (RSI) here; we cannot equate “automatic iteration” with open-ended self-improvement already achieved.
03 / Own continuous delivery
For a pull-request or issue-fix digital worker, the job begins by deciding whether the issue is worth fixing. After delivery, CI, review feedback, branch changes, and the merge result still remain. New tasks keep arriving, and external facts keep changing.
Responsibility does not end when the patch is generated.
This synthetic scenario starts after a fix has been submitted while CI and review are still running. Choose an external state to see how the next step changes.
- External fact
- GitHub reports that checks are pending for the current revision.
- Domain state
- Record the pull request, revision, and check state; an unchanged observation does not create progress.
- Capability proposal
- Preserve the monitor and recovery condition, then wait for the result.
- Control-plane check
- Admit work by authority, budget, and eligibility; work outside the wait scope can still proceed.
- Meaning for people
- Stay quiet when nothing material changed, so polling does not become nagging.
This interactive diagram is a design explanation. It is not connected to a real repository and performs no GitHub operations.
Here, a “digital worker” means continuing responsibility, boundaries, memory, and a feedback loop. Its value should be judged by accepted fixes, reopenings and regressions, handling latency, delivery cost, and how often a person must repeatedly supervise it. Pull-request count and uptime are only process signals.
The richer the domain, the clearer the division of responsibility must be
Authority and common lifecycleWhich goals and work items are valid, who may act, which authority and budgets apply, and how work is handed off, paused, and committed.
Domain continuityWhether this issue is actionable, which revision the pull request names, where CI and review stand, and which conclusions remain valid.
Outcome contract and judgmentTranslate domain facts into a verifiable next step: repair, wait, ask a person, or close out—and define acceptance for the scenario.
External I/O and executionCall repositories, tests, and tools; perform authorized actions; and read back the result. GitHub still owns the external facts about code, CI, and review.
For checks failing, a domain capability can propose a successor task to repair CI. The fact that a repair is needed does not itself grant repository write or merge authority. In an experiment, check state becomes metrics and held-out conclusions; the common claim, authority, and recovery rules should remain consistent.
That is the value of a capability: it captures recurring judgments and outcome contracts for a scenario without forcing every business stage into the Kernel.
Evolution: add one verifiable capability at a time
- One-shot execution → recoverable long-running taskRecover the goal, evidence, and next step across sessions and process interruptions. First prove one thing can be finished reliably.
- Generic task → domain delivery and explorationUse issue-fix and research work for real outcomes, preserve domain state, and define what makes a result valuable.
- One agent → small-team collaborationFirst have two or three workers complete two rounds of dependent delivery: receive artifacts, accept independently, absorb corrections, and return results.
- Local collaboration → continuous work across hostsValidate shared state, authority revocation, stale-worker isolation, network failure, and shared budgets. Successful registration is not proof of collaboration.
- Sustainable operation → measurable improvementUse outcome feedback to improve selection, memory, and strategy; prove gains through controlled experiments, held-out validation, and rollback. Qualify scaling separately.
These are a build order and validation targets, not a completion checklist. Foundations, partial implementations, blueprints, and end-to-end qualification should be read separately.
I want what we hand to an agent to evolve from a single instruction into a work agreement it can keep fulfilling.
That agreement should remain legible and editable to people, actionable to agents, and auditable through evidence after failure.
Public sources and further reading
The technical content is derived only from public repository material. Source and data links are pinned to the revisions read for this article; historical experiments retain their own versions. This page introduces no new experimental results.
- LHTB: compared with Plain and native Goal; public aggregates for 46 tasks; setup and evidence boundary; mechanisms and cases reported in the brief; official benchmark. See the study for contributor attribution.
- SWE-Marathon: continuous self-verification; setup, positive and negative cases, and limitations; public aggregate data. See the study for contributors and case provenance.
- DeepSWE × Sol: from continued execution to valid delivery (Chinese). The standalone brief covers historical results over 113 tasks, mechanism diagrams, and pinned primary sources; research archive contribution: @gwh6669999, #4502.
- DeepSWE × V4 Flash max: from hints to behavior; disclosure boundary; charts, cases, and metrics at the pinned revision.
- Explore: evidence graphs, planning, and authority boundaries.
- Auto Research: public blueprint and command path.
- PR / Issue Fix: how State Kernel and domain state work together (Chinese).
- Overall roadmap: product goals and independent acceptance milestones.
- From one-shot agents to long-horizon control; LoopX: Native Kanban for long-running agents.