domains / research / paper-improvement-loop / auto-paper-improvement-loop

auto-paper-improvement-loop

Skill card

combined

Both copies describe the same two-round review→fix→recompile loop, but they disagree on the reviewer model (gpt-6-astra vs gpt-5.4) and on a core loop mechanic: the wanshuiyin copy mandates a fresh reviewer thread each round (REVIEWER_BIAS_GUARD, no codex-reply) and adds style-ref/edit-whitelist options, while the brycewang copy explicitly uses codex-reply with the saved threadId in Round 2 and adds state-persistence recovery. Crowning one would hide a real alternative procedure.

Repositories

Other copies

Combined skill

---
name: auto-paper-improvement-loop
description: "Autonomously improve a generated paper via external LLM review → implement fixes → recompile, for 2 rounds. Use when user says \"改论文\", \"improve paper\", \"论文润色循环\", \"auto improve\", or wants to iteratively polish a generated paper."
argument-hint: "[paper-directory] [— style-ref: <source>] [— edit-whitelist <path>]"
allowed-tools: Bash(*), Read, Write, Edit, Grep, Glob, Skill, mcp__codex__codex, mcp__codex__codex-reply
---

# Auto Paper Improvement Loop: Review → Fix → Recompile

> 🔒 **Do not wrap this skill in `/loop`, `/schedule`, or `CronCreate`.** It already loops internally with its own round structure. Schedule the external wait that precedes it, not the improvement loop.

Autonomously improve the paper at: **$ARGUMENTS**

## Context

This skill runs **after** Workflow 3 (`/paper-plan` → `/paper-figure` → `/paper-write` → `/paper-compile`). It iterates on **paper writing quality** — fixing theoretical inconsistencies, softening overclaims, adding missing content, and improving presentation. It does **not** iterate on research (experiments, data collection, narrative rewriting).

## Constants

- **MAX_ROUNDS = 2** — Two rounds of review→fix→recompile.
- **REVIEWER_MODEL** — Model used via Codex MCP for paper review. (See Conflicts: wanshuiyin uses `gpt-6-astra`; brycewang uses `gpt-5.4`.)
- **REVIEWER_BIAS_GUARD** — When `true`, every review round uses a fresh `mcp__codex__codex` thread with no prior review context; never use `mcp__codex__codex-reply` for review rounds. When `false`, Round 2 may use `mcp__codex__codex-reply` with the saved threadId. (See Conflicts: wanshuiyin defaults to `true`; brycewang uses `codex-reply` in Round 2.)
- **REVIEW_LOG = `PAPER_IMPROVEMENT_LOG.md`** — Cumulative log of all rounds, stored in paper directory.
- **HUMAN_CHECKPOINT = false** — When `true`, pause after each round's review and present score + weaknesses to the user. The user can approve fixes, provide custom modification instructions, skip specific fixes, or stop early. When `false` (default), runs fully autonomously.
- **EDIT_WHITELIST = `null`** — Optional path to a YAML/JSON whitelist file constraining which paths and operations the fix-implementation step may touch. When `null` (default), all edits proceed unconstrained. When set via `— edit-whitelist <path>`, the loop loads the file at startup and consults it before each edit; rejected edits are logged to `PAPER_IMPROVEMENT_LOG.md`.

> 💡 Override: `/auto-paper-improvement-loop "paper/" — human checkpoint: true`

## Optional: Style reference (`— style-ref: <source>`, opt-in)

Lets the user steer **structural fixes only** during improvement toward a reference paper. **Default OFF.** Only when `— style-ref: <source>` appears in `$ARGUMENTS`, run the style-extraction helper before the loop starts. Sources accepted: local TeX dir/file, local PDF, arXiv id, http(s) URL. Overleaf URLs/IDs are rejected — clone via `/overleaf-sync setup <id>` first. Use the resulting `style_profile.md` only during the fix-implementation phase for structural nudges (section reordering, paragraph length, figure density).

## Inputs

1. **Compiled paper** — `paper/main.pdf` + LaTeX source files
2. **All section `.tex` files** — concatenated for review prompt

## State Persistence (Compact Recovery)

If the context window fills up mid-loop, write `PAPER_IMPROVEMENT_STATE.json` after each round:

```json
{
  "current_round": 1,
  "threadId": "019ce736-...",
  "last_score": 6,
  "status": "in_progress",
  "timestamp": "2026-03-13T21:00:00"
}
```

**On startup**: if `PAPER_IMPROVEMENT_STATE.json` exists with `"status": "in_progress"` AND `timestamp` is within 24 hours, read it + `PAPER_IMPROVEMENT_LOG.md` to recover context, then resume from the next round. Otherwise, start fresh.

**After each round**: overwrite the state file. **On completion**: set `"status": "completed"`.

## Workflow

### Step 0: Preserve Original

```bash
cp paper/main.pdf paper/main_round0_original.pdf
```

### Step 1: Collect Paper Text

Concatenate all section files into a single text block for the review prompt:

```bash
for f in paper/sections/*.tex; do
    echo "% === $(basename $f) ==="
    cat "$f"
done > /tmp/paper_full_text.txt
```

### Step 2: Round 1 Review

Send the full paper text to the reviewer model via `mcp__codex__codex` with `model_reasoning_effort: "xhigh"`. Prompt the model to act as a senior ML reviewer and provide: Overall Score (1-10), Summary, Strengths (ranked), Weaknesses (ranked CRITICAL > MAJOR > MINOR), specific actionable fixes for each CRITICAL/MAJOR weakness, Missing References, and Verdict (Ready for submission? Yes / Almost / No). Focus on theoretical rigor, claims vs evidence alignment, writing clarity, self-containedness, notation consistency.

Save the threadId for potential Round 2 use (if REVIEWER_BIAS_GUARD is false).

### Step 2b: Human Checkpoint (if enabled)

**Skip if `HUMAN_CHECKPOINT = false`.** Present the review results and wait for user input: approve all fixes, give custom instructions, skip specific fixes, or stop.

### Step 3: Implement Round 1 Fixes

Parse the review and implement fixes by severity: CRITICAL first, then MAJOR, then MINOR (if time permits). Common fix patterns include: rewriting assumptions to match the model, softening overclaims, adding missing metrics tables, making theorems self-contained, renaming conflicting notation, adding missing references, and explicitly framing theory-practice gaps.

If EDIT_WHITELIST is set, consult it before each edit; log rejected edits to PAPER_IMPROVEMENT_LOG.md.

### Step 4: Recompile Round 1

```bash
cd paper && latexmk -C && latexmk -pdf -interaction=nonstopmode -halt-on-error main.tex
cp main.pdf main_round1.pdf
```

Verify: 0 undefined references, 0 undefined citations.

### Step 5: Round 2 Review

If REVIEWER_BIAS_GUARD is `true`: start a fresh `mcp__codex__codex` thread with no prior review context. If `false`: use `mcp__codex__codex-reply` with the saved threadId from Round 1. Use the same review prompt structure as Round 1.

### Step 5b: Human Checkpoint (if enabled)

Same as Step 2b.

### Step 6: Implement Round 2 Fixes

Same as Step 3.

### Step 7: Recompile Round 2

```bash
cd paper && latexmk -C && latexmk -pdf -interaction=nonstopmode -halt-on-error main.tex
cp main.pdf main_round2.pdf
```

Verify: 0 undefined references, 0 undefined citations.

### Step 8: Finalize

Set `PAPER_IMPROVEMENT_STATE.json` status to `"completed"`. Append final summary to `PAPER_IMPROVEMENT_LOG.md`.

## Conflicts

- **Reviewer model**: wanshuiyin/Auto-claude-code-research-in-sleep specifies `gpt-6-astra`; brycewang-stanford/Auto-Empirical-Research-Skills specifies `gpt-5.4`. Choose the model available in your environment.
- **Round 2 review mechanism**: wanshuiyin enforces a fresh thread each round (REVIEWER_BIAS_GUARD=true, no codex-reply) based on empirical evidence that codex-reply inflated scores; brycewang uses codex-reply with the saved threadId for Round 2. If you adopt the fresh-thread approach, set REVIEWER_BIAS_GUARD=true and never call codex-reply for review rounds. If you adopt the reply approach, set REVIEWER_BIAS_GUARD=false and use codex-reply in Round 2.