domains / research / research-idea-generation / idea-creator

idea-creator

Skill card

combined

The two copies share the same core pipeline (landscape survey, LLM brainstorm, filtering, deep validation, pilots) and identical pilot constants, but they disagree on reviewer model, reviewer backend, and Phase 0 wiki handling. The first copy is the more complete, current procedure (manual-review backend, cross-model invariant, wiki loading, threat scanner, query-pack scan), while the second copy contains a concrete Phase 1 landscape-survey procedure and a detailed Phase 2 brainstorm prompt that the first copy truncates or omits. Crowning one would hide the other's material steps and the conflicting reviewer configuration.

Repositories

Other copies

Combined skill

---
name: idea-creator
description: Generate and rank research ideas given a broad direction. Use when user says "找idea", "brainstorm ideas", "generate research ideas", "what can we work on", or wants to explore a research area for publishable directions.
argument-hint: "[research-direction]"
allowed-tools: Bash(*), Read, Write, Grep, Glob, WebSearch, WebFetch, Agent, Skill, mcp__codex__codex, mcp__codex__codex-reply, mcp__manual_review__review, mcp__manual_review__review_reply
---

# Research Idea Creator

Generate publishable research ideas for: $ARGUMENTS

## Overview

Given a broad research direction from the user, systematically generate, validate, and rank concrete research ideas. Phase 1's landscape survey is inline (WebSearch). Phases 4-5 invoke /novelty-check, /run-experiment, and /monitor-experiment for validation and pilots. For the full sub-skill pipeline, run /idea-discovery (Workflow 1), which orchestrates this skill.

## Constants

- **PILOT_MAX_HOURS = 2** — Skip any pilot estimated to take > 2 hours per GPU. Flag as "needs manual pilot".
- **PILOT_TIMEOUT_HOURS = 3** — Hard timeout: kill pilots exceeding 3 hours. Collect partial results if available.
- **MAX_PILOT_IDEAS = 3** — Pilot at most 3 ideas in parallel. Additional ideas are validated on paper only.
- **MAX_TOTAL_GPU_HOURS = 8** — Total GPU budget for all pilots combined.
- **REVIEWER_MODEL = `gpt-6-astra`** — Default model for the Codex backend. Must be an OpenAI model (e.g., `gpt-6-astra`, `o3`, `gpt-4o`). Manual backend uses a model the user chooses, but it must be a non-Claude model ARIS can classify (OpenAI, Google, DeepSeek, Moonshot/Kimi, Qwen) — the executor is Claude, so pasting into any Claude product makes Claude judge Claude and voids the cross-model invariant (see `shared-references/reviewer-routing.md`).
- **REVIEWER_BACKEND = `codex`** — Default: Codex MCP (xhigh). Override with `— reviewer: oracle-pro` for Oracle MCP, or `— reviewer: manual` for Manual Review MCP. If manual-review MCP is unavailable, stop and print the install command; do not fall back to Codex. See `shared-references/reviewer-routing.md`.
- **OUTPUT_DIR = `idea-stage/`** — All idea-stage outputs go here. Create the directory if it doesn't exist.

> 💡 Override via argument, e.g., `/idea-creator "topic" — pilot budget: 4h per idea, 20h total`.

## Reviewer Calling Convention

When calling the reviewer for idea evaluation, branch on REVIEWER_BACKEND:

**If REVIEWER_BACKEND = `codex`:**
  Use `mcp__codex__codex` for new review threads.
  Use `mcp__codex__codex-reply` for follow-up rounds (reuse threadId).

**If REVIEWER_BACKEND = `manual`:**
  Use `mcp__manual_review__review` for new review threads with:
    prompt: [exact same prompt that would go to Codex]
    config: {"model_reasoning_effort": "xhigh", "executor_model": "<actual executor model>", "require_reviewer_model": true}
  Save the returned `threadId`.
  Use `mcp__manual_review__review_reply` for follow-up rounds with:
    threadId: [saved manual-review threadId]
    prompt: [follow-up prompt]
    config: {"model_reasoning_effort": "xhigh", "executor_model": "<actual executor model>", "require_reviewer_model": true}

Content fidelity: the manual reviewer should see the same substantive bundle content Codex would read. If the manual UI supports file upload / attachment, reuse the same bundle file; otherwise paste the bundle contents inline because remote web UIs cannot read your local filesystem paths. Review tracing applies equally to both backends.

## Workflow

### Phase 0: Load Research Wiki (if active)

A verdict-bearing manual response MUST begin with `Reviewer-Model: <exact-model-id>` — pass the model THIS session is actually running as in `executor_model`. Missing, unknown, or same-family identity cannot acquit; emit `REVIEW_UNAVAILABLE` rather than guessing. If the executor model cannot be named, manual review's cross-family claim is unprovable — say so in the report instead of asserting it.

**Skip this phase entirely if `research-wiki/` does not exist.**

If `research-wiki/` exists, resolve the canonical helper using the shared resolution chain (see `../research-wiki/SKILL.md` for the contract):

```bash
cd "$(git rev-parse --show-toplevel 2>/dev/null || pwd)" || exit 1
ARIS_REPO="${ARIS_REPO:-}"
ARIS_HOME="${HOME:-}"
if [ -z "${ARIS_REPO:-}" ] && [ -f .aris/installed-skills.txt ]; then
  ARIS_REPO=$(awk -F'\t' '$1=="repo_root"{print $2; exit}' .aris/installed-skills.txt 2>/dev/null) || true
fi
if [ -z "${ARIS_REPO:-}" ] && [ -n "$ARIS_HOME" ] && [ -f "$ARIS_HOME/.aris/repo" ]; then
  ARIS_REPO=$(cat "$ARIS_HOME/.aris/repo" 2>/dev/null) || true
fi
WIKI_SCRIPT=".aris/tools/research_wiki.py"
[ -f "$WIKI_SCRIPT" ] || WIKI_SCRIPT="tools/research_wiki.py"
[ -f "$WIKI_SCRIPT" ] || { [ -n "${ARIS_REPO:-}" ] && WIKI_SCRIPT="$ARIS_REPO/tools/research_wiki.py"; }
[ -f "$WIKI_SCRIPT" ] || {
  echo "WARN: research_wiki.py not found at .aris/tools/, tools/, $ARIS_REPO/tools/, or via ~/.aris/repo." >&2
  echo "      The idea-creation primary output (idea ranking) will still be produced." >&2
  echo "      Wiki writes and query_pack rebuilds will be skipped; a fresh cached pack may still be loaded through the scanner." >&2
  echo "      Fix: rerun 'bash tools/install_aris.sh' or 'smart_update.sh' (refreshes ~/.aris/repo), export ARIS_REPO, or 'cp <ARIS-repo>/tools/research_wiki.py tools/'." >&2
  WIKI_SCRIPT=""
}

THREAT_SCANNER=".aris/tools/threat_scan.py"
[ -f "$THREAT_SCANNER" ] || THREAT_SCANNER="tools/threat_scan.py"
[ -f "$THREAT_SCANNER" ] || { [ -n "${ARIS_REPO:-}" ] && THREAT_SCANNER="$ARIS_REPO/tools/threat_scan.py"; }
[ -f "$THREAT_SCANNER" ] || THREAT_SCANNER=""

# ARIS_QUERY_PACK_SCAN_START -- exercised by
# tests/test_idea_creator_query_pack_scan.py; keep both skill mirrors identical.
aris_scan_query_pack() {
  lo
```

### Phase 1: Landscape Survey (5-10 min)

Map the research area to understand what exists and where the gaps are.

1. **Scan local paper library first**: Check `papers/` and `literature/` in the project directory for existing PDFs. Read first 3 pages of relevant papers to build a baseline understanding before searching online. This avoids re-discovering what the user already knows.

2. **Search recent literature** using WebSearch:
   - Top venues in the last 2 years (NeurIPS, ICML, ICLR, ACL, EMNLP, etc.)
   - Recent arXiv preprints (last 6 months)
   - Use 5+ different query formulations
   - Read abstracts and introductions of the top 10-15 papers

3. **Build a landscape map**:
   - Group papers by sub-direction / approach
   - Identify what has been tried and what hasn't
   - Note recurring limitations mentioned in "Future Work" sections
   - Flag any open problems explicitly stated by multiple papers

4. **Identify structural gaps**:
   - Methods that work in domain A but haven't been tried in domain B
   - Contradictory findings between papers (opportunity for resolution)
   - Assumptions that everyone makes but nobody has tested
   - Scaling regimes that haven't been explored
   - Diagnostic questions that nobody has asked

### Phase 2: Idea Generation (brainstorm with external LLM)

Use the external LLM via the configured reviewer backend for divergent thinking:

```
model: REVIEWER_MODEL
config: {"model_reasoning_effort": "xhigh"}
prompt: |
  You are a senior ML researcher brainstorming research ideas.

  Research direction: [user's direction]

  Here is the current landscape:
  [paste landscape map from Phase 1]

  Key gaps identified:
  [paste gaps from Phase 1]

  Generate 8-12 concrete research ideas. For each idea:
  1. One-sentence summary
  2. Core hypothesis (what you expect to find and why)
  3. Minimum viable experiment (what's the cheapest way to test this?)
  4. Expected contribution type: empirical finding / new method / theoretical result / diagnostic
  5. Risk level: LOW (likely works) / MEDIUM (50-50) / HIGH (speculative)
  6. Estimated effort: days / weeks / months

  Prioritize ideas that are:
  - Testable with moderate compute (8x RTX 3090 or less)
  - Likely to produce a clear positive OR negative result (both are publishable)
  - Not "apply X to Y" unless the application reveals genuinely surprising insights
  - Differentiated from the 10-15 papers above

  Be creative but grounded. A great idea is one where the answer matters regardless of which way it goes.
```

Save the threadId for follow-up.

### Phase 3: First-Pass Filtering

For each generated idea, quickly evaluate:

1. **Feasibility check**: Can we actually run this experiment with available resources?
   - Compute requirements (estimate GPU-hours)
   - Data availability
   - Implementation complexity
   - Skip ideas requiring > 1 week of GPU time or unavailable datasets

2. **Novelty quick-check**: For each idea, do 2-3 targeted searches to see if it's already been done. Full `/novelty-check` comes later for survivors.

3. **Impact estimation**: Would a reviewer care about the result?
   - "So what?" test: if the experiment succeeds, does it change how people think?
   - Is the finding actionable or just interesting?

Eliminate ideas that fail any of these. Typically 8-12 ideas reduce to 4-6.

### Phase 4: Deep Validation (for top ideas)

For each surviving idea, run a deeper evaluation:

1. **Novelty check**: Use the `/novelty-check` workflow (multi-source search + cross-verification) for each idea

2. **Critical review**: Use the reviewer via follow-up (same thread):
   ```
   Here are our top ideas after filtering:
   [paste surviving ideas with novelty check results]

   For each, play devil's advocate:
   - What's the strongest objection a reviewer would raise?
   - What's the most likely failure mode?
   - How would you rank these for a top venue submission?
   - Which 2-3 would you actually work on?
   ```

3. **Combine rankings**: Merge your assessment with the reviewer's ranking. Select top 2-3 ideas for pilot experiments.

### Phase 5: Parallel Pilot Experiments (for top 2-3 ideas)

Before committing to a full research effort, run cheap pilot experiments to get empirical signal. This is the key differentiator from paper-only validation.

1. **Design pilots**: For each top idea, define the minimal experiment that would give a positive or negative signal:
   - Single seed, small scale (e.g., small dataset subset, fewer epochs)

## Conflicts

- **wanshuiyin/Auto-claude-code-research-in-sleep** sets REVIEWER_MODEL to `gpt-6-astra` and supports REVIEWER_BACKEND of `codex` or `manual` with a cross-model invariant; **brycewang-stanford/Auto-Empirical-Research-Skills** sets REVIEWER_MODEL to `gpt-5.4` and only supports Codex MCP.
- **wanshuiyin/Auto-claude-code-research-in-sleep** includes Phase 0 (load research wiki, threat scanner, query-pack scan) and references /idea-discovery orchestration; **brycewang-stanford/Auto-Empirical-Research-Skills** has no Phase 0 and describes a simpler composition with /research-lit, /novelty-check, /research-review.
- **brycewang-stanford/Auto-Empirical-Research-Skills** provides an explicit Phase 1 landscape-survey procedure and a detailed Phase 2 brainstorm prompt; **wanshuiyin/Auto-claude-code-research-in-sleep** truncates before these details.