domains / agent-tooling / skill-optimization / skill-optimizer

skill-optimizer

Skill card

combined

The two copies describe different procedures for the same skill name. mcollina/skills defines a forward-looking optimization loop (baseline, failure pattern, salience edits, re-run, ship) and points to rule files. sickn33/agentic-awesome-skills defines a read-only diagnostic workflow that scans session JSONL files, runs 8 analysis dimensions, and outputs a P0/P1/P2 report. They overlap in the goal of improving skill activation and context cost, but each contains material steps the other lacks, and they disagree on whether the skill should modify files or only report.

Repositories

Other copies

Combined skill

---
name: skill-optimizer
description: "Optimizes AI skills for activation, clarity, and cross-model reliability. Use when creating or editing skill packs, diagnosing weak skill uptake, reducing regressions, tuning instruction salience, improving examples, shrinking context cost, or setting benchmark/release gates for skills. Trigger terms: skill optimization, activation gap, benchmark skill, with/without skill delta, regression, context budget, prompt salience."
metadata:
  tags: skills, optimization, benchmarking, activation, regressions, prompt-engineering
---

## When to use

Use this skill when you need to:
- Improve whether a skill is actually applied by models
- Diagnose why some criteria fail across all models
- Prevent a skill from making outputs worse
- Refactor skill text for stronger retrieval under context pressure
- Build repeatable benchmark loops and release gates
- Audit and improve your skill library's quality using session data

## Optimization loop (default workflow)

1. **Measure baseline and skill-on behavior** (per model, per scenario, per criterion)
2. **Find failure pattern**:
   - universal failure (0% with skill)
   - model-specific weakness
   - regression (negative delta)
3. **Edit for salience**:
   - add explicit triggers
   - add concrete integrated examples
   - tighten checklists and decision rules
4. **Re-run evals** and compare deltas
5. **Ship with guardrails** (documented gate + run history + follow-up issues)

## Diagnostic workflow (read-only)

Use this workflow when you need to audit existing skills using session data without modifying files.

### Step 1: Identify Target Skills

Scan skill directories in order: `~/.claude/skills/`, `~/.codex/skills/`, `~/.agents/skills/`. Deduplicate by skill name. For each, read `SKILL.md` and extract:
- name, description (from YAML frontmatter)
- trigger keywords (from description field)
- defined workflow steps
- word count

### Step 2: Collect Session Data

Use python3 scripts via Bash to scan session JSONL files. Extract:

**Claude Code sessions** (`~/.claude/projects/**/*.jsonl`):
- `Skill` tool_use calls
- User messages
- Assistant messages after skill invocation
- User messages after skill invocation

**Codex sessions** (`~/.codex/sessions/**/*.jsonl`):
- `session_meta` events for skill loading evidence
- `response_item` events for assistant outputs
- `event_msg` events for tool execution
- User messages from `turn_context` events

**Note:** Codex injects skills via context rather than explicit `Skill` tool calls. Skill loading does NOT equal active invocation. To detect actual use, search for skill-specific workflow markers in `response_item` content.

### Step 3: Run 8 Analysis Dimensions

**You MUST run ALL 8 dimensions.**

#### 4.1 Trigger Rate
Count how many times each skill was actually invoked vs how many times its trigger keywords appeared in user messages.

#### 4.2 Post-Invocation User Reaction
After a skill is invoked, read the user's next 3 messages. Classify as Negative, Correction, Positive, or Silent switch. Report per-skill satisfaction rate.

#### 4.3 Workflow Completion Rate
For each skill invocation, extract the skill's defined steps and search assistant messages for step markers.

#### 4.4-4.8
Run remaining dimensions as defined in the source skill.

### Step 4: Compute Composite Scores

Score each skill on a 5-point composite scale across 8 dimensions.

### Step 5: Output Report

Output a diagnostic report with P0/P1/P2 prioritized fixes.

## Practical heuristics

- Prefer **few high-signal rules** over many soft recommendations
- Put fragile, high-value behaviors in **top-level checklists**
- Include at least one **integrated example** per common scenario
- Add explicit wording for what must **not** be omitted
- Track gains/losses with **with-skill vs without-skill** comparisons
- **Read-only**: never modify skill files during diagnostic workflow. Only output report.
- **Quantify**: use specific numbers rather than vague claims
- **Suggest, don't prescribe**: give specific wording suggestions but frame as suggestions
- **Show evidence**: quote actual user messages that should have triggered the skill
- **Evidence-based suggestions**: cite specific research findings that motivate changes

## Conflicts

- mcollina/skills instructs editing skill text for salience and shipping with guardrails, while sickn33/agentic-awesome-skills is read-only and only outputs a report
- mcollina/skills uses a 5-step optimization loop, while sickn33/agentic-awesome-skills uses an 8-dimension diagnostic report with P0/P1/P2 prioritization
- sickn33/agentic-awesome-skills requires scanning session JSONL files for evidence, while mcollina/skills does not mention session data collection