domains / agent-tooling / prompt-engineering / senior-prompt-engineer

senior-prompt-engineer

Skill card

combined

The two copies share the same three tool names (prompt_optimizer, rag_evaluator, agent_orchestrator) but disagree on their purpose and usage. The first copy frames the skill as a broad 'Senior AI/ML Engineer' with generic MLOps patterns, Kubernetes deployment, and high-level performance targets, while the second copy provides specific, actionable workflows for prompt optimization, RAG evaluation metrics, and agent flow visualization. Because the first copy lacks the specific evaluation metrics and the second lacks the broader MLOps/deployment context, crowning one would hide the specific tooling details or the broader architectural guidance respectively.

Repositories

Other copies

Combined skill

---
name: senior-prompt-engineer
description: World-class prompt engineering and LLM system design. Use for prompt optimization, RAG evaluation, agent orchestration, and production AI architecture.
---

# Senior Prompt Engineer

This skill covers prompt engineering patterns, LLM evaluation, and agentic system design for production-grade AI systems.

## Quick Start

### 1. Prompt Optimization
Analyze prompts for token efficiency, clarity, and structure.

```bash
# Analyze a prompt file
python scripts/prompt_optimizer.py prompt.txt --analyze

# Generate an optimized version
python scripts/prompt_optimizer.py prompt.txt --optimize --output optimized.txt

# Count tokens for cost estimation
python scripts/prompt_optimizer.py prompt.txt --tokens --model gpt-4
```

**Key Metrics:**
- Token count and estimated cost
- Clarity score (0-100)
- Redundancy detection

### 2. RAG Evaluation
Evaluate Retrieval-Augmented Generation quality.

```bash
# Evaluate retrieval and generation quality
python scripts/rag_evaluator.py --contexts retrieved.json --questions eval_set.json

# Export detailed results
python scripts/rag_evaluator.py --contexts retrieved.json --questions eval_set.json --output report.json --verbose
```

**Key Metrics:**
- Context Relevance (target: >0.80)
- Answer Faithfulness
- Retrieval Precision@5

### 3. Agent Orchestration
Validate and visualize agent workflows.

```bash
# Validate agent configuration
python scripts/agent_orchestrator.py agent.yaml --validate

# Visualize workflow (ASCII or Mermaid)
python scripts/agent_orchestrator.py agent.yaml --visualize
```

**Validation Checks:**
- Tool configuration (API keys, permissions)
- Flow depth and potential infinite loops
- Token estimation per run

## Production Patterns

### Scalable Data Processing
- Horizontal scaling architecture
- Fault-tolerant design
- Real-time and batch processing

### ML Model Deployment
- Model serving with low latency
- A/B testing infrastructure
- Model monitoring and drift detection

### Real-Time Inference
- Batching and caching strategies
- Load balancing and auto-scaling
- Latency optimization

## Best Practices

### Development
- Test-driven development
- Documentation as code
- Continuous integration

### Production
- Monitor everything critical
- Feature flags for releases
- Canary deployments

## Performance Targets

**Latency:**
- P50: < 50ms
- P95: < 100ms
- P99: < 200ms

**Availability:**
- Uptime: 99.9%
- Error rate: < 0.1%

## Security & Compliance

- Authentication & authorization
- Data encryption (at rest & in transit)
- PII handling and anonymization
- GDPR/CCPA compliance

## Resources

- Advanced Patterns: `references/prompt_engineering_patterns.md`
- Evaluation Frameworks: `references/llm_evaluation_frameworks.md`
- Agentic Design: `references/agentic_system_design.md`